Virtual reality display method and system

Abstract

The embodiment of the present application relates to the application field of virtual reality technology, and discloses a virtual reality display method and system. The virtual reality display method of the present application determines whether the display device on the corresponding A-pillar can meet the use requirements of the driver user according to the line of sight direction of the vehicle driver when judging that the vehicle is approaching the top of the slope. When the display device on the corresponding A-pillar cannot meet the use requirements of the driver, it means that the A-pillar blind spot scene cannot be displayed through the display device. At this time, the virtual reality technology is used to display the virtual three-dimensional image of the A-pillar blind spot scene at the best viewing angle. When it is judged that the vehicle is at the bottom of the slope and is preparing to go uphill, the virtual three-dimensional model of the vehicle is displayed just above the top of the slope through the virtual reality technology. In this way, the vehicle driver's needs for safety warnings can be met, and the technical defect that the vehicle cannot provide effective warnings in time due to the existence of the A-pillar blind spot during the uphill process can be compensated.

Description

Technical Field

[0001] The present invention relates to the application field of virtual reality technology, and in particular to a virtual reality display method and system. Background Art

[0002] The A-pillar refers to the pillar between the front windshield and the front door of the vehicle body. The A-pillar makes the vehicle body more stable and rigid, and plays an important role in protecting the driving safety of the driver and passengers. At the same time, due to the existence of the A-pillar, the A-pillar visual blind spot appears, which poses a safety hazard to vehicle driving. At present, the perspective effect is usually achieved by imaging the scene outside the vehicle on the A-pillar display screen, thereby expanding the driver's field of vision. However, when the vehicle is driving uphill, the blind spot formed by the A-pillar is larger than that when driving on a flat road.

[0003] During the uphill process, although the scene outside the vehicle is imaged on the A-pillar display screen, the A-pillar blind spot perspective effect is poor, especially when the vehicle is at the bottom of the slope, the camera can only capture a limited range of blind spot images, thereby failing to provide timely safety warnings to its own vehicle and other vehicles. Summary of the invention

[0004] The main purpose of the present invention is to provide a virtual reality display method and system, aiming to solve the technical problem in the prior art that the A-pillar perspective display effect is poor and cannot provide timely safety warnings to the own vehicle and other vehicles.

[0005] To achieve the above-mentioned purpose, in a first aspect, a virtual reality display method is provided in an embodiment of the present application, which is applied to a vehicle virtual reality display system, wherein the vehicle virtual reality display system includes a vehicle in an uphill state and a roadside sensor disposed at the top of the slope, wherein the vehicle includes a first A-pillar, a second A-pillar, a first display device disposed on the first A-pillar, and a second display device disposed on the second A-pillar, wherein both the first display device and the second display device have adjustable states, and the method includes:

[0006] Acquiring a distance parameter between the vehicle and the roadside sensor during an uphill process;

[0007] When the distance between the vehicle and the roadside sensor is less than or equal to a preset threshold, obtaining the sight direction of the driver of the vehicle;

[0008] Determine a target display device from the first display device and the second display device according to the sight direction of the vehicle driver, wherein the target display device is a display device on the A-pillar corresponding to the sight direction of the driver;

[0009] Determining whether the target display device is at the driver's optimal viewing angle position or whether it can be adjusted to the driver's optimal viewing angle position according to the sight direction of the vehicle driver, wherein the plane where the optimal viewing angle position is located is parallel to the display interface of the target display device;

[0010] It is determined that the target display device is not at the driver's optimal viewing position and cannot be adjusted to the driver's optimal viewing position, and a virtual three-dimensional image of the A-pillar blind spot scene is displayed at the optimal viewing position through virtual reality technology.

[0011] Furthermore, the method also includes: when the distance value between the vehicle and the roadside sensor is greater than a preset threshold, displaying a virtual three-dimensional model of the vehicle directly above the top of the slope through virtual reality technology.

[0012] Furthermore, when the distance between the vehicle and the roadside sensor is less than or equal to a preset threshold, the method further includes:

[0013] Obtaining the sight visibility of the vehicle during the uphill process, wherein the sight visibility is used to represent the farthest distance that the vehicle can clearly see during the uphill process;

[0014] Determine that the distance value between the vehicle and the roadside sensor is greater than or equal to the line of sight visibility, and display a virtual three-dimensional model of the vehicle directly above the top of the slope through virtual reality technology.

[0015] Furthermore, the vehicle virtual reality display system further includes a warning receiver disposed on the top of the slope. When the distance value between the vehicle and the roadside sensor is less than or equal to a preset threshold, the method further includes:

[0016] Obtaining the strength of the warning signal received by the warning receiver;

[0017] It is determined that the strength of the warning signal received by the warning receiver is less than or equal to a signal strength threshold, and a virtual three-dimensional model of the vehicle is displayed directly above the top of the slope through virtual reality technology.

[0018] Furthermore, the vehicle further includes a first camera device and a second camera device provided on a rearview mirror, and the virtual three-dimensional image of the A-pillar blind spot scene is displayed at the optimal viewing angle position by using virtual reality technology, including:

[0019] Determine a target camera device from the first camera device and the second camera device according to the sight direction of the vehicle driver, wherein the target camera device is a camera device on the rearview mirror corresponding to the sight direction of the driver;

[0020] Adjusting the shooting angle of the target camera device according to the sight direction of the vehicle driver to obtain a target road image of the A-pillar blind spot scene;

[0021] The target road image is subjected to virtual reality processing to display a virtual three-dimensional image of the A-pillar blind spot scene at the optimal viewing angle, wherein the virtual three-dimensional image is a virtual three-dimensional image of the target road image.

[0022] Furthermore, after determining whether the target display device is at the optimal viewing angle of the driver or whether it can be adjusted to the optimal viewing angle of the driver, the method further includes:

[0023] Determine the optimal viewing angle position of the target display device at the driver, and control the target display device to display the A-pillar blind spot scene image; or,

[0024] Determine that the target display device is not at the driver's optimal viewing angle position but can be adjusted to the driver's optimal viewing angle position, control the target display device to move to the optimal viewing angle position and display the A-pillar blind spot scene image.

[0025] Furthermore, the vehicle further includes a first camera device and a second camera device provided on a rearview mirror, and the control of the target display device to move to the optimal viewing angle position and display the A-pillar blind spot scene image includes:

[0026] Determine a target camera device from the first camera device and the second camera device according to the sight direction of the vehicle driver, wherein the target camera device is a camera device on the rearview mirror corresponding to the sight direction of the driver;

[0027] Adjusting the shooting angle of the target camera device according to the sight direction of the vehicle driver to obtain a target road image of the A-pillar blind spot scene;

[0028] The target display device is controlled to move to the optimal viewing angle position and the target road image is displayed on the target display device.

[0029] Furthermore, obtaining the sight direction of the vehicle driver includes:

[0030] Obtaining the eye position and head posture of the driver of the vehicle;

[0031] The sight direction of the driver is obtained according to the eye position and the head posture.

[0032] Further, judging whether the target display device is at the driver's optimal viewing angle position or whether it can be adjusted to the driver's optimal viewing angle position according to the sight direction of the vehicle driver includes:

[0033] Acquire the eye viewing angle range pre-entered by the vehicle driver, wherein the eye viewing angle range includes a left-right viewing angle range or a top-bottom viewing angle range;

[0034] Determining an optimal viewing angle range according to the sight direction and eye viewing angle range of the vehicle driver;

[0035] When the target display device is within the optimal viewing angle range or can be completely moved into the optimal viewing angle range, it is determined that the target display device is at the optimal viewing angle position for the driver or can be adjusted to the optimal viewing angle position for the driver.

[0036] In a second aspect, an embodiment of the present application further provides a virtual reality display system, comprising a memory and a processor, wherein the memory is used to store program code, and the processor is used to call the program code to execute the method described in the first aspect.

[0037] Different from the prior art, the embodiment of the present application provides a virtual reality display method, which first obtains the distance parameters between the vehicle and the roadside sensor during the uphill process; when the distance value between the vehicle and the roadside sensor is less than or equal to a preset threshold, obtains the line of sight direction of the vehicle driver; then determines the target display device from the first display device and the second display device according to the line of sight direction of the vehicle driver; then determines whether the target display device is at the driver's optimal viewing position or whether it can be adjusted to the driver's optimal viewing position according to the line of sight direction of the vehicle driver; when it is determined that the target display device is not at the driver's optimal viewing position and cannot be adjusted to the driver's optimal viewing position, a virtual three-dimensional image of the A-pillar blind spot scene is displayed at the optimal viewing position through virtual reality technology. That is, when it is determined that the vehicle is approaching the top of the slope, it is determined whether the display device on the corresponding A-pillar can meet the driver's usage needs based on the driver's line of sight. When the display device on the corresponding A-pillar cannot meet the driver's usage needs, it means that the A-pillar blind spot scene cannot be displayed through the display device. At this time, virtual reality technology is used to display a virtual three-dimensional image of the A-pillar blind spot scene at the best viewing angle. In this way, the vehicle driver's needs for safety warnings can be met, and the technical defect that the vehicle cannot provide effective warnings in time due to the existence of the A-pillar blind spot during the uphill process can be compensated. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0039] Figure 1A schematic diagram of an application environment of a virtual reality display method in some embodiments of the present application;

[0040] Figure 2 A schematic diagram of an application environment of a virtual reality display method in some other embodiments of the present application;

[0041] Figure 3 This is a flowchart of a virtual reality display method in some embodiments of the present application;

[0042] Figure 4 A flowchart of a virtual reality display method in some other embodiments of the present application;

[0043] Figure 5 A flowchart of a virtual reality display method in some other embodiments of the present application;

[0044] Figure 6 This is a flowchart of a virtual reality display method in some embodiments of the present application;

[0045] Figure 7 This is a schematic diagram of the hardware structure of a virtual reality display system in some embodiments of the present application.

[0046] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0049] In addition, the descriptions of "first", "second", etc. in the present invention are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, "and / or" in the full text includes three solutions. Taking A and / or B as an example, it includes technical solution A, technical solution B, and technical solution that satisfies both A and B. In addition, the technical solutions between the various embodiments can be combined with each other, but it must be based on the ability of ordinary technicians in this field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0050] The A-pillar refers to the pillar between the front windshield and the front door of the vehicle body. The A-pillar makes the vehicle body more stable and rigid, and plays an important role in protecting the driving safety of the driver and passengers. At the same time, due to the existence of the A-pillar, the A-pillar visual blind spot appears, which poses a safety hazard to vehicle driving. At present, the perspective effect is usually achieved by imaging the scene outside the vehicle on the A-pillar display screen, thereby expanding the driver's field of vision. However, when the vehicle is driving uphill, the blind spot formed by the A-pillar is larger than that when driving on a flat road.

[0051] During the uphill process, although the scene outside the vehicle is imaged on the A-pillar display screen, the A-pillar blind spot perspective effect is poor, especially when the vehicle is at the bottom of the slope, the camera can only capture a limited range of blind spot images, thereby failing to provide timely safety warnings to its own vehicle and other vehicles.

[0052] In view of the above problems, the present application proposes a virtual reality display method, which is applied to a vehicle virtual reality display system. The vehicle virtual reality display system includes a vehicle 100 in an uphill state and a roadside sensor 200 arranged at the top of the slope. The vehicle 100 includes a first A-pillar 101, a second A-pillar 102, a first display device arranged on the first A-pillar 101, and a second display device arranged on the second A-pillar 102. The first display device and the second display device are both set to an adjustable state (adjustable left and right and up and down) to adjust the position of the display device according to user needs, but the movement adjustment of the first display device and the second display device is limited to a certain extent. The movement of the display device is affected by the structure inside the vehicle and the function of the rearview mirror. For example, when the display device moves to a position close to the rearview mirror, it will affect the driver's observation of the rearview mirror; therefore, the first display device and the second display device both have a corresponding adjustment range, but the movement of the display device within the adjustment range does not affect the display of the A-pillar blind spot scene nor the functions of other vehicle components (such as the rearview mirror). When the adjustment range is exceeded, the functions of other vehicle components will be affected.

[0053] The overall idea of ​​the virtual reality display method of the present application is: when the vehicle is driving to a position close to the top of the slope, since the driver can observe the scene of the A-pillar blind spot through the external camera at this time, the image of the A-pillar blind spot is first displayed through the target display device. When the target display device cannot meet the driver's user needs, the virtual reality technology is used to display the virtual three-dimensional image of the A-pillar blind spot scene at the driver's best viewing angle, thereby providing the driver with an effective A-pillar blind spot scene display, so as to facilitate the driver to make safety warnings during the vehicle going uphill; and when the vehicle is driving at the bottom of the slope and preparing to go uphill, due to the large occlusion range of the A-pillar blind spot, the external camera cannot capture other vehicles preparing to go downhill and other road conditions. At this time, the virtual reality technology is used to display the virtual three-dimensional model of the vehicle just above the top of the slope, thereby providing safety warnings to other vehicles preparing to go downhill.

[0054] like Figure 1-6 As shown, the specific steps of the virtual reality display method will be mainly described below. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described here can be performed in a different order. Figure 1 , the method comprises the following steps:

[0055] S100, obtaining distance parameters between the vehicle and the roadside sensor during the uphill process;

[0056] The roadside sensor can be an electromagnetic wave transceiver, which is used to receive electromagnetic waves sent by the vehicle and resend the electromagnetic waves to the vehicle. Therefore, the distance parameter between the vehicle and the roadside sensor during the uphill process can be calculated based on the time required for the vehicle to send and receive electromagnetic waves and the propagation speed of the electromagnetic waves.

[0057] S200, when the distance between the vehicle and the roadside sensor is less than or equal to a preset threshold, obtaining the sight direction of the vehicle driver;

[0058] It can be understood that when the distance between the vehicle and the roadside sensor is less than or equal to the preset threshold (the preset value can be set to 1 / 10 of the slope length), it means that the vehicle is close to the top of the slope. At this time, the external camera installed in the rearview mirror can capture the A-pillar blind spot scene, especially the Figure 2 The vehicle situation at the A position in the middle, therefore, can be displayed in the blind spot scene through the display device corresponding to the A-pillar, so that the driver can be warned in advance to make a risk warning, such as honking the horn to remind other vehicles to pass carefully. If you want to display the blind spot scene through the display device on the A-pillar, you first need to obtain the direction of the driver's line of sight, so as to adjust the camera's shooting angle and adjust the position of the display device according to the line of sight to meet the needs of different users.

[0059] In one embodiment, the step: obtaining the sight line direction of the vehicle driver includes: S210, obtaining the eye position and head posture of the vehicle driver;

[0060] S220: Obtain the driver's sight direction according to the eye position and head posture.

[0061] In some embodiments, the vehicle may include a DMS eye tracking capture system, which can obtain the driver's eye coordinates and head posture, and determine whether the driver is looking at the first A-pillar or the second A-pillar based on the eye position and head posture.

[0062] In some embodiments, the processor in the vehicle can also obtain a more specific gaze direction of the driver according to the eye position and head posture based on a preset gaze tracking algorithm or model, such as a DCF algorithm, a KCF algorithm, a LCT algorithm, an MDNet algorithm, a convolutional neural network, a convolutional regression network, etc.

[0063] Exemplarily, after the vehicle is started, the camera inside the vehicle obtains the three-dimensional coordinates of the eyebrow position based on the driver's facial features, with the x-axis being the left-right coordinate, the y-axis being the high-low coordinate, and the z-axis being the front-back coordinate. The x-axis coordinate, y-axis and z-axis coordinates of the eyebrow center are calculated based on the median of the x-axis coordinates of the eyes, and the three-dimensional coordinates are transmitted to the processor. The system gives a standard three-dimensional coordinate of the eyebrow center when the driver is driving as a reference and a reference line of sight under the three-dimensional coordinate of the eyebrow center. When the driver turns his head, the system obtains the actual three-dimensional coordinate of the eyebrow center and obtains the driver's specific line of sight direction based on the reference three-dimensional coordinate of the eyebrow center and the reference line of sight, that is, the angle between the real-time line of sight and the reference line of sight.

[0064] S300, determining a target display device from the first display device and the second display device according to the sight direction of the vehicle driver, wherein the target display device is a display device on the A-pillar corresponding to the sight direction of the driver;

[0065] It can be understood that when the vehicle driver's line of sight is directed towards the first A-pillar, it means that the vehicle driver wants to know the blind spot of the first A-pillar. At this time, the first display device is used as the target display device to display the A-pillar blind spot. The position of the first display device can be adjusted when necessary.

[0066] Similarly, when the vehicle driver's line of sight is directed toward the second A-pillar, it means that the vehicle driver wants to know the blind spot of the second A-pillar. At this time, the second display device is used as the target display device to display the A-pillar blind spot. The position of the second display device can be adjusted when necessary.

[0067] S400, judging whether the target display device is at the driver's optimal viewing angle position or whether it can be adjusted to the driver's optimal viewing angle position according to the sight direction of the vehicle driver, wherein the plane where the optimal viewing angle position is located is parallel to the display interface of the target display device;

[0068] It should be noted that each user's eye field of vision is different. The eye field of vision is divided into left and right viewing angles and up and down viewing angles. For a normal user, the left and right viewing angles can reach 150-180°, that is, when the user looks forward, he can see the scene within the range of 75-90° on the left and 75-90° on the right at the same time; for drivers with eye diseases, the eye field of vision is greatly reduced. For users with severe eye diseases, the left and right viewing angles of the eye are even only 30-40°. Therefore, for normal drivers, the optimal viewing angle is larger (150-180°), and for abnormal drivers, the optimal viewing angle is smaller (30-40°).

[0069] The optimal viewing angle position refers to the viewing angle position that can meet the vehicle driver's eye field range and the driver does not feel fatigued when observing. Therefore, the optimal viewing angle position is a concept of viewing angle range.

[0070] In one embodiment, step S400: determining whether the target display device is at the driver's optimal viewing angle according to the vehicle driver's sight direction includes:

[0071] S410, obtaining an eye viewing angle range pre-entered by the vehicle driver, wherein the eye viewing angle range includes a left-right viewing angle range or a top-bottom viewing angle range;

[0072] S420, determining an optimal viewing angle range according to the sight direction and eye viewing angle range of the vehicle driver;

[0073] S430, when the target display device is within the optimal viewing angle range, determining that the target display device is at an optimal viewing angle position for the driver;

[0074] S340: When the target display device is not within the optimal viewing angle range, determine that the target display device is not at the optimal viewing angle position for the driver.

[0075] In the embodiment of the present application, the vehicle driver can first input his own eye viewing angle range in advance, and then the system determines the optimal viewing angle range according to the vehicle driver's line of sight direction and eye viewing angle range. The optimal viewing angle range is the angle where the driver's line of sight extends to the left and right sides by half of the driver's eye viewing angle range. After obtaining the driver's optimal viewing angle range, it can be determined whether the initial position of the target display device is in the driver's optimal viewing angle position, that is, when the initial position of the target display device is in the driver's optimal viewing angle range, the driver can fully observe the target display device at this time, and it is determined that the target display device is in the driver's optimal viewing angle position. When the initial position of the target display device is not in the driver's optimal viewing angle range, the driver cannot fully observe the target display device at this time, and it is determined that the target display device is not in the driver's optimal viewing angle position. In this way, by pre-collecting the eye viewing angle ranges of different drivers, it is possible to accurately determine whether the target display device is in the driver's optimal viewing angle range.

[0076] S500: Determine that the target display device is not at the optimal viewing position for the driver and cannot be adjusted to the optimal viewing position for the driver, and display a virtual three-dimensional image of the A-pillar blind spot scene at the optimal viewing position through virtual reality technology.

[0077] As mentioned above, the position adjustment of the target display device is affected by many factors. Therefore, when the target display device is not at the driver's optimal viewing angle and cannot be adjusted to the driver's optimal viewing angle, it means that the A-pillar blind spot cannot be displayed through the display device at this time; therefore, in order to avoid the driving risks caused by the existence of the A-pillar blind spot, a virtual three-dimensional image of the A-pillar blind spot scene is displayed at the optimal viewing angle through virtual reality technology (such as Figure 2 As shown, 300 is a virtual three-dimensional image), to make up for the defect that the display device cannot meet the driver's usage needs, thereby improving the display effect of the A-pillar blind spot.

[0078] In one embodiment, the vehicle further includes a first camera device and a second camera device provided on a rearview mirror, the first camera device is used to capture an image of a first A-pillar blind spot, and the second camera device is used to capture an image of a second A-pillar blind spot. The step of displaying a virtual three-dimensional image of an A-pillar blind spot scene at an optimal viewing angle position by using virtual reality technology includes:

[0079] Determine a target camera device from the first camera device and the second camera device according to the sight direction of the vehicle driver, wherein the target camera device is the camera device on the rearview mirror corresponding to the sight direction of the driver;

[0080] Adjusting the shooting angle of the target camera device according to the sight direction of the vehicle driver to obtain a target road image of the A-pillar blind spot scene;

[0081] The target road image is subjected to virtual reality processing to display a virtual three-dimensional image of the A-pillar blind spot scene at an optimal viewing angle, wherein the virtual three-dimensional image is a virtual three-dimensional image of the target road image.

[0082] Specifically, the virtual three-dimensional image of the A-pillar blind spot scene is obtained by accurately photographing the target road image with a camera device and then performing virtual reality processing; and the shooting angle of the camera device can be adjusted according to the direction of the driver's line of sight. In this way, when the movement range of the target display device is limited and the A-pillar blind spot scene cannot be effectively displayed on the in-vehicle display screen, the perspective display of the A-pillar blind spot scene is performed through virtual reality technology, which can not only ensure the perspective display effect of the A-pillar, but also will not block the driver's observation line of sight of the rearview mirror, thereby improving the display effect of the A-pillar blind spot.

[0083] In other embodiments, when it is determined that the target display device is at the driver's optimal viewing position, the target display device is controlled to display the A-pillar blind spot scene image; or, when it is determined that the target display device is not at the driver's optimal viewing position but can be adjusted to the driver's optimal viewing position, the target display device is controlled to move to the optimal viewing position and display the A-pillar blind spot scene image.

[0084] In one embodiment, controlling the target display device to move to the optimal viewing angle position and displaying the A-pillar blind spot scene image includes:

[0085] Determine a target camera device from the first camera device and the second camera device according to the sight direction of the vehicle driver, wherein the target camera device is the camera device on the rearview mirror corresponding to the sight direction of the driver;

[0086] Adjusting the shooting angle of the target camera device according to the sight direction of the vehicle driver to obtain a target road image of the A-pillar blind spot scene;

[0087] The target display device is controlled to move to the best viewing angle position and the target road image is displayed on the target display device.

[0088] Specifically, when the initial position of the target display device is at the driver's optimal viewing angle position, the A-pillar blind spot display can be performed through the target display device; when the initial position of the target display device is not at the driver's optimal viewing angle position but can be adjusted to the driver's optimal viewing angle position, the corresponding target display device is controlled to move to the optimal viewing angle position and display the A-pillar blind spot scene image. The shooting angle of the camera device can be adjusted according to the line of sight of the vehicle driver to obtain the A-pillar blind spot scene image with accurate orientation and display it on the corresponding target display device. In this way, it can adapt to the usage needs of different driver users.

[0089] It can be understood that when the vehicle is just about to go uphill and is at the bottom of the slope ( Figure 2Due to the obstruction of the slope, the vehicle cannot be photographed by the external camera. Figure 2 The vehicle condition at position A in the middle, so that the vehicle cannot send out a warning signal for safe driving in advance. Therefore, in one embodiment, the virtual reality display method also includes:

[0090] S600: When the distance between the vehicle and the roadside sensor is greater than a preset threshold, display a virtual three-dimensional model of the vehicle directly above the top of the slope using virtual reality technology, that is, send a virtual three-dimensional model directly above the top of the slope using virtual reality technology. Figure 2 A warning signal (shown as 300 in the figure) is provided to remind other vehicles that there is a vehicle going uphill on the slope, so please pay attention to safety.

[0091] In other embodiments, when the vehicle is close to the top of the slope, but due to external factors the vehicle is unable to provide an effective warning signal or is unable to provide a warning signal in a timely manner, virtual reality technology can also be used to display a virtual three-dimensional model of the vehicle directly above the top of the slope, thereby providing a more direct warning signal to remind other vehicles that there is a vehicle going uphill on the slope, so please pay attention to safety.

[0092] In one embodiment, the virtual reality display method further includes:

[0093] S700: When the distance between the vehicle and the roadside sensor is less than or equal to a preset threshold, obtain the sight visibility of the vehicle during the uphill process, where the sight visibility is used to represent the farthest distance that the vehicle can clearly see during the uphill process;

[0094] S800: Determine whether the distance between the vehicle and the roadside sensor is greater than or equal to the line of sight visibility, and display a virtual three-dimensional model of the vehicle directly above the top of the slope through virtual reality technology.

[0095] Specifically, when the visibility of the vehicle is low during the uphill process, the camera shooting is unclear and the A-pillar blind spot cannot be effectively displayed, which may cause the driver to be unable to provide a warning signal in time. At this time, virtual reality technology is used to display a virtual three-dimensional model of the vehicle just above the top of the slope, thereby providing a more direct warning signal to remind other vehicles that there is a vehicle going uphill on the slope, so please pay attention to safety.

[0096] In another embodiment, the vehicle virtual reality display system further includes a warning receiver disposed on the top of the slope, and the virtual reality display method further includes:

[0097] S900, when the distance between the vehicle and the roadside sensor is less than or equal to a preset threshold, obtaining the strength of the warning signal received by the warning receiver;

[0098] S1000: Determine whether the strength of the warning signal received by the warning receiver is less than or equal to a signal strength threshold, and display a virtual three-dimensional model of the vehicle directly above the top of the slope through virtual reality technology.

[0099] Specifically, when the warning signal (sound signal or light signal) received at the top of the slope is affected, resulting in the vehicle being unable to provide or receive an effective warning signal, virtual reality technology can be used to display a virtual three-dimensional model of the vehicle directly above the top of the slope, thereby providing a more direct warning signal to remind other vehicles that there is a vehicle going uphill on the slope, so please pay attention to safety.

[0100] Based on this, when judging that the vehicle is approaching the top of the slope, the present application judges whether the display device on the corresponding A-pillar can meet the usage needs of the driver-user according to the line of sight of the vehicle driver. When the display device on the corresponding A-pillar cannot meet the usage needs of the driver, it means that the A-pillar blind spot scene cannot be displayed through the display device. At this time, the virtual reality technology is used to display the virtual three-dimensional image of the A-pillar blind spot scene at the best viewing angle. When judging that the vehicle is at the bottom of the slope and preparing to go uphill, the virtual three-dimensional model of the vehicle is displayed just above the top of the slope through the virtual reality technology. In this way, the vehicle driver's needs for safety warnings can be met, and the technical defect that the vehicle cannot provide effective warnings in time due to the existence of the A-pillar blind spot during the uphill process can be compensated.

[0101] The present application also provides a virtual reality display system. Figure 7 , Figure 7 A schematic diagram of the hardware structure of a virtual reality display system provided in some embodiments of the present application; the virtual reality display system includes a memory 410 and a processor 420, the memory 410 is used to store program codes, and the processor 420 is used to call program codes to execute the above method.

[0102] The processor 420 is used to provide computing and control capabilities to control the virtual reality display system to perform corresponding tasks, for example, to control the virtual reality display system to perform the virtual reality display method in any of the above method embodiments, the method comprising: obtaining a distance parameter between the vehicle and a roadside sensor during an uphill process; when the distance value between the vehicle and the roadside sensor is less than or equal to a preset threshold, obtaining the line of sight direction of the vehicle driver; determining a target display device from the first display device and the second display device according to the line of sight direction of the vehicle driver, wherein the target display device is a display device on the A-pillar corresponding to the driver's line of sight direction; judging whether the target display device is at the driver's optimal viewing angle position or can be adjusted to the driver's optimal viewing angle position according to the vehicle driver's line of sight direction, wherein the plane where the optimal viewing angle position is located is parallel to the display interface of the target display device; determining that the target display device is not at the driver's optimal viewing angle position and cannot be adjusted to the driver's optimal viewing angle position, and displaying a virtual three-dimensional image of the A-pillar blind spot scene at the optimal viewing angle position through virtual reality technology.

[0103] The processor 420 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The above-mentioned PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0104] The memory 410 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the virtual reality display method in the embodiments of the present application. The processor 420 can implement the virtual reality display method in any of the above method embodiments by running the non-transitory software programs, instructions and modules stored in the memory 410.

[0105] Specifically, the memory 410 may include a volatile memory (VM), such as a random access memory (RAM); the memory 110 may also include a non-volatile memory (NVM), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD) or other non-transitory solid-state storage device; the memory 410 may also include a combination of the above types of memory.

[0106] In summary, the virtual reality display system of the present application adopts the technical solution of any one of the above-mentioned virtual reality display method embodiments, and therefore, has at least the beneficial effects brought by the technical solution of the above-mentioned embodiments, which will not be described one by one here.

[0107] The embodiment of the present application also provides a computer-readable storage medium, such as a memory including a program code, and the program code can be executed by a processor to complete the virtual reality display method in the above embodiment. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CDROM), a magnetic tape, a floppy disk, and an optical data storage device.

[0108] The present application also provides a computer program product, which includes one or more program codes stored in a computer-readable storage medium. A processor of an electronic device reads the program code from the computer-readable storage medium, and the processor executes the program code to complete the steps of the virtual reality display method provided in the above embodiment.

[0109] A person skilled in the art will appreciate that all or part of the steps for implementing the above embodiments may be accomplished by hardware or by hardware associated with a program code, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0110] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0111] Through the description of the above implementation methods, ordinary technicians in this field can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Ordinary technicians in this field can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0112] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A virtual reality display method, applied to a vehicle virtual reality display system, characterized in that: The vehicle virtual reality display system includes a vehicle in an uphill state and a roadside sensor arranged at the top of the slope, the vehicle includes a first A-pillar, a second A-pillar, a first display device arranged at the first A-pillar, and a second display device arranged at the second A-pillar, the first display device and the second display device both have adjustable states, and the method includes: Acquiring a distance parameter between the vehicle and the roadside sensor during an uphill process; When the distance between the vehicle and the roadside sensor is less than or equal to a preset threshold, obtaining the sight direction of the driver of the vehicle; Determine a target display device from the first display device and the second display device according to the sight direction of the vehicle driver, wherein the target display device is a display device on the A-pillar corresponding to the sight direction of the driver; Determining whether the target display device is at the driver's optimal viewing angle position or whether it can be adjusted to the driver's optimal viewing angle position according to the sight direction of the vehicle driver, wherein the plane where the optimal viewing angle position is located is parallel to the display interface of the target display device; It is determined that the target display device is not at the driver's optimal viewing position and cannot be adjusted to the driver's optimal viewing position, and a virtual three-dimensional image of the A-pillar blind spot scene is displayed at the optimal viewing position through virtual reality technology.

2. The virtual reality display method according to claim 1, characterized in that: The method further includes: when the distance value between the vehicle and the roadside sensor is greater than a preset threshold, displaying a virtual three-dimensional model of the vehicle directly above the top of the slope by using virtual reality technology.

3. The virtual reality display method according to claim 1, characterized in that: When the distance between the vehicle and the roadside sensor is less than or equal to a preset threshold, the method further includes: Obtaining the sight visibility of the vehicle during the uphill process, wherein the sight visibility is used to represent the farthest distance that the vehicle can clearly see during the uphill process; Determine that the distance value between the vehicle and the roadside sensor is greater than or equal to the line of sight visibility, and display a virtual three-dimensional model of the vehicle directly above the top of the slope through virtual reality technology.

4. The virtual reality display method according to claim 1, characterized in that: The vehicle virtual reality display system further includes a warning receiver disposed on the top of the slope. When the distance value between the vehicle and the roadside sensor is less than or equal to a preset threshold, the method further includes: Obtaining the strength of the warning signal received by the warning receiver; It is determined that the strength of the warning signal received by the warning receiver is less than or equal to a signal strength threshold, and a virtual three-dimensional model of the vehicle is displayed directly above the top of the slope through virtual reality technology.

5. The virtual reality display method according to claim 1, characterized in that: The vehicle further includes a first camera device and a second camera device disposed on a rearview mirror, and the virtual three-dimensional image of the A-pillar blind spot scene is displayed at the optimal viewing angle position by using virtual reality technology, including: Determine a target camera device from the first camera device and the second camera device according to the sight direction of the vehicle driver, wherein the target camera device is a camera device on the rearview mirror corresponding to the sight direction of the driver; Adjusting the shooting angle of the target camera device according to the sight direction of the vehicle driver to obtain a target road image of the A-pillar blind spot scene; The target road image is subjected to virtual reality processing to display a virtual three-dimensional image of the A-pillar blind spot scene at the optimal viewing angle, wherein the virtual three-dimensional image is a virtual three-dimensional image of the target road image.

6. The virtual reality display method according to claim 1, characterized in that: After determining whether the target display device is at the optimal viewing angle of the driver or whether it can be adjusted to the optimal viewing angle of the driver, the method further includes: Determine the optimal viewing angle position of the target display device at the driver, and control the target display device to display the A-pillar blind spot scene image; or, Determine that the target display device is not at the driver's optimal viewing angle position but can be adjusted to the driver's optimal viewing angle position, control the target display device to move to the optimal viewing angle position and display the A-pillar blind spot scene image.

7. The virtual reality display method according to claim 6, characterized in that: The vehicle further includes a first camera device and a second camera device disposed on a rearview mirror, and the control of the target display device to move to the optimal viewing angle position and display the A-pillar blind spot scene image includes: Determine a target camera device from the first camera device and the second camera device according to the sight direction of the vehicle driver, wherein the target camera device is a camera device on the rearview mirror corresponding to the sight direction of the driver; Adjusting the shooting angle of the target camera device according to the sight direction of the vehicle driver to obtain a target road image of the A-pillar blind spot scene; The target display device is controlled to move to the optimal viewing angle position and the target road image is displayed on the target display device.

8. The virtual reality display method according to any one of claims 1 to 7, characterized in that: The obtaining of the sight direction of the vehicle driver includes: Obtaining the eye position and head posture of the driver of the vehicle; The sight direction of the driver is obtained according to the eye position and the head posture.

9. The virtual reality display method according to claim 1, characterized in that: The determining, based on the sight line direction of the vehicle driver, whether the target display device is at the driver's optimal viewing angle position or whether it can be adjusted to the driver's optimal viewing angle position includes: Acquire the eye viewing angle range pre-entered by the vehicle driver, wherein the eye viewing angle range includes a left-right viewing angle range or a top-bottom viewing angle range; Determining an optimal viewing angle range according to the sight direction and eye viewing angle range of the vehicle driver; When the target display device is within the optimal viewing angle range or can be completely moved into the optimal viewing angle range, it is determined that the target display device is at the optimal viewing angle position for the driver or can be adjusted to the optimal viewing angle position for the driver.

10. A virtual reality display system, characterized in that: The method comprises a memory and a processor, wherein the memory is used to store program codes, and the processor is used to call the program codes to execute the method according to any one of claims 1 to 9.

Images