Vehicle driving assistance system based on VR glasses and automobile

Abstract

The invention discloses a vehicle driving assistance system and device based on VR glasses and a storage medium, the system comprises at least one vehicle-mounted camera, the VR glasses and a control module, the vehicle-mounted camera obtains a first image through shooting, and a head-mounted camera in the VR glasses obtains a second image through shooting; and the control module sends the first image and / or the second image to the VR display for display. A driver can wear the VR glasses, shoot the first image through the vehicle-mounted camera and shoot the second image through the head-mounted camera, the VR display displays the first image and / or the second image, and the driver can directly observe the environment outside / inside the vehicle by watching the first image and / or the second image instead of naked eyes. Therefore, under the condition that the weather condition is poor and the visual field of naked eyes is affected, good observation can be performed by means of the VR glasses, so that a driver can observe the traffic condition, and the traffic safety can be guaranteed. The invention is widely applied to the technical field of automobiles.

Description

Technical Field

[0001] The present invention relates to the field of automobile technology, and in particular to a vehicle driving assistance system and a vehicle based on VR glasses. Background Art

[0002] When driving a car, ensuring good visibility is crucial for traffic safety. Current automotive technology requires drivers to observe road conditions with their naked eyes. However, due to the ever-changing driving environment, poor lighting conditions or strong direct sunlight can disrupt the driver's field of vision, making it difficult to see the road clearly. Furthermore, in heavy rain, the windshield is flooded with water, and activated wipers can also affect the driver's field of vision, compromising driving safety. Summary of the Invention

[0003] In order to address the technical issues such as the driver's field of vision being disturbed by changes in the external environment when observing road conditions with the naked eye, affecting driving safety, the purpose of the present invention is to provide a vehicle driving assistance system, device and storage medium based on VR glasses.

[0004] In one aspect, an embodiment of the present invention includes a vehicle driving assistance system based on VR glasses, the vehicle driving assistance system based on VR glasses including:

[0005] At least one vehicle-mounted camera; the vehicle-mounted camera is used to be installed on the body of the vehicle, and the vehicle-mounted camera is used to capture the environment outside the vehicle to obtain a first image;

[0006] VR glasses; the VR glasses are worn by a person in the car, and the VR glasses include a VR display and a head-mounted camera, and the head-mounted camera is used to capture the face of the person in the car to obtain a second image;

[0007] Control module; the control module is used to obtain the first image and the second image, and send the first image and / or the second image to the VR display for display.

[0008] Furthermore, the at least one vehicle-mounted camera includes multiple vehicle-mounted cameras, each of which is distributed at multiple different positions on the body of the car, each of which forms multiple different shooting field of view directions, and the shooting field of view direction of at least one vehicle-mounted camera is toward the front of the car.

[0009] Furthermore, the VR glasses also include a head posture sensor, which is used to detect head posture information of people in the car.

[0010] Furthermore, sending the first image and / or the second image to the VR display for display includes:

[0011] Performing panoramic stitching on all of the first images to obtain a stitched image;

[0012] An image is selected from the second image and the stitched image and sent to the VR display for display.

[0013] Furthermore, selecting an image from the second image and the stitched image and sending it to the VR display for display includes:

[0014] determining the facial orientation of the person in the vehicle based on the head posture information;

[0015] Determining whether the facial direction is toward the inside of the vehicle or toward the outside of the vehicle;

[0016] When the face is facing inward of the vehicle, selecting the second image and sending it to the VR display for display;

[0017] When the face is facing outside the vehicle, the stitched image is selected and sent to the VR display for display.

[0018] Furthermore, the determining whether the facial direction is toward the inside of the vehicle or toward the outside of the vehicle includes:

[0019] Establishing a connection line between the position of the person on the vehicle and each body pillar of the vehicle;

[0020] Detect the current opening of each car window;

[0021] For any two adjacent lines, if there is a windshield or a window whose current opening is greater than the opening threshold in the area between the two lines, the area between the two lines is marked as an outward-facing area; otherwise, the area between the two lines is marked as an inward-facing area;

[0022] When the facial direction is located in the in-vehicle direction area, determining that the facial direction is facing inward of the vehicle;

[0023] When the facial direction is located in the vehicle-outward-facing area, it is determined that the facial direction is facing outward from the vehicle.

[0024] Furthermore, sending the first image and / or the second image to the VR display for display includes:

[0025] determining the facial orientation of the person in the vehicle based on the head posture information;

[0026] Determining, according to the facial direction, one of the vehicle-mounted cameras having a field of view direction closest to the facial direction;

[0027] Selecting the first image taken by the determined vehicle-mounted camera;

[0028] An image is selected from the second image and the selected first image and sent to the VR display for display.

[0029] Furthermore, selecting an image from the second image and the selected first image and sending it to the VR display for display includes:

[0030] Performing traffic risk perception on each of the first images to obtain a traffic risk value corresponding to each of the first images;

[0031] When at least one of the traffic risk values ​​is greater than or equal to a risk threshold, selecting the first image to be sent to the VR display for display;

[0032] When all of the traffic risk values ​​are less than the risk threshold, the second image is selected and sent to the VR display for display.

[0033] Furthermore, the VR glasses further include a head force applying module, which is used to controllably apply a force in a specific direction to the head of the person in the vehicle;

[0034] The control module is used to determine the force application direction according to each traffic risk value, and control the head force application module to apply force to the head of the person in the vehicle according to the force application direction.

[0035] On the other hand, an embodiment of the present invention also includes a car, which is equipped with the vehicle driving assistance system based on VR glasses in the embodiment.

[0036] The beneficial effect of the present invention is that in a car equipped with the vehicle driving assistance system based on VR glasses in this embodiment, the driver can wear VR glasses, capture a first image through the vehicle-mounted camera and a second image through the head-mounted camera, and the first image and / or the second image are displayed on the VR display. The driver can watch the first image and / or the second image instead of directly observing the environment outside / inside the car with the naked eye. Therefore, even in bad weather conditions and when the naked eye's vision is affected, the driver can make good observations with the help of VR glasses, thereby ensuring the driver's observation of traffic conditions and facilitating traffic safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 Schematic diagram of the structure of a vehicle driving assistance system based on VR glasses in an embodiment;

[0038] Figure 2 Schematic diagram of the basic structure of VR glasses in the embodiment;

[0039] Figure 3Schematic diagram of the installation position of each vehicle-mounted camera in the embodiment;

[0040] Figure 4 Schematic diagram of the in-vehicle facing area and the out-vehicle facing area in the embodiment;

[0041] Figure 5 Schematic diagram of the structure of VR glasses provided with a head force application module in an embodiment. DETAILED DESCRIPTION

[0042] Explanation of terms:

[0043] VR: Virtual Reality is a technology that uses computers and other devices to produce a realistic virtual visual effect that reflects reality. On this basis, it can also realize a virtual world with multiple sensory experiences such as three-dimensional vision, touch, and smell, so that people in the virtual world have an immersive feeling.

[0044] In this embodiment, the vehicle driving assistance system based on VR glasses is as follows: Figure 1 As shown. Figure 1 The vehicle driving assistance system based on VR glasses includes a control module, VR glasses and at least one vehicle-mounted camera, namely, vehicle-mounted camera 1, vehicle-mounted camera 2, and vehicle-mounted camera 3.

[0045] In this embodiment, the control module is a component that has the functions of data acquisition, processing, output and control of other components. For example, an electronic processing unit ECU can be used as the control module. The structure of VR glasses is as follows Figure 2 As shown, the structure includes a VR display, a head-mounted camera, and a glasses frame. The VR display and head-mounted camera are mounted on the glasses frame. When the user wears VR glasses, the glasses frame is fixed to the user's head, so that the VR display and head-mounted camera remain relatively stationary with the user's head. The VR display can receive images sent by the control module and display the virtual reality effect to the user wearing the VR glasses. The field of view of the head-mounted camera is the same as the field of view of the user's eyes (assuming it is not blocked by the VR glasses).

[0046] In this embodiment, the VR glasses are further provided with a head posture sensor. The head posture sensor can detect head posture information such as the rotation angle and side rotation angle of the vehicle occupant's head. Specifically, the VR glasses can be equipped with a dedicated hardware component such as a gyroscope as a head posture sensor. Such hardware components can directly detect and obtain head posture information through their own performance. Alternatively, a head-mounted camera can also be used as a head posture sensor, and head posture information can be obtained by software processing of images captured by the head-mounted camera.

[0047] In this embodiment, the VR glasses are worn by people in the car, and the people in the car can specifically be the driver of the car.

[0048] In this embodiment, the control module is connected to each vehicle-mounted camera and the VR display and head-mounted camera in the VR glasses through a data bus, so that the control module can receive data sent by these components and send data to these components.

[0049] In this embodiment, refer to Figure 3 The vehicle-mounted camera is used to be installed on the body of the car, and the shooting direction of the vehicle-mounted camera is toward the outside of the car, so that the vehicle-mounted camera can shoot the environment outside the car.

[0050] In this embodiment, when there is only one vehicle-mounted camera (for example, vehicle-mounted camera 1), this vehicle-mounted camera can be preferentially installed in the front of the car, such as above the front windshield of the car, so that the shooting field of view of this vehicle-mounted camera is facing directly in front of the car. The shooting field of view of this vehicle-mounted camera is basically the same as the field of view of the user's eyes (assuming it is not blocked by VR glasses), both of which are forward along the face direction of the people in the car.

[0051] In this embodiment, when there are multiple vehicle-mounted cameras, Figure 3 As shown, a vehicle-mounted camera is installed in the front of the car, for example, above the front windshield of the car, so that the shooting field of view of this vehicle-mounted camera is facing the front of the car. The shooting field of view of this vehicle-mounted camera is basically the same as the field of view of the user's eyes (assuming that it is not blocked by VR glasses); other vehicle-mounted cameras are distributed at multiple different positions on the body of the car, and each vehicle-mounted camera forms multiple different shooting field of view directions. The shooting field of view direction of at least one vehicle-mounted camera is facing the front of the car, for example, Figure 3 In the figure, vehicle-mounted camera 2, vehicle-mounted camera 3, vehicle-mounted camera 4 and vehicle-mounted camera 5 are respectively installed on the left front side, right front side, left rear side and right rear side of the vehicle body, so as to respectively shoot the environment on the left front side, right front side, left rear side and right rear side of the vehicle body.

[0052] In this embodiment, each vehicle-mounted camera and head-mounted camera can shoot in the form of a video stream, for example, to obtain multiple images by high-speed shooting. Each vehicle-mounted camera and head-mounted camera can shoot at the same frame rate. Since the vehicle driving assistance system based on VR glasses works in real time and dynamically, the images captured by each vehicle-mounted camera (referred to as the first image) and the image captured by the head-mounted camera (referred to as the second image) at one of the shooting moments and the image processing by the control module can be used as an example for explanation. Unless otherwise specified, the first and second images mentioned are captured at the same shooting moment.

[0053] In this embodiment, the head posture sensor in the VR glasses can maintain the same sampling rate as each vehicle-mounted camera and head-mounted camera, thereby dynamically detecting head posture information in real time.

[0054] In this embodiment, each vehicle-mounted camera transmits its respective first image to the control module, and the head-mounted camera transmits its second image to the control module. The control module may not perform any processing on the first and second images, but may directly transmit them to the VR display for display. Alternatively, the control module may perform some image processing on the first and second images and then transmit the processed first and / or second images to the VR display for display. For example, the control module may set a constant display effect (constant regional brightness, human target brightness, vehicle target brightness, object target brightness, etc.). As the shooting time progresses, the position of the vehicle changes due to driving, and the external lighting conditions, weather, etc. may change. Therefore, the original display effects of the first image and the second image captured at different shooting times may change (for example, at the previous shooting time, the lighting conditions are good, and the original display effects of each first image and the second image have a higher brightness, while at the subsequent shooting time, the lighting conditions deteriorate, and the original display effects of each first image and the second image have a lower brightness. At the subsequent shooting time, the lighting conditions suddenly improve significantly, and the original display effects of each first image and the second image suddenly increase significantly). In this case, the control module may set a constant display effect value (for example, constant brightness) to adjust the display effects (for example, brightness) of the first image and the second image to a constant display effect value regardless of the original display effects (for example, brightness) of the first image and the second image captured at each shooting time.

[0055] Specifically, the constant display effect value set by the control module can be the overall display effect value (overall brightness) of the first image and the second image, or the display effect value of a partial area of ​​the first image and the second image (for example, the brightness of the center position of each image), or the display effect value of a specific target of the first image and the second image (for example, a person, a car, an object, etc.). In this way, even if the shooting conditions at the location of the car change at different shooting moments, resulting in changes in the original display effects of the first image and the second image, the control module processes the constant display effects of each first image and the second image, so that the overall display effect value (overall brightness), the display effect value of the partial area (for example, the brightness of the center position), and the display effect value of the specific target in the image (for example, the brightness of people, cars, objects, etc.) of each first image and the second image captured at different shooting moments can remain constant, so that the control module sends the processed first image and / or second image to the VR display for display. The display effect observed by the people on the car through the VR display is that the brightness and other data of the entire first image and the second image captured at different shooting moments, the partial area or the specific target are all constant, which is conducive to maintaining the stability of the visual effect of the entire first image and the second image, the partial area or the specific target, and reducing the impact of glare on the driver's observation of the overall situation, partial area or specific target outside the vehicle.

[0056] In this embodiment, the control module can send only the first image captured by one or more vehicle-mounted cameras to the VR display for display, or only send the second image captured by the head-mounted camera to the VR display for display, or send the first image and the second image to the VR display for display at the same time, based on manual or voice selection instructions from the people on the vehicle.

[0057] Specifically, if the control module chooses to send only the first image captured by one vehicle-mounted camera to the VR display for display, or chooses to send only the second image captured by the head-mounted camera to the VR display for display, that is, the VR display only receives one image at each moment, then the VR display can display the first image and the second image through its entire available display area; if the control module chooses to send the first images captured by multiple vehicle-mounted cameras to the VR display for display at the same time, or chooses to send both the first image and the second image to the VR display for display, that is, the VR display receives multiple images at each moment, then the VR display can divide its entire available display area into multiple sub-areas, with each sub-area displaying one of the first images or the second image.

[0058] In a car equipped with the VR glasses-based vehicle driving assistance system of this embodiment, the driver can wear VR glasses, capture a first image through the car-mounted camera and a second image through the head-mounted camera, and the first image and / or the second image are displayed on the VR display. The driver can view the first image and / or the second image instead of directly observing the environment outside / inside the car with the naked eye. Therefore, even in bad weather conditions and when the naked eye's field of vision is affected, the driver can use the VR glasses to make a good observation, thereby ensuring the driver's observation of the traffic situation and facilitating traffic safety. Moreover, the control module can process the first image and / or the second image, thereby ensuring that the visual effect observed by the driver remains stable when the shooting conditions of the car's environment are extreme (for example, at night without good external lighting and the car's lighting equipment malfunctions, or strong light is directly exposed) or changing (for example, when entering or exiting a tunnel or parking lot), thereby reducing the impact of adverse external observation conditions on the driver and facilitating traffic safety.

[0059] In this embodiment, when executing the step of sending the first image and / or the second image to the VR display for display, the control module may specifically perform the following steps:

[0060] S1. Perform panoramic stitching of all first images to obtain a stitched image;

[0061] S2. Select an image from the second image and the stitched image and send it to the VR display for display.

[0062] In step S1, the control module executes an image panoramic stitching algorithm to stitch together all first images captured by all vehicle cameras, including vehicle camera 1, vehicle camera 2, vehicle camera 3, vehicle camera 4 and vehicle camera 5, to obtain a panoramic stitching image.

[0063] In step S2, at the same time, the control module may choose to send the second image to the VR display for display, or choose to send the spliced ​​image to the VR display for display.

[0064] When the control module chooses to send the stitched image to the VR display for display, the driver can simultaneously observe the situations in the fields of view of vehicle-mounted camera 1, vehicle-mounted camera 2, vehicle-mounted camera 3, vehicle-mounted camera 4 and vehicle-mounted camera 5 through the VR display, thereby increasing the driver's observation range, which is beneficial for the driver to obtain more traffic information in the same time, and thus helps the driver to make appropriate driving behaviors and ensure traffic safety.

[0065] When the control module selects to send the second image to the VR display for display, the driver sees what is captured by the head-mounted camera on the VR glasses. Because the driver cannot directly observe with the naked eye while wearing VR glasses, the head-mounted camera acts as a substitute for the naked eye, allowing the driver to simulate the effect of naked-eye observation when wearing VR glasses.

[0066] For example, when the driver turns his head toward the back seat of the car, the field of view of the head-mounted camera will also turn toward the back seat. At this time, the second image captured by the head-mounted camera is an image of the back seat of the car, so that the driver can observe the situation in the back seat of the car by observing the second image without taking off the VR glasses. Specifically, if the driver is driving a car and there are passengers such as children who need care in the back seat, the driver may need to observe the situation of the back seat passengers from time to time; when the driver parks the car in a safe position (such as pulling over or waiting for a traffic light), the driver can turn his head to observe the back seat passengers without taking off the VR glasses, and when the car needs to be continued to be driven (for example, the traffic light turns green), the driver does not need to put on the VR glasses again and can drive the car directly in a normal driving posture, thereby improving the convenience of using the vehicle driving assistance system based on VR glasses.

[0067] In this embodiment, when the control module executes step S2, that is, the step of selecting an image from the second image and the stitched image and sending it to the VR display for display, the control module may specifically execute the following steps:

[0068] S201. Determine the facial direction of the vehicle's personnel based on head posture information;

[0069] S202. Determine whether the facial direction is toward the inside or outside of the vehicle;

[0070] S203. When the face is facing the inside of the car, a second image is selected and sent to the VR display for display;

[0071] S204. When the face is facing outside the vehicle, select the stitched image and send it to the VR display for display.

[0072] In step S201, the control module may utilize the head posture information detected by the head posture sensor to determine the facial orientation of the vehicle occupant based on the head posture sensor. For example, when the head rotation angle in the head posture information is 0, the control module may determine the facial orientation as the front of the vehicle body. When the head rotation angle changes to other values, the control module may use the head rotation angle as the angle between the front of the vehicle body and the facial orientation to determine the facial orientation.

[0073] In step S202, you can refer to Figure 4The control module can establish a vehicle coordinate system. The occupant's position is the driver's position. The vehicle's body pillars, including the left A-pillar, right A-pillar, left B-pillar, right B-pillar, left C-pillar, and right C-pillar, have fixed coordinates in the vehicle coordinate system. This allows for establishing lines connecting the occupant's position and the vehicle's body pillars. The occupant's facial orientation can be represented as a vector in the vehicle coordinate system.

[0074] In this embodiment, refer to Figure 4 Any two adjacent lines define a region where the faces of the vehicle occupants may be located. Within each region, there is a front windshield, rear windshield, or window. The control module determines whether these regions are facing outward or inward, respectively.

[0075] Specifically, the control module can make judgments based on the following rules:

[0076] (1) The area where the front windshield and rear windshield are located can be marked as the area facing outward from the vehicle; the area where there is no front windshield or rear windshield, that is, the area where the vehicle windows are located, can be further judged;

[0077] (2) For any area with a window, the control module can call the opening sensor to detect the current opening of the window. If the current opening is greater than or equal to the opening threshold (for example, 50%), then this area is marked as an area facing outward from the vehicle; if the current opening is less than the opening threshold, then this area is marked as an area facing inward from the vehicle.

[0078] Through the above rules (1)-(2), it can be determined that the area where the front windshield and the rear windshield are located is fixed as the outward-facing area, while other areas belong to the outward-facing area or the inward-facing area. It is dynamically determined that if the current opening of the window in the area is greater than or equal to the opening threshold (that is, the current opening is large), then this area belongs to the outward-facing area. Conversely, if the current opening is small, then this area belongs to the inward-facing area.

[0079] After determining whether each area belongs to the outward-facing area or the inward-facing area, the driver's facial direction is detected as belonging to the inward-facing area or the outward-facing area. If the facial direction is within the inward-facing area, the control module determines that the facial direction is facing inward, thereby triggering the execution of step S203, in which the control module selects a second image and sends it to the VR display for display, allowing the driver to observe the in-vehicle environment by observing the second image captured by the head-mounted camera in the VR glasses. If the facial direction is within the outward-facing area, the control module determines that the facial direction is facing outward, thereby triggering the execution of step S204, in which the control module selects a stitched image and sends it to the VR display for display, allowing the driver to observe the outside environment by observing the stitched images captured by the various on-board cameras.

[0080] In this embodiment, the principle behind executing steps S201-S204 is that the windshield (which can be limited to the front windshield) and the areas where the currently wide-open windows are located are areas where the driver typically needs to observe the outside of the vehicle while driving. Therefore, these areas are marked as outward-facing areas. When the facial direction is within the outward-facing area, it can be determined that the facial direction is facing outward, meaning that the driver's head-turning movement is a natural movement when driving to observe the outside of the vehicle. Therefore, a stitched image is selected and sent to the VR display for display. When the VR display displays the stitched image, the stitched image can be rotated with the VR display. Other areas are areas where the driver needs to observe the inside of the vehicle. Therefore, these areas are marked as inward-facing areas. When the facial direction is within the inward-facing area, it can be determined that the facial direction is facing inward, meaning that the driver's head-turning movement is intended to observe the inside of the vehicle (e.g., the back seat). Therefore, a second image is selected and sent to the VR display for display. By executing steps S201-S204, the driver's intention can be automatically identified, thereby displaying the appropriate image.

[0081] In this embodiment, when executing the step of sending the first image and / or the second image to the VR display for display, the control module may specifically perform the following steps:

[0082] P1. Determine the facial orientation of the occupants based on head posture information;

[0083] P2. Determine the vehicle-mounted camera with the closest field of view based on the facial orientation.

[0084] P3 selects the first image captured by the determined vehicle camera;

[0085] P4. Select an image from the second image and the selected first image and send it to the VR display for display.

[0086] The principle of step P1 is the same as that of step S201.

[0087] In step P2, the control module can establish a connection line between the driver's position and each vehicle-mounted camera, and calculate the angle between the facial direction and each connection line. The vehicle-mounted camera corresponding to the connection line with the smallest angle is the vehicle-mounted camera with the direction closest to the field of view. Step P3 is executed to obtain the first image taken by this vehicle-mounted camera.

[0088] In step P4, one of the second image captured by the head-mounted camera in the VR glasses and the first image selected in step P3 is selected and sent to the VR display for display.

[0089] Specifically, when the control module executes step P4, that is, the step of selecting an image from the second image and the selected first image and sending it to the VR display for display, the control module may specifically execute the following steps:

[0090] P401. Perform traffic risk perception on each first image to obtain a traffic risk value corresponding to each first image;

[0091] P402. When there is at least one traffic risk value greater than or equal to the risk threshold, select the first image to be sent to the VR display for display;

[0092] P403. When all traffic risk values ​​are less than the risk threshold, select the second image and send it to the VR display for display.

[0093] In step P401, taking one of the first images as an example, the control module can execute a target recognition algorithm and a trajectory prediction algorithm to identify targets such as pedestrians, bystanders, and traffic signs, and predict the targets' movement trajectories relative to the vehicle. The control module can also assign weights to targets based on their type, such as a smaller weight for pedestrians, a medium weight for traffic signs, and a larger weight for bystanders. The control module can also negatively correlate the minimum distance between the target's trajectory and the vehicle by determining a baseline risk value for the target. The target's risk value is then determined by multiplying the baseline risk value by the target's weight. The risk values ​​of all targets in the first image are then accumulated to determine a traffic risk value corresponding to the first image. The traffic risk value represents the risk of a collision, such as a collision, with the vehicle, for targets such as pedestrians and bystanders within the vehicle's field of view.

[0094] The control module can set a risk threshold to measure the size of the traffic risk value corresponding to each first image. For example, when the traffic risk value is greater than or equal to the risk threshold, the traffic risk value is determined to be large; when the traffic risk value is less than the risk threshold, the traffic risk value is determined to be small.

[0095] The control module compares the traffic risk values ​​corresponding to each first image with the risk threshold. If at least one traffic risk value is greater than or equal to the risk threshold, meaning at least one traffic risk value is high, the control module executes step P402 and selects a first image to be sent to the VR display for display. Specifically, the control module may select first images with high traffic risk values ​​for display on the VR display, or may select the first image captured by the on-board camera with the closest field of view direction, as determined in steps P2-P3, for display on the VR display. If all traffic risk values ​​are less than the risk threshold, meaning all traffic risk values ​​are low, the control module executes step P403 and selects a second image for display on the VR display.

[0096] In this embodiment, the principle of executing steps P1-P4 is that: when there is a first image with a larger traffic risk value, the VR display preferentially displays the first image, which can enable the driver to pay attention to the higher traffic risk outside the vehicle, so that the driver can make appropriate driving operations in time, which is conducive to ensuring traffic safety; when there is no first image with a larger traffic risk value, the VR display can display a second image that follows the driver's head movement, so that the driver can clearly observe the situation inside the vehicle when wearing VR glasses, and thus provide care to passengers in need, etc., thereby improving the convenience of using the vehicle driving assistance system based on VR glasses.

[0097] In this embodiment, refer to Figure 5 VR glasses are also equipped with a head force application module. Specifically, the head force application module can be a massage module distributed in different parts of the glasses frame. These massage modules can apply force in a specific direction to the head of the driver wearing the VR glasses, thereby achieving a head massage effect on the occupants. For example, the massage module installed on the left temple of the glasses frame can apply a force to the right of the driver's head, and the massage module installed on the right temple of the glasses frame can apply a force to the left of the driver's head.

[0098] In this embodiment, when a large traffic risk value is detected, the control module can determine the direction of the vehicle-mounted camera that captured the first image with a large traffic risk value (generally, only one vehicle-mounted camera will capture the first image with a large traffic risk value at the same time) relative to the driver, determine the same direction as the force application direction, and control the head force application module to apply force to the head of the person in the vehicle according to the force application direction.

[0099] For example, when the control module detects that the traffic risk value of the first image captured by the vehicle-mounted camera 3 is large, and the direction of the vehicle-mounted camera 3 relative to the driver is to the right, then the force direction is determined to be the same direction, that is, to the right, and the head force application module is controlled to apply a force to the right on the head of the person in the car. For example, the control module can control the massage module installed on the left leg of the glasses frame to apply a force to the right on the driver's head, thereby prompting the driver that there is a greater traffic risk on his right, thereby attracting the driver's attention and guiding the driver to make appropriate driving operations to ensure traffic safety.

[0100] The vehicle driving assistance system based on VR glasses can be installed on a car so that the vehicle driving assistance system based on VR glasses is installed to form a whole with other components on the car. Such a car has the same technical effect as the vehicle driving assistance system based on VR glasses.

[0101] It should be noted that, unless otherwise specified, when a feature is referred to as being "fixed" or "connected" to another feature, it may be directly fixed or connected to the other feature, or it may be indirectly fixed or connected to the other feature. In addition, the descriptions of up, down, left, right, etc. used in this disclosure are only relative to the relative positional relationships of the components of the present disclosure in the accompanying drawings. The singular forms of "a", "" and "the" used in this disclosure are also intended to include the plural forms, unless the context clearly indicates otherwise. In addition, unless otherwise defined, all technical and scientific terms used in this embodiment have the same meaning as those generally understood by those skilled in the art. The terms used in the specification of this embodiment are only for describing specific embodiments and are not intended to limit the invention. The term "and / or" used in this embodiment includes any combination of one or more related listed items.

[0102] It should be understood that, although the present disclosure may adopt the term first, second, third etc. to describe various elements, these elements should not be limited to these terms.These terms are only used to distinguish the elements of the same type from each other.For example, without departing from the scope of the present disclosure, the first element may also be referred to as the second element, and similarly, the second element may also be referred to as the first element.The use of any and all examples or exemplary language ("for example", "such as" etc.) provided by the present embodiment is only intended to better illustrate embodiments of the present invention, and unless otherwise required, the scope of the present invention will not be limited.

[0103] It should be appreciated that embodiments of the present invention can be implemented or practiced by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable memory. The methods can be implemented in a computer program using standard programming techniques - including a non-transitory computer-readable storage medium configured with a computer program, wherein the storage medium so configured causes the computer to operate in a specific and predefined manner - according to the methods and figures described in the specific embodiments. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if desired, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. In addition, the program can be run on a programmed application-specific integrated circuit for this purpose.

[0104] In addition, the operations of the process described in this embodiment may be performed in any suitable order, unless otherwise indicated in this embodiment or otherwise clearly contradicted by the context. The process described in this embodiment (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as a code (e.g., executable instructions, one or more computer programs, or one or more applications) executed on one or more processors, by hardware or a combination thereof. A computer program includes a plurality of instructions that may be executed by one or more processors.

[0105] Furthermore, the method can be implemented in any type of computing platform that is operably connected to a suitable computer, including but not limited to a personal computer, a minicomputer, a mainframe, a workstation, a network or distributed computing environment, a separate or integrated computer platform, or in communication with a charged particle tool or other imaging device, etc. Various aspects of the present invention can be implemented as machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, an optical read and / or write storage medium, RAM, ROM, etc., so that it can be read by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the process described herein. In addition, the machine-readable code, or portions thereof, can be transmitted over a wired or wireless network. When such media includes instructions or programs that implement the above steps in conjunction with a microprocessor or other data processor, the invention of this embodiment includes these and other different types of non-transitory computer-readable storage media. When programmed according to the methods and techniques of the present invention, the present invention also includes the computer itself.

[0106] The computer program can be applied to input data to perform the functions of the present embodiment, thereby converting the input data to generate output data that is stored in a non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present invention, the converted data represents a physical and tangible object, including a specific visual depiction of the physical and tangible object produced on the display.

[0107] The above are merely preferred embodiments of the present invention. The present invention is not limited to the aforementioned embodiments. As long as the technical effects of the present invention are achieved by the same means, any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention. Within the scope of protection of the present invention, various modifications and variations of the technical solutions and / or implementation methods may be made.

Claims

1. A vehicle driving assistance system based on VR glasses, characterized in that: The vehicle driving assistance system based on VR glasses includes: At least one vehicle-mounted camera; the vehicle-mounted camera is used to be installed on the body of the vehicle, and the vehicle-mounted camera is used to capture the environment outside the vehicle to obtain a first image; VR glasses; the VR glasses are worn by a person in the car, and the VR glasses include a VR display and a head-mounted camera, and the head-mounted camera is used to capture the face of the person in the car to obtain a second image; Control module; the control module is used to obtain the first image and the second image, and send the first image and / or the second image to the VR display for display.

2. The vehicle driving assistance system based on VR glasses according to claim 1, characterized in that: The at least one vehicle-mounted camera includes multiple vehicle-mounted cameras, each of which is distributed at multiple different positions on the body of the car, each of which forms multiple different shooting field of view directions, and the shooting field of view direction of at least one vehicle-mounted camera is toward the front of the car.

3. The vehicle driving assistance system based on VR glasses according to claim 1 or 2, characterized in that: The VR glasses also include a head posture sensor, which is used to detect head posture information of people in the car.

4. The vehicle driving assistance system based on VR glasses according to claim 3 is characterized in that: The sending the first image and / or the second image to the VR display for display includes: Performing panoramic stitching on all of the first images to obtain a stitched image; An image is selected from the second image and the stitched image and sent to the VR display for display.

5. The vehicle driving assistance system based on VR glasses according to claim 4 is characterized in that: The selecting an image from the second image and the stitched image and sending it to the VR display for display includes: determining the facial orientation of the person in the vehicle based on the head posture information; Determining whether the facial direction is toward the inside of the vehicle or toward the outside of the vehicle; When the face is facing inward of the vehicle, selecting the second image and sending it to the VR display for display; When the face is facing outside the vehicle, the stitched image is selected and sent to the VR display for display.

6. The vehicle driving assistance system based on VR glasses according to claim 5, characterized in that: The determining whether the facial direction is facing inside the vehicle or outside the vehicle includes: Establishing a connection line between the position of the person on the vehicle and each body pillar of the vehicle; Detect the current opening of each car window; For any two adjacent lines, if there is a windshield or a window whose current opening is greater than the opening threshold in the area between the two lines, the area between the two lines is marked as an outward-facing area; otherwise, the area between the two lines is marked as an inward-facing area; When the facial direction is located in the in-vehicle direction area, determining that the facial direction is facing inward of the vehicle; When the facial direction is located in the vehicle-outward-facing area, it is determined that the facial direction is facing outward from the vehicle.

7. The vehicle driving assistance system based on VR glasses according to claim 4, characterized in that: The sending the first image and / or the second image to the VR display for display includes: determining the facial orientation of the person in the vehicle based on the head posture information; Determining, according to the facial direction, one of the vehicle-mounted cameras having a field of view direction closest to the facial direction; Selecting the first image taken by the determined vehicle-mounted camera; An image is selected from the second image and the selected first image and sent to the VR display for display.

8. The vehicle driving assistance system based on VR glasses according to claim 7, characterized in that: The selecting an image from the second image and the selected first image and sending the image to the VR display for display includes: Performing traffic risk perception on each of the first images to obtain a traffic risk value corresponding to each of the first images; When at least one of the traffic risk values ​​is greater than or equal to a risk threshold, selecting the first image to be sent to the VR display for display; When all of the traffic risk values ​​are less than the risk threshold, the second image is selected and sent to the VR display for display.

9. The vehicle driving assistance system based on VR glasses according to claim 8, characterized in that: The VR glasses further include a head force applying module, which is used to controllably apply a force in a specific direction to the head of the person in the vehicle; The control module is used to determine the force application direction according to each traffic risk value, and control the head force application module to apply force to the head of the person in the vehicle according to the force application direction.

10. An automobile, characterized in that: The car is installed with the vehicle driving assistance system based on VR glasses as described in any one of claims 1-9.

Images