Vehicle-mounted VR glasses driving system and method
Through the on-board VR glasses driving system, the vehicle sensor and central processing unit are used to provide 360-degree panoramic images, combined with motion capture and eye tracking modules, the problem of blind spots in the traditional system is solved, driving safety and convenience are improved, and the The driver's comprehensive perception of the surrounding environment.
Patent Information
- Application Number
- CN202410947002.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-15
- Publication Date
- 2024-11-08
AI Technical Summary
Traditional driving assistance systems rely on cameras and sensors with fixed viewing angles, which increase driving risks, especially in narrow streets and inclement weather conditions.
The vehicle-mounted VR glasses driving system is adopted to collect environmental information through the vehicle sensor system, and the central processor analyzes and adjusts the information reception direction in real time. The VR assisted driving system provides a 360-degree panoramic image, combining motion capture and eye tracking modules to adjust the camera viewing angle. The interactive system receives driver instructions, and the cloud server performs data sharing and coordinated driving.
Provides a comprehensive and immersive driving vision, which significantly improves driving safety and convenience, avoids blind spots in the field of vision, ensures drivers' perception of the surrounding environment, balances driving functions diversity and safety, and reduces distraction The accident caused.
Smart Images

Figure CN118907134A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of intelligent driving assistance technology, and in particular to a vehicle-mounted VR glasses driving system and method. Background Art
[0002] In recent years, with the rapid development of smart cars and autonomous driving technology, vehicle safety and driving experience have become the focus of research and development in the automotive industry. Traditional driving assistance systems, such as reversing radar, blind spot monitoring, lane keeping assistance, etc., have been widely used in vehicles on the market, effectively improving driving safety and convenience. However, most of these systems rely on cameras and sensors with fixed viewing angles, and there are certain blind spots. It is difficult for a single-point camera to cover the entire area around the vehicle, especially the near-ground blind spot and side blind spot in front of the vehicle, which is particularly prominent in narrow streets and parking lots. In addition, at night, in rain and fog and other bad weather conditions, the driver's field of vision is also limited, increasing the risk of accidents. Summary of the invention
[0003] The purpose of the present invention is to provide a vehicle-mounted VR glasses driving system and method, which provides an immersive and all-round driving vision for the driver by integrating virtual reality technology and vehicle-mounted sensor networks, greatly improving the safety and convenience of driving, and solving the problems raised in the above background technology. To achieve the above purpose, the present invention provides the following technical solutions:
[0004] A vehicle-mounted VR glasses driving system includes: an off-vehicle sensing system for collecting environmental information around the vehicle; a central processing unit for processing data from the off-vehicle sensing system, analyzing the vehicle's surrounding environment in real time and identifying potential dangers, and adjusting the information receiving direction of the off-vehicle sensing system according to the motion information of the wearer; a VR assisted driving system for the vehicle driver to wear and receive and display a 360-degree panoramic image of the vehicle's surrounding environment in real time, while collecting the wearer's motion information for interaction; an interactive system for receiving the driver's instructions to help the driver obtain vehicle driving information and solve corresponding driving problems; the off-vehicle sensing system, the central processing unit, the VR assisted driving system and the interactive system are connected via optical fibers or wires; a cloud server is used to collect and store driving data and environmental data uploaded by all intelligent driving vehicles, and to exchange data and coordinate driving between vehicles, and is wirelessly connected to the central processing unit.
[0005] Preferably, the off-vehicle sensing system includes a camera, a radar, an environmental adaptation module and an alarm module; the camera is installed around the vehicle body and can rotate 360° in all directions to take photos, record videos and store the off-vehicle environment, while uploading the images to the central processor for analysis in real time, and supporting infrared night vision, thermal imaging and optical zoom; the environmental adaptation module can clean and dry the camera to ensure a clear field of view of the camera; the radar can detect the position and distance of vehicles coming from the side and rear; the alarm module can sound an alarm when the radar detects objects that are too close around the vehicle body.
[0006] Preferably, the VR assisted driving system includes a helmet and glasses, a display module, a motion capture module, a protection module and an identity authentication module. The helmet and glasses are used to be worn on the driver's head and face and serve as a carrier of the display module, the motion capture module, the protection module and the identity authentication module; the display module is located in front of the driver's field of vision and is used to display multi-directional images outside the vehicle and radar reminders and alarms, as well as display vehicle operating conditions such as vehicle gear position, fuel tank capacity, navigation, vehicle speed, engine speed, temperature, etc.
[0007] Preferably, the motion capture module includes a head motion capture unit and a line of sight tracking unit. The head motion capture unit is used to sense the wearer's head turning movement, and the line of sight tracking unit is used to record the wearer's eye movement and gaze direction, and determine the line of sight point through analysis. The head motion capture unit and the line of sight tracking unit transmit the head rotation information and the line of sight point information to the central processing unit, which analyzes and sends a follow-up instruction to the camera.
[0008] Preferably, the protection module is used to automatically pop open the helmet glasses when the system is abnormal or fails, thereby restoring the driver's natural field of vision; the identity authentication module is used to identify the wearer's identity and select a corresponding driving mode according to different driving identities.
[0009] Preferably, the interactive system includes a steering wheel operation interface and a voice interaction module. The steering wheel operation interface is electrically connected to the camera. The voice interaction module is used to recognize and convert the driver's voice commands, switch the camera's viewing angle, and control the camera's zoom, photo taking and video recording through the steering wheel operation interface or the voice interaction module.
[0010] Preferably, the driving mode includes a driving mode applicable to the main driver and a navigation mode applicable to the co-pilot, the driving mode only supports some functions, and the navigation mode supports all functions.
[0011] An operation method of a vehicle-mounted VR glasses driving system is implemented based on any one of the vehicle-mounted VR glasses driving systems described above, comprising the following steps:
[0012] Step 1: Before driving, the wearer is selected as the main driver or the co-driver through the identity verification module. If the wearer is selected as the main driver, the wearer is set to driving mode; if the wearer is selected as the co-driver, the wearer is set to navigation mode.
[0013] Step 2: During driving, the motion capture module and the eye movement module instantly detect the wearer's head movement and steering as well as the eye gaze point, and transmit the data to the central processing unit. The central processing unit's algorithm converts the data into actual viewing angle adjustment commands, and sends them to the adjustable seat at the bottom of the camera to adjust the angle of the camera to achieve follow-up;
[0014] Step 3: The driver issues a command to the camera to take photos, record videos or store data through the steering wheel operation interface or voice interaction module, or responds to the alarm issued by the radar alarm module. The camera and radar transmit the external environment and objects to the central processor. The central processor processes the data and transmits it to the display module, so that the wearer can observe the panoramic image and dynamics around the vehicle in real time. The central processor uploads the environment and driving data to the cloud server through wireless communication.
[0015] Step 4: When part of the camera is blocked or the system is delayed or freezes, the central processing unit detects that it cannot receive and process data normally, and sends a command to the safety protection module to quickly detach the helmet glasses and restore the wearer's normal field of vision.
[0016] Compared with the prior art, the present invention has the following beneficial effects:
[0017] 1. The present invention sets a VR assisted driving system to receive data transmitted by an off-vehicle sensor system in real time, display a 360-degree panoramic image of the vehicle's surrounding environment and the location of objects outside the vehicle, and a motion capture module and an eye movement module to instantly detect the wearer's head movement and steering as well as the point of sight of the eyes, and transmit the data to a central processing unit. The data is converted into actual viewing angle adjustment commands through the algorithm of the central processing unit, and is sent to the adjustable seat at the bottom of the camera to adjust the angle of the camera to achieve follow-up, thereby providing the wearer with an all-round and immersive driving field of view, clearing blind spots, significantly improving the driver's perception of the surrounding environment, and improving driving safety and convenience.
[0018] 2. The present invention selects the driving identity of the wearer by setting an identity authentication module. When the wearer is selected as the main driver, the driving mode is set to support only some functional applications and displays. When the wearer is selected as the co-driver, the driving mode is set to support all functions. Differentiation and screening are performed according to the driving identity to avoid the support and display of a large amount of driving information interfering with the main driver's attention, so as to achieve a balance between the diversity of VR driving functions and driving safety, and avoid accidents caused by distraction.
[0019] 3. The present invention can upload vehicle driving conditions and environmental information through the wireless communication connection between the central processor and the cloud server, realize information sharing between vehicles, and promote the construction of collaborative driving and intelligent transportation systems. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] Figure 1 It is the overall structure diagram of the present invention;
[0021] Figure 2 The module structure diagram of the vehicle exterior sensing system, VR assisted driving system, interactive system and central processing unit of the present invention. DETAILED DESCRIPTION
[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0023] In order to solve the problem that the fixed-angle cameras and sensors used in traditional vehicles have large blind spots and cannot cover the entire area around the vehicle, resulting in increased driving risks, please refer to Figure 1-2 , this embodiment provides the following technical solutions:
[0024] A vehicle-mounted VR glasses driving system includes: an off-vehicle sensing system for collecting environmental information around the vehicle; a central processing unit for processing data from the off-vehicle sensing system, analyzing the vehicle's surrounding environment in real time and identifying potential dangers, and adjusting the information receiving direction of the off-vehicle sensing system according to the motion information of the wearer; a VR assisted driving system for the vehicle driver to wear and receive and display a 360-degree panoramic image of the vehicle's surrounding environment in real time, while collecting the wearer's motion information for interaction, and the VR assisted driving system is the core of the entire system; an interactive system for receiving the driver's instructions to help the driver obtain vehicle driving information and solve corresponding driving problems; the off-vehicle sensing system, the central processing unit, the VR assisted driving system and the interactive system are connected via optical fibers or wires; a cloud server for collecting and storing driving data and environmental data uploaded by all intelligent driving vehicles, and for data exchange and collaborative driving between vehicles, and wirelessly connected to the central processing unit.
[0025] The exterior sensing system includes cameras, radars, environmental adaptation modules and alarm modules. The cameras are installed around the vehicle body and can rotate 360° in all directions to take photos, record videos and store them. At the same time, the images are uploaded to the central processor for analysis in real time, and support infrared night vision, thermal imaging and optical zoom. The environmental adaptation module can clean and dry the camera to ensure a clear field of view of the camera. The wiper structure is installed on the camera to automatically clean and dry the camera to ensure clear image capture. The radar can detect the position and distance of vehicles coming from the side and rear. The alarm module can sound an alarm when the radar detects objects that are too close around the vehicle body, and the driver responds to the alarm instructions to cancel the alarm.
[0026] The VR assisted driving system includes a helmet, a display module, a motion capture module, a protection module and an identity verification module. The helmet is worn on the driver's head and face and serves as a carrier of the display module, the motion capture module, the protection module and the identity verification module. The helmet is similar to the design of HTC VR glasses, connected to the camera and radar outside the vehicle through optical fibers or wires, and has a high-resolution display module to provide zero-delay real-time images and display 360-degree surround vision, so that the driver can obtain a full range of driving vision. The display module is located in front of the driver's field of vision and is used to display multi-directional images outside the vehicle and radar reminders and alarms, as well as display vehicle gear position, fuel tank capacity, navigation, vehicle speed, engine speed, temperature and other vehicle operating conditions. The motion capture module includes a head motion capture unit and a line of sight tracking unit. The head motion capture unit is used to sense the wearer's head turning movement, and the line of sight tracking unit is used to record the wearer's eye movement and gaze direction, and determine the line of sight by analysis. The head motion capture unit and the line of sight tracking unit transmit the head rotation information and line of sight landing information to the central processor, which analyzes and sends a follow-up instruction to the camera.
[0027] The protection module is used to automatically pop open the helmet glasses when the system is abnormal or fails, restoring the driver's natural field of vision; the identity authentication module is used to identify the wearer's identity and select the corresponding driving mode according to different driving identities.
[0028] The interactive system includes a steering wheel operation interface and a voice interaction module. The steering wheel operation interface is electrically connected to the camera, and the camera focus, viewing angle, recording or taking pictures are controlled by shortcut keys on the steering wheel operation interface; the voice interaction module is used to recognize and convert the driver's voice commands, making it easier for the driver to operate when both hands are busy.
[0029] The driving modes include the driving mode for the main driver and the navigation mode for the co-driver. The driving mode only supports some necessary functions, such as gear position, fuel tank capacity, navigation, vehicle speed, engine speed and temperature, etc., to ensure driving concentration; the navigation mode supports all functions, including optical zoom, infrared and thermal imaging, 360-degree panoramic view, etc., to increase the interactive experience of passengers.
[0030] An operation method of a vehicle-mounted VR glasses driving system is implemented based on any one of the vehicle-mounted VR glasses driving systems described above, comprising the following steps:
[0031] Step 1: Before driving, the wearer is selected as the main driver or the co-driver through the identity verification module. If the wearer is selected as the main driver, the wearer is set to driving mode; if the wearer is selected as the co-driver, the wearer is set to pilot mode.
[0032] Step 2: During driving, the motion capture module and the eye movement module instantly detect the wearer's head movement and steering as well as the eye gaze point, and transmit the data to the central processing unit. The central processing unit's algorithm converts the data into actual viewing angle adjustment commands, and sends them to the adjustable seat at the bottom of the camera to adjust the angle of the camera to achieve follow-up;
[0033] Step 3: The driver issues a command to the camera to take photos, record videos or store data through the steering wheel operation interface or voice interaction module, or responds to the alarm issued by the radar alarm module. The camera and radar transmit the external environment and objects to the central processor. The central processor processes the data and transmits it to the display module, so that the wearer can observe the panoramic image and dynamics around the vehicle in real time. The central processor uploads the environment and driving data to the cloud server through wireless communication.
[0034] Step 4: When part of the camera is blocked or the system is delayed or freezes, the central processing unit detects that it cannot receive and process data normally, and sends a command to the safety protection module to quickly detach the helmet glasses and restore the wearer's normal field of vision.
[0035] To sum up: before the vehicle drives, the identity authentication module selects the corresponding driving mode according to the wearer's driving identity. During driving, the motion capture module and the eye movement module detect the wearer's head movement and turning as well as the eye's line of sight. The central processing unit sends the actual viewing angle adjustment command to the camera to adjust its angle. The driver issues photo, video or storage commands to the camera through the steering wheel operation interface or voice interaction module, or responds to the alarm issued by the radar alarm module. On the one hand, the central processing unit processes the sensor data and transmits it to the display module, and on the other hand, uploads it to the cloud server through wireless communication. When the system is abnormal, the safety protection module quickly pops open the helmet glasses to restore the wearer's normal field of vision.
[0036] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0037] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.
Claims
1. A vehicle-mounted VR glasses driving system, characterized in that: include: An off-vehicle sensor system, used to collect environmental information around the vehicle; The central processing unit is used to process data from the off-vehicle sensor system, analyze the vehicle's surrounding environment in real time and identify potential dangers, and adjust the information receiving direction of the off-vehicle sensor system according to the wearer's motion information; the VR assisted driving system is used for the vehicle driver to wear and receive and display the 360-degree panoramic image of the vehicle's surrounding environment in real time, and collect the wearer's motion information for interaction; the interactive system is used to receive the driver's instructions, help the driver obtain vehicle driving information and solve corresponding driving problems; the off-vehicle sensor system, the central processing unit, the VR assisted driving system and the interactive system are connected through optical fibers or wires; The cloud server is used to collect and store driving data and environmental data uploaded by all smart driving vehicles, and to exchange data and coordinate driving between vehicles, and is wirelessly connected to the central processor.
2. The vehicle-mounted VR glasses driving system according to claim 1, characterized in that: The vehicle exterior sensing system includes cameras, radars, an environmental adaptation module and an alarm module; the cameras are installed around the vehicle body and can rotate 360° to take photos, videos and store the exterior environment, while uploading the images to the central processor for analysis in real time, and supporting infrared night vision, thermal imaging and optical zoom; The environmental adaptation module can clean and dry the camera to ensure a clear field of view; the radar can detect the position and distance of vehicles coming from the side and rear; the alarm module can issue an alarm when the radar detects objects that are too close around the vehicle body.
3. The vehicle-mounted VR glasses driving system according to claim 2, characterized in that: The VR assisted driving system includes a helmet, a display module, a motion capture module, a protection module and an identity verification module. The helmet is worn on the driver's head and face and serves as a carrier of the display module, the motion capture module, the protection module and the identity verification module. The display module is located in front of the driver's field of vision and is used to display multi-directional images outside the vehicle, radar reminders and alarms, as well as vehicle operating conditions such as vehicle gear position, fuel tank capacity, navigation, vehicle speed, engine speed, temperature, etc.
4. The vehicle-mounted VR glasses driving system according to claim 3, characterized in that: The motion capture module includes a head motion capture unit and a line of sight tracking unit. The head motion capture unit is used to sense the wearer's head turning movement, and the line of sight tracking unit is used to record the wearer's eye movement and gaze direction, and determine the line of sight point through analysis. The head motion capture unit and the line of sight tracking unit transmit the head rotation information and the line of sight point information to the central processing unit, and the central processing unit sends a follow-up instruction to the camera after analysis.
5. The vehicle-mounted VR glasses driving system according to claim 4, characterized in that: The protection module is used to automatically pop open the helmet glasses when the system is abnormal or fails, so as to restore the driver's natural vision; The identity authentication module is used to identify the wearer and select the corresponding driving mode according to different driving identities.
6. The vehicle-mounted VR glasses driving system according to claim 5, characterized in that: The interactive system includes a steering wheel operation interface and a voice interaction module. The steering wheel operation interface is electrically connected to the camera. The voice interaction module is used to recognize and convert the driver's voice commands, switch the camera's viewing angle, and control the camera's zoom, photo taking and video recording through the steering wheel operation interface or the voice interaction module.
7. The vehicle-mounted VR glasses driving system according to claim 6, characterized in that: The driving modes include a driving mode applicable to the main driver and a navigation mode applicable to the co-driver. The driving mode only supports some functions, while the navigation mode supports all functions.
8. An operating method of a vehicle-mounted VR glasses driving system, implemented based on a vehicle-mounted VR glasses driving system according to any one of claims 1 to 7, characterized in that: The following steps are involved: Step 1: Before driving, the wearer is selected as the main driver or the co-driver through the identity verification module. If the wearer is selected as the main driver, the wearer is set to driving mode; if the wearer is selected as the co-driver, the wearer is set to pilot mode. Step 2: During driving, the motion capture module and the eye movement module instantly detect the wearer's head movement and steering as well as the eye gaze point, and transmit the data to the central processing unit. The central processing unit's algorithm converts the data into actual viewing angle adjustment commands, and sends them to the adjustable seat at the bottom of the camera to adjust the angle of the camera to achieve follow-up; Step 3: The driver issues a command to take photos, record videos or store data to the camera through the steering wheel operation interface or voice interaction module. The driver responds to the alarm issued by the radar alarm module. The camera and radar transmit the external environment and objects to the central processor. The central processor processes the data and transmits it to the display module, so that the wearer can observe the panoramic image and dynamics around the vehicle in real time. The central processor uploads the environment and driving data to the cloud server through wireless communication. Step 4: When part of the camera is blocked or the system is delayed or freezes, the central processing unit detects that it cannot receive and process data normally, and sends a command to the safety protection module to quickly detach the helmet glasses and restore the wearer's normal field of vision.
