Vehicle virtual reality system and vehicle-mounted computing unit

Abstract

The invention belongs to the technical field of vehicle-mounted virtual reality equipment, and discloses a vehicle-mounted virtual reality system and a vehicle-mounted computing unit, the system comprises the vehicle-mounted computing unit and one or more VR display equipment; wherein the vehicle-mounted calculation unit can be connected with a vehicle controller and is used for acquiring real-time data of a vehicle-mounted sensor and calculating a real-time motion state and a real-time position of a vehicle; in response to a user interaction signal, determining a virtual carrier and / or a virtual environment, and sending the virtual carrier and / or the virtual environment to the VR display device for display; and in the display process, the motion state and the position of the carrier are updated in real time according to the motion state and the real-time position of the vehicle. The motion state of the user in the virtual environment and the motion state of the real vehicle are synchronized in real time, so that the possible motion sickness symptom is effectively relieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle-mounted virtual reality equipment, and in particular relates to a vehicle-mounted virtual reality system and a vehicle-mounted computing unit. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Metaverse is a virtual world that is constructed by humans using digital technology. It is a reflection of the real world or transcends the real world. It can interact with the real world and has a digital living space for a new social system. Metaverse has been applied in the fields of vehicle development, design verification, logistics management and brand marketing. Some car companies have also begun to explore how to combine the Metaverse experience with the user's riding experience.

[0004] Existing car virtual reality systems focus on enhancing the real world to enrich the user experience. For example, the surrounding environment data is obtained through sensors installed around the car body, and reality augmentation is performed based on the surrounding environment data. Although the experience can be changed by modifying the style of augmented reality, this method relies on the real scene around the vehicle and has poor flexibility. If augmented reality is not based on real scenes, and virtual scenes are used completely, such as existing indoor virtual reality equipment, some users are prone to motion sickness. If a completely virtual scene is applied to a moving vehicle, the reality and the sense of space are more likely to be inconsistent, which will aggravate this phenomenon. Summary of the invention

[0005] In order to overcome the deficiencies of the above-mentioned prior art, the present invention provides a vehicle-based virtual reality system and a vehicle-mounted computing unit, a VR / metaverse experience device specifically used for an in-vehicle environment, and supporting VR content for users to experience as passengers while driving.

[0006] To achieve the above-mentioned purpose, the first aspect of the present invention provides a vehicle virtual reality system, comprising an on-board computing unit and one or more VR display devices; wherein the on-board computing unit can establish a connection with a vehicle controller to obtain real-time data from on-board sensors and calculate the real-time motion state and real-time position of the vehicle; in response to user interaction signals, determine a virtual vehicle and / or a virtual environment, and send them to the VR display device for display; and during the display process, update the motion state and position of the vehicle in real time according to the motion state and real-time position of the vehicle, respectively.

[0007] In some embodiments, the VR display device includes a display module, a posture sensor and an interaction sensor. The posture sensor is used to detect the user's head movement and send it to the vehicle-mounted computing unit; the interaction sensor includes at least two eye tracking recognition sensors and at least one structured light sensor, which are used to identify the user's eye position and user gestures, respectively, and send them to the vehicle-mounted computing unit.

[0008] In some embodiments, the on-board computing unit has a built-in temperature monitor connected to the on-board air conditioning controller; at the same time, an independent air duct is provided between the air conditioner and the on-board computing unit, and whether to cool the on-board computing unit alone is controlled according to the on-board computing unit temperature output by the temperature monitor.

[0009] In some embodiments, a plurality of connecting members are evenly arranged below the vehicle-mounted computing unit for connecting to the vehicle chassis, and a shock absorbing device is also arranged in the connecting member.

[0010] A second aspect of the present invention provides a vehicle-mounted computing unit, which is applied to the vehicle-mounted virtual reality system, and the vehicle-mounted computing unit is configured to execute:

[0011] Acquire real-time data from vehicle-mounted sensors and calculate the real-time motion state and real-time position of the vehicle;

[0012] In response to an initialization interaction signal from a user, selecting at least one of a virtual vehicle and a virtual environment;

[0013] In response to the user's confirmation operation instruction, the specified virtual environment data and virtual vehicle are acquired and sent to the VR display device for display;

[0014] During the display process, the motion state and position of the vehicle are updated in real time according to the motion state and real-time position of the vehicle respectively.

[0015] In some embodiments, vehicle navigation data is also obtained, the vehicle navigation data including map data and navigation routes; and the virtual environment data is transformed to the same scale as the map data according to the scale of the map data and the scale factor of the virtual environment data.

[0016] In some embodiments, the map data and the navigation route are combined to determine the route environment in which the virtual vehicle is running according to the selected virtual vehicle and the virtual environment in which it is located.

[0017] In some embodiments, for each virtual environment, a plurality of connectable virtual scene units are preset. If the virtual vehicle is a ground transportation vehicle, the plurality of virtual scene units are connected together outside the route environment.

[0018] In some embodiments, if the virtual vehicle is a ground transportation vehicle, the width of the route environment is greater than or equal to the actual road corresponding to the navigation route.

[0019] In some embodiments, during the display process, the posture parameters and eye movement parameters fed back by the VR display device are obtained in real time to determine the user's line of sight direction, and the display range of the virtual environment in the screen is updated according to the line of sight direction.

[0020] One or more of the above technical solutions effectively alleviate possible motion sickness symptoms by synchronizing the user's motion state in the virtual environment with the motion state of the real vehicle in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0022] Figure 1 This is a diagram of the architecture of a vehicle virtual reality system in an embodiment of the present invention;

[0023] Figure 2 This is a schematic diagram of the structure of a connecting piece in an embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram of the appearance of a VR device in an embodiment of the present invention;

[0025] Figure 4 It is a schematic diagram of the overall interaction logic of the device in the embodiment of the present invention;

[0026] Figure 5 The present invention is a flowchart of a method executed by a vehicle-mounted computing unit in an embodiment of the present invention.

[0027] In the figure, 1-1, vehicle-mounted computing unit, 2-1, in-vehicle air conditioning, 2-2, air duct, 2-3, air conditioning controller, 3-1, shock absorber, 4-1, vehicle-mounted sensor, 5-1, VR display device. DETAILED DESCRIPTION

[0028] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0029] In the description of the embodiments of the present application, the term “including” and similar terms should be understood as open inclusion, that is, “including but not limited to.” The term “based on” should be understood as “based at least in part on.”

[0030] Figure 1 The framework diagram of the vehicle virtual reality system provided by one or more embodiments of the present invention is shown. The system includes a vehicle-mounted computing unit and one or more VR display devices. The vehicle-mounted computing unit can establish a connection with the vehicle controller, and is used to obtain real-time data from the vehicle-mounted sensors, calculate the real-time motion state and real-time position of the vehicle; determine the virtual vehicle and / or virtual environment in response to the user interaction signal, and send it to the VR display device for display; and during the display process, the motion state and position of the vehicle are updated in real time according to the motion state and real-time position of the vehicle respectively; the VR display device includes a display module and an interactive sensor, which is used to display the picture, and receive the user interaction signal, and send it to the vehicle-mounted computing unit.

[0031] Among them, the on-board sensors are used to monitor the movement state of the vehicle, including sensors such as accelerometers, gyroscopes and magnetometers, which are used to measure information such as the vehicle's speed, acceleration and displacement in space, so as to realize real-time movement state and positioning of the vehicle and other information.

[0032] As an example, the on-board computing unit and the VR display device are connected by wire to realize image and data transmission. Specifically, the on-board computing unit is provided with one or more interfaces for connecting one or more VR display devices, which can meet the needs of two people sitting in a single-row rear seat or multiple people sitting in multiple rows of rear seats, for example, connected by a USB-Type C data cable. Depending on the vehicle model, the on-board computing unit can usually be arranged in the co-pilot position (the co-pilot seat needs to be removed in advance to install the fixed base of the on-board computing unit), the middle seat in the rear row of the vehicle, the trunk or other spaces in the vehicle. The VR device is placed in the storage space behind the rear seats. When in use, the user turns on the on-board computing unit and connects the VR device to the on-board computing unit by wire, operates the on-board computing unit through gestures / eye movements, etc., and starts the VR / metaverse experience. At the same time, the on-board computing unit is connected to the on-board sensor to obtain information such as the vehicle's motion status and position in real time. Finally, the user experiences the virtual display content generated by the on-board computing unit by wearing VR. The overall structure of the system is as follows Figure 1 shown.

[0033] The overall shape of the VR display device is similar to the mainstream VR / MR devices on the market. Since the image generation and data processing are completed on the vehicle-mounted computing unit, the VR display device mainly undertakes image display and interaction functions. The vehicle-mounted computing unit and the VR display device are connected by wire, which can meet the data transmission and power supply requirements at the same time. Figure 3 As shown, the VR display device includes a display screen, a posture sensor and several different types of interactive sensors.

[0034] The posture sensor includes a gyroscope sensor for detecting the relative position of the VR display device and the vehicle; the interaction sensor includes at least two eye tracking and recognition sensors and at least one structured light sensor, which are used to identify the user's eye position and user gestures, respectively. It can be understood that when the rear passengers wear the VR display device, based on the gyroscope sensor, the passenger's head posture can be determined, and combined with the eye position identified by the eye tracking and recognition sensor, the user's actual line of sight direction can be determined. Based on this, on the one hand, options can be selected through eye movement and line of sight dwell time, and on the other hand, the display perspective of the virtual environment data in the screen can be adjusted during the display of the virtual environment.

[0035] The relative position between the VR display device and the vehicle is recorded as position information 1, the vehicle positioning data based on the vehicle sensor sent by the vehicle terminal / external device terminal is recorded as position information 2, and the map data and navigation information sent by the vehicle terminal / external device terminal (sent by the vehicle inertial navigation or external navigation device) are output as position information 3; these three types of position information are calculated together to obtain the relative relationship between the user wearing VR and the vehicle and the external environment, and finally generate VR experience content for the user to experience, making the VR internal environment and the external vehicle state completely consistent. The interaction logic between the VR device and the vehicle computing unit is as follows: Figure 4 shown.

[0036] The vehicle-mounted computing unit is similar to a home desktop computer in overall shape, and its specific performance meets the requirements of the in-vehicle working environment. A heat dissipation device is also provided inside the vehicle-mounted computing unit, and a shock absorbing device is also provided between the vehicle-mounted computing unit and the vehicle floor.

[0037] The heat dissipation device adopts liquid cooling. As a backup temperature control measure, in addition to the built-in heat dissipation device, the on-board computing unit has a built-in temperature monitor, which is connected to the air conditioning controller in the car. At the same time, an independent air duct is provided between the air conditioning and the on-board computing unit. One end of the air duct is connected to the inside of the on-board computing unit, and the other end is connected to the air conditioning duct in the car. A switch valve is provided on the air duct to control whether to cool the on-board computing unit alone according to the temperature control of the on-board computing unit output by the temperature monitor. When the temperature monitor detects that the internal temperature of the on-board computing unit is higher than the maximum operating temperature of the device, the temperature monitor sends a signal to the air conditioning controller to control the switch valve to open and cool the on-board computing unit. The air duct for cooling the on-board computing unit is independently distinguished from the air duct of the entire vehicle and will not affect each other. The cold air generated by the air conditioner is introduced into the on-board computing unit to assist the on-board computing unit in dissipating heat in a high temperature environment.

[0038] In order to support the on-board computing unit and provide shock absorption for the on-board computing unit, and prevent the bumps generated during the movement of the vehicle from damaging the internal components of the on-board computing unit, in some embodiments, a plurality of connectors are evenly arranged under the on-board computing unit for connecting to the vehicle chassis, and a shock absorbing device is also provided in the connector. As an example, there are 4 connectors. On the one hand, the connector realizes the connection between the on-board computing unit and the chassis, and the shock absorbing device therein can also prevent the internal electronic components of the on-board computing unit from being damaged due to the bumps of the vehicle. Specifically, the shock absorbing device includes several shock absorbing silicone blocks, and shock absorbing springs that can move in the vertical direction are provided under the shock absorbing silicone. When kinetic energy is transmitted from the bottom of the vehicle to the on-board computing unit connector, several shock absorbing springs will be the first to buffer, and the remaining kinetic energy will be further absorbed by the shock absorbing silicone blocks. The overall structure is as follows Figure 2 shown.

[0039] At the same time, the car VR / metaverse experience equipment also includes a VR experience content based on the real movement status and navigation planning of the vehicle. By sending multiple position information, map data, navigation route and other information to the on-board computing unit, the on-board computing unit uses it as a parameter to generate a corresponding VR environment for the user to experience.

[0040] As a specific implementation, the vehicle-mounted computing unit is configured as follows:

[0041] Step 1: Acquire real-time data from vehicle-mounted sensors and calculate the real-time motion state and real-time position of the vehicle.

[0042] Step 2: Obtain vehicle navigation data, which includes map data and navigation routes.

[0043] Step 3: In response to the user's initialization interaction signal, select at least one of the virtual vehicle and the virtual environment.

[0044] The virtual vehicle can be a vehicle on the ground, such as various vehicles, or in the air, such as a cable car, an airplane, or even a cloud that does not exist in reality. The virtual environment is constructed through a model, and can be a city, a village, an ocean, a desert, etc.

[0045] Step 4: Combine the map data and the navigation route, and determine the route environment for the virtual vehicle to operate according to the selected virtual vehicle and the virtual environment.

[0046] In order to make the operation of the virtual vehicle in the virtual environment consistent with that in the real environment, the spatial coordinate system of the virtual environment needs to be matched to the real coordinate system. Specifically, a spatial coordinate system with a scale factor is established for each preset virtual model, and the virtual model is transformed to the same scale as the map data according to the scale of the acquired map data.

[0047] As an example, when the vehicle is a vehicle, the route environment is a road, and the parameters such as the width of the road can be consistent with the actual road corresponding to the navigation route, or the road can be widened with the actual road corresponding to the navigation route as the center; when the vehicle is a boat, the route environment is a river, and the parameters such as the width of the river can be consistent with the actual road corresponding to the navigation route, or the road can be widened with the actual road corresponding to the navigation route as the center. Similarly, when the vehicle is a cable car, the route environment is a cable, and the extension direction of the cable is consistent with the navigation route.

[0048] Different distances correspond to different map area sizes. Longer distances require larger model spaces to provide support. To meet the needs of different space sizes and to reduce the sense of scene fragmentation of the virtual environment caused by the roads and rivers on which the virtual vehicle runs, in some embodiments, for each virtual environment, multiple virtual scene units that can be spliced ​​can be preset. After determining the route environment of the virtual vehicle, if the virtual vehicle is a ground transportation vehicle, multiple virtual scene units are spliced ​​outside the route environment. In addition, if the size of the virtual environment space obtained after splicing does not meet the requirements, the repeated virtual scene units can be spliced. Based on this, virtual scenes that adapt to different route environments can be quickly obtained.

[0049] Step 5: In response to the user's confirmation operation instruction, the specified virtual environment data, vehicle and route environment data are obtained and sent to the VR display device for display.

[0050] Step 6: During the display process, the motion state and position of the vehicle are updated in real time according to the motion state and real-time position of the vehicle respectively; at the same time, the posture parameters and eye movement parameters fed back by the VR display device are obtained in real time to determine the user's line of sight direction, and the display range of the virtual environment in the screen is updated according to the line of sight direction.

[0051] To further enrich the user experience, virtual elements of different styles can be set, such as horror style, fantasy style, fairy tale style, etc. In response to the user's style setting instructions, during the display process, virtual elements of corresponding styles appear randomly or according to certain display rules in the virtual environment in the direction of the user's sight.

[0052] Compared with the existing method of improving the user experience by enhancing the vehicle's surrounding environment, the present application focuses more on the movement state of the user when experiencing the virtual environment, making it consistent with the actual operating state of the vehicle, thereby avoiding motion sickness symptoms caused by the body's spatial positioning disorder, and improving the user experience on this basis; at the same time, through scaling, the scale of the virtual scene is transformed to be consistent with the real world based on real map data, further synchronizing the spatial movement of the senses and the body.

[0053] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the vehicle-mounted computing unit or terminal, the process or function described in the embodiment of the present application is generated in whole or in part. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, vehicle-mounted computing unit or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, vehicle-mounted computing unit or data center. The computer-readable storage medium can be any available medium that can be accessed by the vehicle-mounted computing unit or terminal, or a data storage device such as a vehicle-mounted computing unit, a data center, etc. that includes one or more available media integrated. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape, etc.), an optical medium (such as a digital video disk (DVD), etc.), or a semiconductor medium (such as a solid state drive, etc.).

[0054] In addition, although each operation is described in a specific order, this should be understood as requiring such operation to be performed in the specific order shown or in a sequential order, or requiring that all illustrated operations should be performed to obtain desired results. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the application. Some features described in the context of a separate embodiment can also be implemented in a single implementation in combination. On the contrary, the various features described in the context of a single implementation can also be implemented in multiple implementations individually or in any suitable sub-combination mode.

[0055] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A virtual reality system for a vehicle, characterized in that: It includes an on-board computing unit and one or more VR display devices; wherein the on-board computing unit can establish a connection with a vehicle controller to obtain real-time data from on-board sensors and calculate the real-time motion state and real-time position of the vehicle; in response to a user interaction signal, a virtual vehicle and / or a virtual environment is determined and sent to the VR display device for display; and during the display process, the motion state and position of the vehicle are updated in real time according to the motion state and real-time position of the vehicle, respectively.

2. The vehicle virtual reality system according to claim 1, characterized in that: The VR display device includes a display module, a posture sensor and an interaction sensor. The posture sensor is used to detect the user's head movement and send it to the vehicle-mounted computing unit; the interaction sensor includes at least two eye tracking recognition sensors and at least one structured light sensor, which are used to recognize the user's eye position and user gestures respectively, and send them to the vehicle-mounted computing unit.

3. The vehicle virtual reality system according to claim 1, characterized in that: The vehicle-mounted computing unit has a built-in temperature monitor, which is connected to the vehicle-mounted air-conditioning controller; at the same time, an independent air duct is provided between the air-conditioning and the vehicle-mounted computing unit, and a switch valve is provided on the air duct. When the temperature monitored by the temperature monitor exceeds a set threshold, the switch valve is controlled to open to cool the vehicle-mounted computing unit.

4. The vehicle virtual reality system according to claim 1, characterized in that: A plurality of connecting members are evenly arranged below the vehicle-mounted computing unit for connecting with the vehicle chassis, and a shock absorbing device is also arranged in the connecting member.

5. A vehicle-mounted computing unit, applied to the vehicle-mounted virtual reality system according to any one of claims 1 to 4, characterized in that: The on-board computing unit is configured to perform: Acquire real-time data from vehicle-mounted sensors and calculate the real-time motion state and real-time position of the vehicle; In response to an initialization interaction signal from a user, selecting at least one of a virtual vehicle and a virtual environment; In response to the user's confirmation operation instruction, the specified virtual environment data and virtual vehicle are acquired and sent to the VR display device for display; During the display process, the motion state and position of the vehicle are updated in real time according to the motion state and real-time position of the vehicle respectively.

6. The vehicle-mounted computing unit according to claim 5, characterized in that: The vehicle navigation data is also acquired, the vehicle navigation data including map data and navigation routes; and the virtual environment data is transformed to the same scale as the map data according to the scale of the map data and the scale factor of the virtual environment data.

7. The vehicle-mounted computing unit according to claim 6, characterized in that: The map data and the navigation route are combined to determine the route environment in which the virtual vehicle operates according to the selected virtual vehicle and the virtual environment in which it is located.

8. The vehicle-mounted computing unit according to claim 7, characterized in that: For each virtual environment, a plurality of virtual scene units that can be spliced ​​are preset. If the virtual vehicle is a ground transportation tool, the plurality of virtual scene units are spliced ​​together outside the route environment.

9. The vehicle-mounted computing unit according to claim 8, characterized in that: If the virtual vehicle is a ground transportation vehicle, the width of the route environment is greater than or equal to the actual road corresponding to the navigation route.

10. The vehicle-mounted computing unit according to any one of claims 6 to 9, characterized in that: During the display process, the posture parameters and eye movement parameters fed back by the VR display device are obtained in real time to determine the user's line of sight direction, and the display range of the virtual environment in the screen is updated according to the line of sight direction.

Images