A method for XR simulation based on autonomous driving

Abstract

The present invention belongs to the field of VR device application technology, and discloses an XR simulation real-scene simulation method based on autonomous driving, comprising the following steps: step S001, preparing an intelligent driving computing platform, wherein the intelligent driving computing platform includes an intelligent driving computing system and vehicle data; step S002, in the intelligent driving computing platform, adding an XR system, and when the vehicle is started or stationary, making the computing resources of the overall intelligent computing platform idle, the XR system is in a dormant state; step S003, through the installation of the XR system, the XR system and the autonomous driving computing system carry heterogeneous operating systems on the same hardware platform, thereby realizing the integration of computing units of the XR system and the autonomous driving computing system and sharing of computing power by using hypervisor virtualization.

Description

Technical Field

[0001] The present invention belongs to the technical field of VR equipment application, and specifically relates to an XR simulation real scene simulation method based on autonomous driving. Background Art

[0002] At present, most XR devices use high-configuration PCs for graphics calculations in highly simulated rendering games or scenes, but it is unrealistic to deploy computers in a car environment. In order to perfectly experience the driving control and game immersion in the car, such as driving simulation games through the vehicle steering wheel, accelerator and brakes, high-simulation real-life simulation is required, and general all-in-one machines cannot run at a high frame rate; this kind of simulation game is generally run when the vehicle is stationary, and the autonomous driving computing system does not work when it is stationary. Therefore, the XR system can be installed on the autonomous driving platform, and the system can be activated and high-quality games in the system can be run when the vehicle is parked. The system can access the vehicle signal to control the XR simulation game, so that the game rendering effect, frame rate, etc. are very good. The reused autonomous driving computing platform also reduces the cost of the XR solution in terms of rendering and computing costs. Summary of the invention

[0003] The purpose of the present invention is to solve the technical problems in the background technology and provide an XR simulation real-scene simulation method based on autonomous driving, which enables passengers to perfectly experience the driving control and game immersion in the car.

[0004] To achieve the above object, the present invention provides the following technical solution: an XR simulation method based on autonomous driving, comprising the following steps:

[0005] Step S001, preparing an intelligent driving computing platform, wherein the intelligent driving computing platform includes an intelligent driving computing system and vehicle data;

[0006] Step S002, in the intelligent driving computing platform, the XR system is added, and when the vehicle is started or stationary, the computing resources of the entire intelligent driving computing platform are idle, and the XR system is in a dormant state;

[0007] Step S003, by carrying the XR system, the XR system and the autonomous driving computing system carry heterogeneous operating systems on the same hardware platform, thereby realizing the integration of computing units and computing power sharing of the XR system and the autonomous driving computing system by using hypervisor virtualization;

[0008] Step S004: When the user needs to experience the XR device, he activates it through the button of the car computer application or the vehicle control button. At this time, the autonomous driving computing system and the XR system coexist, and the XR system enters the running state. At this time, the entire system will dynamically adjust the computing resources of the autonomous driving computing system and the XR system.

[0009] Among them, XR devices can allocate computing resources from the XR system to meet the high-simulation gaming immersion experience of XR devices. In addition, the computing resources allocated by the entire system to the autonomous driving computing system are used to ensure the driving safety of autonomous driving.

[0010] Preferably, in step S004, when the vehicle is stationary and not started, after the XR system is running, the entire system will dynamically adjust the CPU, GPU, memory and other resources to allocate more computing resources to the XR system; the minimum GPU requirement of 3TFLOPS for meeting the high-simulation gaming experience, at this time, the XR device is mainly for the driver to experience.

[0011] Preferably, in step S004, after the vehicle is started, the entire system adjusts the resource allocation, the XR system GPU is reduced to 1TFLOPS, and the remaining resources are used for the autonomous driving system. At this time, the XR device is mainly used for the passenger experience. In order to ensure the driving safety of autonomous driving, the overall allocation is biased towards the autonomous driving system.

[0012] Preferably, the intelligent driving computing platform is configured as a core computing platform of an intelligent car or an autonomous driving car, including an autonomous driving or control ADCU.

[0013] Preferably, the XR system refers to a running VR operating system or AR operating system, and the VR operating system or AR operating system contains relevant content such as VR games or videos.

[0014] Preferably, the vehicle data is set to some necessary vehicle operation data for controlling the operation status of the XR system, including vehicle speed, brake, and throttle related data.

[0015] Preferably, the XR device is configured as XR glasses, which mainly include display and optical parts, wherein power supply and computing resources are borne by the intelligent driving computing platform.

[0016] Preferably, the XR device and the XR system are connected via wireless or wired streaming.

[0017] Preferably, the hypervisor includes hardware for intelligent driving computing, including GPU, CPU, and memory, and multiple systems can be run on the platform through virtualization.

[0018] Preferably, the operating system of the XR device and the intelligent driving computing system run in the Guest VM respectively, and the two systems run simultaneously when they are turned on.

[0019] Compared with the prior art, the present invention has the following beneficial effects:

[0020] The present invention increases the installation of an XR system in the intelligent driving computing platform and makes the computing resources of the entire intelligent driving computing platform idle when the vehicle is started or stationary, so that the on-board XR solution can be realized by independently providing a computing power module or by linking an all-in-one machine with the vehicle, that is, effectively utilizing the idle intelligent driving computing platform when stationary; making the XR device experience better and the cost effectively controlled, thereby realizing the user's driving control sense and gaming immersion. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the overall method flow of the present invention; DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] See also Figure 1 As shown, the present invention provides a technical solution: an XR simulation method based on autonomous driving, comprising the following steps:

[0024] Step S001, preparing an intelligent driving computing platform, wherein the intelligent driving computing platform includes an intelligent driving computing system, an XR system, vehicle data, and XR equipment;

[0025] Step S002, in the intelligent driving computing platform, the XR system is added, and when the vehicle is started or stationary, the computing resources of the entire intelligent driving computing platform are idle, and the XR system is in a dormant state;

[0026] Step S003, by carrying the XR system, the XR system and the autonomous driving computing system carry heterogeneous operating systems on the same hardware platform, thereby realizing the integration of computing units and computing power sharing of the XR system and the autonomous driving computing system by using hypervisor virtualization;

[0027] Step S004: When the user needs to experience the XR device, the user activates it through the button of the vehicle application or the vehicle control button. At this time, the autonomous driving computing system and the XR system coexist, and the XR system enters the running state. At this time, the entire system will dynamically adjust the computing resources of the autonomous driving computing system and the XR system.

[0028] Among them, XR devices can allocate computing resources from the XR system to meet the high-simulation gaming immersion experience of XR devices. In addition, the computing resources allocated to the autonomous driving computing system by the entire system are used to ensure the driving safety of autonomous driving and improve the driver's sense of control.

[0029] Furthermore, in step S004, when the vehicle is stationary and not started, after the XR system is running, the entire system will dynamically adjust the CPU, GPU, memory and other resources to allocate more computing resources to the XR system; the minimum GPU requirement for meeting the high-simulation gaming experience is 3TFLOPS. At this time, the XR device is mainly for the driver to experience.

[0030] Furthermore, in step S004, after the vehicle is started, the entire system adjusts resource allocation, the XR system GPU is reduced to 1TFLOPS, and the remaining resources are used for the autonomous driving system. At this time, the XR device is mainly used for passenger experience. In order to ensure the driving safety of autonomous driving, the overall allocation is biased towards the autonomous driving system.

[0031] Furthermore, the entire system is an intelligent driving computing platform, which includes an automatic driving system (assisted driving system) and an XR system. The XR system is turned on through the vehicle computer. Whether it can be turned on depends on the current vehicle status. When the vehicle moves, the XR system exits or limits the operation of computing resources.

[0032] Furthermore, the vehicle data and navigation data are transmitted to the XR device through one or more of the vehicle's diagnostic interface, Bluetooth, wireless hotspot interface, and USB interface, thereby enhancing the user's simulation of the real scene, and at the same time, the XR device can present XR video content and XR game content. Further, the XR device refers to VR glasses, MR glasses, etc. that can be used in the car.

[0033] Furthermore, the intelligent driving computing platform is set as the core computing platform of an intelligent car or an autonomous driving car, including an autonomous driving or control ADCU.

[0034] Furthermore, the XR system refers to a running VR operating system or AR operating system, and the VR operating system or AR operating system contains related content such as VR games or videos.

[0035] Furthermore, the vehicle data is set to be some necessary vehicle operation data for controlling the operation status of the XR system, including vehicle speed, brake, and throttle related data.

[0036] Furthermore, the XR device is configured as XR glasses, which mainly include display and optical parts, wherein power supply and computing resources are borne by the intelligent driving computing platform.

[0037] Furthermore, the XR device and the XR system are connected via wireless or wired streaming.

[0038] Furthermore, the hypervisor includes hardware for intelligent driving computing, including GPU, CPU, and memory, and multiple systems can be run on the platform through virtualization;

[0039] The multi-system approach uses hypervisor virtualization technology. The virtualization (Hypervisor) solution provides the flexibility to carry heterogeneous operating systems on the same hardware platform, while achieving good high reliability and fault control mechanisms to ensure the secure isolation between mission-critical, hard real-time applications and general-purpose, untrusted applications, thereby realizing the integration of on-board computing units and sharing of computing power.

[0040] Hypervisor technology has been widely used in smart cockpits. This solution extends it to the domain of smart driving. The computing hardware based on smart driving, including GPU, CPU, memory, etc., can run multiple systems on the platform through virtualization. This solution is to run the XR operating system and the smart driving operating system in the Guest VM respectively, and both systems will run when they are turned on.

[0041] Furthermore, the operating system of the XR device and the intelligent driving computing system run in the Guest VM respectively, and the two systems run simultaneously when they are turned on.

[0042] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An XR simulation method based on autonomous driving, characterized in that: The following steps are involved: Step S001, preparing an intelligent driving computing platform, wherein the intelligent driving computing platform includes an intelligent driving computing system and vehicle data; Step S002, in the intelligent driving computing platform, the XR system is added, and when the vehicle is started or stationary, the computing resources of the entire intelligent computing platform are idle, and the XR system is in a dormant state; Step S003, by carrying the XR system, the XR system and the autonomous driving computing system carry heterogeneous operating systems on the same hardware platform, thereby realizing the integration of computing units and computing power sharing of the XR system and the autonomous driving computing system by using hypervisor virtualization; Step S004: When the user needs to experience the XR device, he activates it through the button of the car computer application or the vehicle control button. At this time, the autonomous driving computing system and the XR system coexist, and the XR system enters the running state. At this time, the entire system will dynamically adjust the computing resources of the autonomous driving computing system and the XR system. XR devices can allocate computing resources from the XR system to meet the high-simulation game immersion experience of XR devices. In addition, the computing resources allocated to the autonomous driving computing system by the entire system are used to ensure the driving safety of autonomous driving and meet the user's driving control experience. In step S004, when the vehicle is stationary and not started, after the XR system is running, the entire system will dynamically adjust the CPU, GPU, and memory resources, and allocate more computing resources to the XR system; the minimum GPU requirement for meeting the high-simulation gaming experience is 3TFLOPS. At this time, the XR device is mainly for the driver to experience; In step S004, after the vehicle is started, the entire system adjusts resource allocation, the XR system GPU is reduced to 1 TFLOPS, and the remaining resources are used for the autonomous driving system. At this time, the XR device is mainly used for passenger experience. In order to ensure the driving safety of autonomous driving, the overall allocation is biased towards the autonomous driving system. The intelligent driving computing platform is set as the core computing platform of the intelligent car or the autonomous driving car, including the autonomous driving or control ADCU; The vehicle data is set to be some necessary vehicle operation data for controlling the operation status of the XR system, including vehicle speed, brake, and throttle related data.

2. The XR simulation method based on autonomous driving according to claim 1, characterized in that: The XR system refers to a running VR operating system or AR operating system, and the VR operating system or AR operating system contains VR game or video related content.

3. The XR simulation method based on autonomous driving according to claim 1, characterized in that: The XR device is XR glasses, which mainly include display and optical parts, and the power supply and computing resources are borne by the intelligent driving computing platform.

4. The XR simulation method based on autonomous driving according to claim 1, characterized in that: The XR device and the XR system are connected via wireless or wired streaming.

5. The XR simulation method based on autonomous driving according to claim 1, characterized in that: The hypervisor uses hardware for intelligent driving computing, including GPU, CPU, and memory, and can run multiple systems on the platform through virtualization.

6. The XR simulation method based on autonomous driving according to claim 1, characterized in that: The operating system of the XR device and the intelligent driving computing system run in the Guest VM respectively, and the two systems run simultaneously when they are turned on.

Images