Simulation scene real vehicle injection-based automatic driving virtual-real fusion test system and method

Abstract

The invention relates to an automatic driving virtual-real fusion test system and method based on simulation scene real vehicle injection. The system comprises a cloud control platform end and a real vehicle end. The cloud control platform end comprises a map module and a first interactive scene generation module, the real vehicle end comprises a scene injection machine, a domain controller and a real vehicle, and a second interactive scene generation module is embedded in the scene injection machine; the testing process comprises the steps that the cloud control platform end issues a lane-level task path generated by the map module and an interactive scene which is generated by the first interactive scene generation module and contains background traffic flow core data to the scene injection machine; the scene injection machine forwards the lane-level task path and the interactive scene to a domain controller; and the domain controller performs algorithm planning according to the background traffic flow core data, sends a motion instruction to a real vehicle for tracing, and returns sensing information to the scene injection machine. Compared with the prior art, the method has the advantages of high test reliability and high efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous driving testing, and in particular to an autonomous driving virtual-reality fusion testing system and method based on real vehicle injection into a simulation scene. Background Art

[0002] As autonomous driving technology continues to improve, the requirements for its safety, reliability and performance are becoming increasingly higher, and the development of matching test and evaluation technologies is urgent. The scenarios faced by autonomous driving are almost infinite and difficult to generalize. The typical scenario testing methods used in the traditional automotive testing field can no longer meet the testing needs of autonomous driving. Studies have shown that more than one billion kilometers of road testing are required to statistically prove that autonomous driving is safer and more reliable than human driving.

[0003] However, the mileage-based testing method has the disadvantages of long testing cycle, low efficiency and high cost. For the commercialization of autonomous driving vehicles, it is obviously not enough to rely solely on mileage testing methods. High-level scenario simulation testing and performance evaluation are indispensable. During the autonomous driving test process, about 90% of the tests are conducted through virtual simulation, while the remaining 9% of the tests are conducted in closed venues, and only about 1% of the tests are conducted on open roads.

[0004] The three mainstream testing methods for autonomous driving are as follows:

[0005] (1) Open road test: A mileage test using a real vehicle as the test terminal, where the data collected during the road test is uploaded to the cloud for analysis and evaluation. Chinese patent application CN116151650A discloses an open road test and evaluation system for vehicle autonomous driving functions based on vehicle-cloud interaction. Disadvantages: uncontrollable road conditions, low density of effective information, high cost, and poor repeatability.

[0006] (2) Simulation closed-loop test: Chinese patent application CN116151650A discloses a single-machine in-loop simulation and a cloud-based large-scale concurrent simulation solution, where the single-machine in-loop simulation is typically represented by hardware-in-loop and vehicle-in-loop that are suitable for the V process, while the cloud-based large-scale concurrent simulation is suitable for the model development and code development stages of the V process. Disadvantages: The digitally constructed vehicle dynamics model is difficult to accurately reproduce the vehicle dynamics model, which reduces the test accuracy.

[0007] (3) Closed-field testing: The scenarios are controllable and highly safe, but it cannot cover all scenarios on real roads and more complex mixed traffic flow scenarios, and has certain limitations for real scenarios.

[0008] Vehicle-level testing of autonomous vehicles faces challenges such as high testing costs, low scenario coverage, and lack of a complete tool chain. The above-mentioned single testing method can hardly ensure the testing effect. Summary of the invention

[0009] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide an autonomous driving virtual-reality fusion test system and method based on the injection of simulation scenes into real vehicles, which can not only perform performance tests on real autonomous driving vehicles or domain controllers, but also reduce the cost of interactive testing between real vehicles, thereby speeding up the autonomous driving test process.

[0010] The purpose of the present invention can be achieved by the following technical solutions:

[0011] According to a first aspect of the present invention, there is provided an autonomous driving virtual-reality fusion test system based on simulation scene real vehicle injection, comprising a cloud control platform end and a real vehicle end; the cloud control platform end comprises a map module and a first interactive scene generation module, the real vehicle end comprises a scene injection machine, a domain controller and a real vehicle, and the scene injection machine is embedded with a second interactive scene generation module;

[0012] The system's autonomous driving virtual-reality fusion test process includes:

[0013] The cloud control platform packages the lane-level task path generated by the map module and the interactive scene containing the background traffic core data generated by the first interactive scene generation module and sends them to the scene injection machine of the real vehicle.

[0014] The scene injector directly transfers the lane-level task path and interactive scene to the domain controller;

[0015] The scene injection machine generates background traffic core data based on the real vehicle data sent back by the domain controller, and the domain controller plans, tracks and updates based on the background traffic core data, and sends back the real vehicle data to the scene injection machine; the scene injection machine and the domain controller send and receive data at the same time.

[0016] Preferably, the scene injection machine also includes: generating background traffic core data in real time according to the actual vehicle data transmitted back by the domain controller, and sending it to the domain controller.

[0017] Preferably, 5G communication is adopted between the cloud control platform and the real vehicle.

[0018] Preferably, the scene injection machine on the real vehicle side communicates with the domain controller via Ethernet.

[0019] Preferably, the vehicle core data includes vehicle position and vehicle speed.

[0020] Preferably, the real vehicle end also includes a vehicle-mounted unit for collecting the autonomous driving vehicle's real-time perception information, real-time decision information, and real-time driving parameters of the vehicle.

[0021] Preferably, the autonomous driving path planning model carried in the domain controller plans the actual vehicle path and speed according to the reference path and speed of the main vehicle, and generates motion instructions.

[0022] Preferably, the cloud control platform is also provided with a visualized digital twin module for performing visualized digital twin reproduction based on the sensor data transmitted back from the real vehicle.

[0023] Preferably, the cloud control platform is also provided with a test report generation module for generating a test report according to the vehicle core data transmitted back by the real vehicle.

[0024] According to a second aspect of the present invention, a method for testing the virtual-reality fusion of autonomous driving based on real vehicle injection into a simulation scene is provided. The above-mentioned system is applied, and the method comprises:

[0025] Initialize the system, set the reference path and reference speed of the main vehicle, as well as the reference path and reference speed data of the background vehicle on the cloud control platform, and package the data and send it to the scene injection machine on the real vehicle side;

[0026] The scene injection machine on the real vehicle receives the reference path and reference speed of the main vehicle issued by the cloud control platform and forwards it to the domain controller;

[0027] After obtaining the reference path and reference speed of the main vehicle, the domain controller calculates the motion command and sends it to the real vehicle; the real vehicle tracks and returns the perceived vehicle core data in real time and transmits it back to the cloud control platform through the domain controller. Compared with the prior art, the present invention has the following beneficial effects:

[0028] (1) Compared with the previous closed road tests, the virtual-reality fusion autonomous driving simulation test method designed by the present invention for real vehicle injection for simulation testing can set up a more complex interactive environment; compared with the simulation test, the real vehicle injection test process of the present invention specifically places the autonomous driving algorithm into the vehicle-mounted domain controller, which can realize continuous testing and improve test efficiency.

[0029] (2) The present invention integrates a scene injection machine including an interactive scene generation module into the real vehicle and adopts a vehicle-level alignment solution, which can significantly reduce the impact of untimely interaction between the main vehicle and the background vehicle caused by the 5G communication delay between the real vehicle and the cloud control platform.

[0030] (3) The present invention embeds the scene injection machine into the real vehicle, and uses Ethernet communication with the domain controller to generate background traffic in real time and inject it into the real vehicle. It runs frame by frame until the specified task path is completed and then stored. Finally, it can be reproduced offline on the cloud platform or an evaluation report can be generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a schematic diagram of the framework of the autonomous driving virtual-reality fusion test platform system of the present invention;

[0032] Figure 2 It is a detailed framework diagram of the autonomous driving virtual-reality fusion test system of the present invention;

[0033] Figure 3 This is a schematic diagram of the module division of the autonomous driving virtual-reality fusion test system in the embodiment;

[0034] Figure 4 It is a data interaction flow diagram. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0036] Example

[0037] like Figure 1 and Figure 2 As shown, this embodiment provides an automatic driving virtual-real fusion test system based on simulation scene real vehicle injection, including a cloud control platform end and a real vehicle end; the cloud control platform end includes a map module and a first interactive scene generation module, and the real vehicle end includes a scene injection machine, a domain controller and a real vehicle, and the scene injection machine is embedded with a second interactive scene generation module;

[0038] The autonomous driving virtual-reality fusion testing process includes:

[0039] The cloud control platform packages and sends the lane-level task path generated by the map module and the interactive scene containing the background traffic core data generated by the first interactive scene generation module to the scene injection machine on the real vehicle side;

[0040] The scene injector directly transfers the lane-level task path and interactive scene to the domain controller;

[0041] The domain controller calculates the motion instructions based on the data transmitted by the scene injection machine and sends them to the real vehicle for tracking. After the real vehicle tracks, the core data of the vehicle sensed in real time is returned and transmitted back to the cloud control platform through the domain controller.

[0042] In this embodiment, the scene injection machine generates background traffic core data based on the real vehicle data sent back by the domain controller, and the domain controller performs algorithm planning, tracking updates, and sends back the real vehicle data to the scene injection machine based on the background traffic core data; the scene injection machine and the domain controller send and receive data at the same time.

[0043] Next, the system of this embodiment is introduced in detail.

[0044] 1. Cloud control platform, including map module and the first interactive scene generation module

[0045] In this embodiment, the built-in map module is a high-precision map module that can generate lane-level task paths.

[0046] The embedded first interactive scene generation module can generate typical or customized interactive scenes as background traffic, or generate the core parameters of interactive scenes. The lane-level task path and background traffic data can be packaged and sent to the domain controller of the real vehicle, or the core parameters can be packaged and sent to the scene injection machine of the real vehicle, which generates the interactive scene.

[0047] As another preferred embodiment, a visualization-digital twin module is also provided in this embodiment, which can realize digital twin by visually reproducing the perception data sent back from the real vehicle to the cloud control platform.

[0048] As another preferred embodiment, a test evaluation report generation module is also provided in this embodiment to generate a test evaluation report based on the above data.

[0049] On the cloud control platform, based on the high-precision maps and static scenes required by the preset simulation scenarios and the status of the virtual main vehicle, the road data and static scenes of the virtual main vehicle during operation are simulated to obtain the real-time road and main vehicle core data of the virtual main vehicle during operation, which are then sent to the real vehicle through 5G communication.

[0050] Collect traffic condition data for the entire road network, including: (a) static data, including but not limited to: road grade, road attributes, road width, road speed limit, number of road lanes, whether the road is closed, etc.; (b) dynamic data, including but not limited to: road speed, event information, traffic flow, weather, visibility, pedestrian conditions, non-motor vehicle conditions, traffic lights, etc.

[0051] In addition, the cloud control platform has configured the virtual main vehicle and corresponding sensors according to the preset dynamic parameters. Based on the state of the virtual main vehicle, the operation of the virtual main vehicle can be simulated to obtain real-time sensor data and form a simulation closed loop.

[0052] 2. The real vehicle side mainly includes the scene injection machine, domain controller and real vehicle

[0053] The scene injection machine is used to receive the lane-level task path issued by the cloud control platform and the background vehicle information in the typical interactive traffic scene. It should be noted that in order to avoid 5G communication delays, the scene injection machine located on the real vehicle side also has a built-in second interactive scene generation module. Therefore, in addition to receiving the background vehicle information from the cloud control platform, the scene injection machine can also generate background vehicle information in real time based on the returned main vehicle data.

[0054] Specifically, the 5G communication delay in this embodiment includes the following two scenarios:

[0055] (1) Some sections of roads covered by the 5G public network have no signal;

[0056] (2) When the 5G public network communication connection is successful, there is a 70ms delay under normal conditions. If the signal condition is poor, there may even be a 800ms delay, or even a 1-3s connection interruption.

[0057] If Ethernet communication is used for the scene injection machine, the delay problem can be basically overcome.

[0058] The second interactive scene generation module in the scene injection machine can be embedded in the cloud control platform or in the scene injection machine on the real vehicle side. If embedded in the cloud control platform, the real-time interactive simulation between the virtual main vehicle and the background vehicle will not be affected by communication delays. For the automatic driving test platform for simulation test and real vehicle injection, since the real vehicle and the cloud control platform use 5G communication, the interactive scene generation module of the cloud control platform generates interactive background vehicle data in real time based on the real vehicle data sent back at time t and sends it to the real vehicle. However, due to communication delays, the real vehicle has reached time t+n, resulting in invalid interactive tests. Therefore, in order to avoid invalid interactive simulations caused by communication delays, the interactive scene generation module is embedded in the scene injection machine on the real vehicle side. The scene injection machine on the real vehicle side is connected to the domain controller using Ethernet, which can effectively avoid communication delays.

[0059] The domain controller can receive the real-time background vehicle information sent by the scene injection machine, and send motion instructions to the real vehicle for tracking. The core vehicle data (including speed and position, etc.) measured by the on-board perception system is transmitted back to the domain controller.

[0060] During the specific implementation, the actual vehicle also includes an on-board unit for collecting the autonomous driving vehicle's real-time perception information, real-time decision-making information, and real-time driving parameters of the vehicle. The main sensors are lidar, millimeter-wave radar, and cameras.

[0061] Before conducting the autonomous driving virtual-reality fusion test, the following steps are included: 1) First, check whether the device is online; (2) Second, check whether the device is ready; (3) The platform sends a start command and gives the starting position of the scene to the scene injector, and the scene injector feeds back a task start information to the platform; (4) The platform sends the interaction rules to the scene injector, and then the scene injector and domain control start the test. The specific interaction process is as follows: Figure 4 shown.

[0062] This embodiment provides a method for testing the virtual-reality fusion of autonomous driving based on real vehicle injection into a simulation scene, specifically including:

[0063] Pre-setting: Set the reference path and reference speed of the master vehicle, the reference path and reference speed of the slave vehicle (background vehicle) on the cloud control platform and send this data packet to the scene injection machine on the real vehicle side.

[0064] The test begins: The scene injection machine receives the main vehicle reference path and speed data sent by the cloud control platform and transmits it to the domain controller. The autonomous driving algorithm located in the domain controller will plan the actual vehicle path and speed based on the main vehicle reference path and speed. The actual vehicle will track and return the main vehicle core data (including position and speed) measured by the on-board unit to the scene injection machine in real time.

[0065] The scene injection machine generates the core data of the background traffic flow based on the real vehicle data sent back by the domain controller. The domain controller performs algorithm planning, tracking updates, and sends the real vehicle data back to the scene injection machine based on the core data of the background traffic flow. The scene injection machine and the domain controller send and receive data at the same time until the preset path is traced. Embedding the scene injection machine containing the interactive scene generation module in the real vehicle can significantly reduce the delay in the interaction between the main vehicle and the background vehicle caused by the 5G communication delay between the real vehicle and the cloud control platform.

[0066] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An autonomous driving virtual-real fusion test system based on real vehicle injection of simulation scenes, characterized in that: It includes a cloud control platform end and a real vehicle end; the cloud control platform end includes a map module and a first interactive scene generation module, the real vehicle end includes a scene injection machine, a domain controller and a real vehicle, and the scene injection machine is embedded with a second interactive scene generation module; The system's autonomous driving virtual-reality fusion test process includes: The cloud control platform packages the lane-level task path generated by the map module and the interactive scene containing the background traffic core data generated by the first interactive scene generation module and sends them to the scene injection machine of the real vehicle. The scene injector directly transfers the lane-level task path and interactive scene to the domain controller; The domain controller calculates the motion instructions based on the background traffic core data transmitted by the scene injection machine and sends them to the real vehicle for tracking. After the real vehicle tracks, the core data of the vehicle sensed in real time is returned through the domain controller and transmitted back to the scene injection machine.

2. According to claim 1, the automatic driving virtual-real fusion test system based on the injection of simulation scene into real vehicle is characterized in that: The scene injection machine generates background traffic core data based on the real vehicle data sent back by the domain controller, and the domain controller plans, tracks and updates based on the background traffic core data, and sends back the real vehicle data to the scene injection machine; the scene injection machine and the domain controller send and receive data at the same time.

3. The automatic driving virtual-real fusion test system based on the injection of simulation scene into real vehicle according to claim 1 is characterized in that: 5G communication is adopted between the cloud control platform and the real vehicle.

4. The automatic driving virtual-real fusion test system based on the injection of simulation scene into real vehicle according to claim 1 is characterized in that: The scene injection machine on the real vehicle side communicates with the domain controller via Ethernet.

5. The automatic driving virtual-real fusion test system based on the injection of simulation scene into real vehicle according to claim 1 is characterized in that: The vehicle core data includes vehicle position and vehicle speed.

6. The automatic driving virtual-real fusion test system based on the injection of simulation scene into real vehicle according to claim 1 is characterized in that: The real vehicle end also includes a vehicle-mounted unit for collecting the autonomous driving vehicle's real-time perception information, real-time decision information, and real-time driving parameters of the vehicle.

7. The automatic driving virtual-real fusion test system based on the injection of simulation scene into real vehicle according to claim 1 is characterized in that: The autonomous driving path planning model carried in the domain controller plans the actual vehicle path and speed according to the reference path and speed of the main vehicle and generates motion instructions.

8. The automatic driving virtual-real fusion test system based on the injection of simulation scene into real vehicle according to claim 1 is characterized in that: The cloud control platform is also provided with a visualized digital twin module, which is used to reproduce the visualized digital twin based on the sensor data transmitted back from the actual vehicle.

9. The automatic driving virtual-real fusion test system based on the injection of simulation scene into real vehicle according to claim 1 is characterized in that: The cloud control platform is also provided with a test report generation module for generating a test report based on the vehicle core data transmitted back by the real vehicle.

10. A virtual-reality fusion test method for autonomous driving based on real vehicle injection of simulation scenes, characterized in that: Using the system according to any one of claims 1 to 9, the method comprises: Initialize the system, set the reference path and reference speed of the main vehicle, as well as the reference path and reference speed data of the background vehicle on the cloud control platform, and package the data and send it to the scene injection machine on the real vehicle side; The scene injection machine on the real vehicle receives the reference path and reference speed of the main vehicle issued by the cloud control platform and forwards it to the domain controller; After obtaining the reference path and reference speed of the main vehicle, the domain controller calculates the motion command and sends it to the real vehicle; The real vehicle tracks and returns the perceived core vehicle data in real time, which is transmitted back to the scene injection machine through the domain controller.

Images