Simulation test method, device and system

Abstract

The embodiments of the present application provide a simulation test method, device and system, which relate to the field of simulation test, and the method includes: obtaining the processing delay of each virtual sensor; judging whether each processing delay meets the preset condition; if any processing delay meets the preset condition, predicting the first driving state based on the processing delay to obtain the second driving state; based on each second driving state, using the virtual sensor corresponding to the processing delay to simulate and obtain one or more first input signals, wherein each first input signal corresponds one-to-one to each virtual sensor; sending the one or more first input signals to the sensor simulator. The method provided by the embodiments of the present application can accurately simulate the behavior and performance of the sensor, improve the accuracy of the sensor simulation, and improve the efficiency of the simulation test.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of simulation testing, and in particular to a simulation testing method, device and system. Background Art

[0002] Assisted driving and autonomous driving technologies are developing and commercializing rapidly. The use of assisted driving and autonomous driving technologies will greatly change people's transportation methods and have a huge impact on people's work and lifestyle. With assisted driving and autonomous driving, the perception capabilities of the sensors on the car and the ability of the car to drive autonomously are crucial to safe driving in various scenarios. The relevant capabilities need to be tested in different scenarios to ensure that these capabilities are reliable. The current testing methods usually include open road driving tests, closed field driving tests, and simulation tests. It is difficult to traverse various test scenarios based on road driving tests. Some analysis shows that road tests of more than 100 million kilometers are required to ensure coverage of various scenarios, which is relatively inefficient. At the same time, for extreme dangerous scene tests, it is difficult to reproduce them in field tests due to safety considerations. Pure simulation tests, such as Waymo's Carcraft, cannot test the perception capabilities of sensors, the capabilities of the frame and power system, and their coordination with the autonomous driving algorithm, and therefore cannot effectively ensure the consistency between simulation tests and road tests. Summary of the invention

[0003] The embodiments of the present application provide a simulation test method, device and system to provide a way to perform simulation testing in a virtual scene.

[0004] In a first aspect, an embodiment of the present application provides a simulation test method, which is applied to an input signal simulator, wherein the input signal simulator is located in an autonomous driving test architecture, wherein the autonomous driving test architecture further includes a virtual scene simulator and a sensor simulator, wherein the virtual scene simulator is used to simulate a virtual scene, wherein the virtual scene includes a virtual object to be tested, wherein the virtual object to be tested includes a first driving state and a plurality of virtual sensors, including:

[0005] The processing delay of each virtual sensor is obtained; specifically, the processing delay may be the difference between the processing time of the virtual sensor in the simulator and the processing time of the real sensor in the real environment.

[0006] Determine whether each processing delay meets the preset conditions; specifically, the processing delay can be a positive value or a negative value. For example, if the processing time of the virtual sensor in the simulator is greater than the processing time of the real sensor in the real environment, the processing delay is a positive value; if the processing time of the virtual sensor in the simulator is less than the processing time of the real sensor in the real environment, the processing delay is a negative positive value. Therefore, it is possible to determine whether the processing delay meets the preset conditions by determining whether the processing delay is a positive value or a negative value. Among them, each virtual sensor corresponds to its own preset front-end model and preset algorithm, so the virtual sensors all contain their own processing delays, that is, it is necessary to determine whether the processing delay of each virtual sensor meets the preset conditions.

[0007] If any processing delay meets the preset conditions, the first driving state is predicted based on the processing delay to obtain the second driving state; specifically, the first driving state may include the position, speed and acceleration of the virtual object to be measured. Therefore, when the processing delay of any virtual sensor meets the preset conditions, for example, the processing delay of the virtual sensor is a positive value, a prediction can be made based on the processing delay of the virtual sensor to obtain the second driving state, wherein the second driving state is the predicted state of the virtual sensor, and the second driving state is the position, speed and acceleration at a future moment.

[0008] Based on each second driving state, a virtual sensor corresponding to the processing delay is used for simulation to obtain one or more first input signals, wherein each first input signal corresponds one-to-one to each virtual sensor; specifically, the input signal simulator can simulate the second driving state based on each virtual sensor to obtain the first input signal.

[0009] One or more first input signals are sent to a sensor simulator.

[0010] In this embodiment, by determining the processing delay of the virtual sensor and performing delay compensation on the processing delay, the behavior and performance of the sensor can be accurately simulated, the accuracy of the sensor simulation can be improved, and the efficiency of the simulation test can be improved.

[0011] In one possible implementation, using a virtual sensor corresponding to the processing delay to perform simulation to obtain one or more first input signals includes:

[0012] Multiple virtual sensors corresponding to each processing delay are used to perform synchronous simulation to obtain multiple first input signals. Specifically, when multiple virtual sensors are simulated simultaneously to obtain the first input signal, the signal obtained by each virtual sensor simulation can be synchronized to obtain multiple synchronized first input signals, thereby synchronizing the signals and improving the accuracy of the simulation test.

[0013] In this embodiment, the accuracy of the simulation test can be improved by synchronizing the first input signal.

[0014] In one possible implementation, the processing delay is determined by the difference between a first processing time and a second processing time, wherein the first processing time is the processing time of the virtual sensor in the sensor simulator, and the second processing time is the preset real processing time of the real sensor corresponding to the virtual sensor.

[0015] In this embodiment, the performance of the real sensor can be accurately simulated by taking the difference between the processing time of the virtual sensor in the sensor simulator and the preset real processing time of the real sensor as the processing delay.

[0016] One possible implementation includes:

[0017] If any processing delay does not meet the preset condition, then based on the first driving state, a virtual sensor corresponding to the processing delay is used for simulation to obtain a second input signal; and one or more second input signals are sent to the sensor simulator with a delay based on the processing delay. Specifically, if the processing delay of the virtual sensor does not meet the preset condition, exemplarily, if the processing delay is a negative value, that is, the virtual sensor does not obtain the second predicted state, the input signal simulator can simulate according to the first driving state to obtain a second input signal, and send the second input signal with a delay based on the processing delay to compensate for the processing delay.

[0018] In this embodiment, the processing delay can be compensated by delaying the sending of the second input signal, thereby accurately simulating the performance of a real sensor.

[0019] In one possible implementation, the sensor simulator is used to receive a first input signal or a second input signal, and perform calculations based on a preset front-end model and a preset algorithm of the virtual sensor to obtain an output signal. The preset front-end model of the virtual sensor is Y=G*X+N+I, wherein Y is the output signal of the front-end model, X is the first input signal or the second input signal, G is the gain of the front end of the virtual sensor, N is the noise of the front end of the virtual sensor, and I is the interference introduced by the front end of the virtual sensor.

[0020] In this embodiment, by setting a preset front-end model and a preset algorithm in the virtual sensor, the real sensor is simulated, so that the performance of the real sensor can be simulated more accurately.

[0021] In one possible implementation, the virtual scene is obtained by simulating a virtual scene simulator using at least one CPU and / or at least one GPU, and the first input signal or the second input signal is obtained by simulating an input signal simulator using a ray tracing algorithm through at least one GPU.

[0022] In this embodiment, by simulating the signal through at least one hardware unit such as a CPU and / or a GPU, the speed of the signal simulation can be increased, thereby improving the efficiency of the simulation test.

[0023] In one possible implementation, the first driving state includes a first position, a first speed, and a first acceleration of the virtual object to be measured at time t, and the first driving state is predicted based on the processing delay to obtain the second driving state including:

[0024] Based on the processing delay, the Kalman filtering method is used to predict the first driving state of the virtual object to be measured to obtain the second driving state, wherein the second driving state includes the second position, second speed and second acceleration of the virtual object to be measured at t+T, and T is the processing delay.

[0025] In this embodiment, by predicting the position, velocity and acceleration at a future moment, the processing delay can be effectively compensated, thereby improving the accuracy of the simulation test.

[0026] In one possible implementation, the autonomous driving test architecture also includes a digital simulator, a driving system and a power system simulator. The digital simulator is used to receive the output signal sent by the sensor simulator and send the output signal to the driving system. The driving system is used to determine the driving decision based on the output signal. The power system simulator is used to simulate the driving decision to obtain a third driving state, and the third driving state is fed back to the virtual scene simulator, so that the virtual object to be tested updates the first driving state based on the third driving state.

[0027] In this embodiment, by introducing a digital simulator, a driving system and a power system simulator, a driving decision can be obtained based on the output signal, and the driving state of the virtual object to be tested can be updated based on the driving decision, thereby forming a closed loop of the simulation test, and the efficiency of the simulation test can be improved.

[0028] In one possible implementation, the virtual sensor includes at least one of a millimeter wave radar virtual sensor, a lidar virtual sensor, an infrared virtual sensor, and a camera virtual sensor.

[0029] In this embodiment, by introducing a variety of virtual sensors, simulation tests can be performed on different sensors, which can improve the flexibility of the test and further improve the efficiency of the simulation test.

[0030] In a second aspect, an embodiment of the present application provides a simulation test device, which is applied to an input signal simulator, wherein the input signal simulator is located in an autonomous driving test architecture, wherein the autonomous driving test architecture further includes a virtual scene simulator and a sensor simulator, wherein the virtual scene simulator is used to simulate a virtual scene, wherein the virtual scene includes a virtual object to be tested, wherein the virtual object to be tested includes a first driving state and a plurality of virtual sensors, including:

[0031] A receiving circuit, used to obtain a processing delay of each virtual sensor;

[0032] A prediction circuit, used to determine whether each processing delay satisfies a preset condition; if any processing delay satisfies the preset condition, the first driving state is predicted based on the processing delay to obtain a second driving state;

[0033] A first simulation circuit is used to simulate based on each second driving state using a virtual sensor corresponding to the processing delay to obtain one or more first input signals, wherein each first input signal corresponds to each virtual sensor one by one;

[0034] The first sending circuit is used to send one or more first input signals to the sensor simulator.

[0035] In one possible implementation, the first analog circuit is further configured to perform synchronous simulation using a plurality of virtual sensors corresponding to each processing delay, to obtain a plurality of first input signals.

[0036] In one possible implementation, the processing delay is determined by the difference between a first processing time and a second processing time, wherein the first processing time is the processing time of the virtual sensor in the sensor simulator, and the second processing time is the preset real processing time of the real sensor corresponding to the virtual sensor.

[0037] In one possible implementation, the device further includes:

[0038] The second analog circuit is used for, if any processing delay does not meet the preset condition, based on the first driving state,

[0039] Perform simulation using a virtual sensor corresponding to the processing delay to obtain a second input signal;

[0040] The second sending circuit is used to send one or more second input signals to the sensor simulator with a delay based on the processing delay.

[0041] In one possible implementation manner, the first input signal or the second input signal is obtained by an input signal simulator using a ray tracing algorithm through at least one GPU simulation.

[0042] In one possible implementation, the first driving state includes a first position, a first speed and a first acceleration of the virtual object to be tested at time t, and the prediction circuit is further used to predict the first driving state of the virtual object to be tested based on a processing delay and using a Kalman filtering method to obtain a second driving state, wherein the second driving state includes a second position, a second speed and a second acceleration of the virtual object to be tested at t+T, and T is the processing delay.

[0043] In a third aspect, an embodiment of the present application provides a simulation test system, including: a virtual scene simulator, an input signal simulator, a sensor simulator, a digital simulator and a system synchronization module; wherein,

[0044] The virtual scene simulator is used for a virtual scene, the virtual scene includes a virtual object to be tested, and the virtual object to be tested includes a first driving state and a plurality of virtual sensors;

[0045] The input signal simulator is used to obtain the processing delay of each virtual sensor; determine whether each processing delay meets the preset condition; if any processing delay meets the preset condition, predict the first driving state based on the processing delay to obtain the second driving state; based on each second driving state, use the virtual sensor corresponding to the processing delay to simulate to obtain one or more first input signals, wherein each first input signal corresponds to each virtual sensor one by one; send the one or more first input signals to the sensor simulator;

[0046] The sensor simulator is used to receive a first input signal and perform calculations based on a preset front-end model and a preset algorithm of the virtual sensor to obtain an output signal;

[0047] The digital simulator is used to receive the output signal sent by the sensor simulator;

[0048] The system synchronization module is used to provide a synchronous clock to the virtual scene simulator, input signal simulator, sensor simulator and digital simulator.

[0049] In one possible implementation, the input signal simulator is further configured to use a plurality of virtual sensors corresponding to each processing delay to perform synchronous simulation to obtain a plurality of first input signals.

[0050] In one possible implementation, the processing delay is determined by the difference between a first processing time and a second processing time, wherein the first processing time is the processing time of the virtual sensor in the sensor simulator, and the second processing time is the preset real processing time of the real sensor corresponding to the virtual sensor.

[0051] In one possible implementation, the input signal simulator is also used to simulate, based on the first driving state, using a virtual sensor corresponding to the processing delay to obtain a second input signal if any processing delay does not meet a preset condition; and send one or more second input signal delays to the sensor simulator based on the processing delay.

[0052] In one possible implementation, the sensor simulator is further used to receive a second input signal, and the preset front-end model of the virtual sensor is Y=G*X+N+I, wherein Y is the output signal of the front-end model, X is the first input signal or the second input signal, G is the gain of the front end of the virtual sensor, N is the noise of the front end of the virtual sensor, and I is the interference introduced by the front end of the virtual sensor.

[0053] In one possible implementation, the virtual scene is obtained by simulating a virtual scene simulator using at least one CPU and / or at least one GPU, and the first input signal or the second input signal is obtained by simulating an input signal simulator using a ray tracing algorithm through at least one GPU.

[0054] In one possible implementation, the first driving state includes a first position, a first speed and a first acceleration of the virtual object to be tested at time t, and the input signal simulator is further used to predict the first driving state of the virtual object to be tested based on a processing delay and a Kalman filtering method to obtain a second driving state, wherein the second driving state includes a second position, a second speed and a second acceleration of the virtual object to be tested at t+T, where T is the processing delay.

[0055] One possible implementation method also includes a driving system and a power system simulator, wherein:

[0056] The digital simulator is also used to send output signals to the driving system;

[0057] The driving system is used to determine a driving decision based on the output signal;

[0058] The power system simulator is used to simulate the driving decision, obtain the third driving state, and feed the third driving state back to the virtual scene simulator, so that the virtual object to be tested updates the first driving state based on the third driving state.

[0059] In one possible implementation, the virtual sensor includes at least one of a millimeter wave radar virtual sensor, a lidar virtual sensor, an infrared virtual sensor, and a camera virtual sensor.

[0060] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the method described in the first aspect.

[0061] In a fifth aspect, an embodiment of the present application provides a computer program, which, when executed by a computer, is used to execute the method described in the first aspect.

[0062] In one possible design, the program in the fifth aspect may be stored in whole or in part on a storage medium packaged together with the processor, or may be stored in whole or in part on a memory not packaged together with the processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A schematic diagram of the system architecture provided for an embodiment of the present application;

[0064] Figure 2 A flowchart of a simulation test method provided in an embodiment of the present application;

[0065] Figure 3 A schematic diagram of state prediction provided in an embodiment of the present application;

[0066] Figure 4 A schematic diagram of the structure of a simulation test device provided in an embodiment of the present application;

[0067] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The technical solution in the embodiment of the present application will be described below in conjunction with the drawings in the embodiment of the present application. In the description of the embodiment of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0069] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0070] At present, in order to improve the efficiency of assisted driving and autonomous driving tests, and to improve the consistency between simulation tests and road tests, a hardware closed-loop solution is needed that can simulate various scenarios in the laboratory and conduct joint tests on sensor capabilities, the frame of the test vehicle, and the power system capabilities through autonomous driving software and algorithms. There are two ways to simulate scenarios in the laboratory based on analog interfaces and digital interfaces. The analog interface-based approach is to generate analog signals received by each sensor through scenario simulation, and send the signals to each sensor through an analog interface. The analog interface approach currently faces the problems of complex systems and immature solutions. For example, the radar echoes of multiple targets in the simulated scenario can only simulate a few targets, which cannot meet the requirements. The system that uses the probe wall solution to simulate the echoes of dozens of targets is very expensive and complex, and there is no mature solution. There is currently no solution for the LiDAR echo signals of multiple targets in the simulated scenario. Therefore, the digital interface scenario simulation method is generally used. In this simulation method, the generated scenario simulation signal is directly sent to the assisted driving and autonomous driving processing system through a digital interface without passing through the actual sensor. The usual solution is to establish a behavioral model or statistical model for each sensor to simulate the performance impact of the sensor. This method cannot achieve synchronous simulation of multi-sensor behaviors in the same scene. At the same time, behavioral or statistical models cannot accurately characterize the performance of sensors in specific scenarios. There are also proposals to use physical models combined with sensor algorithms to simulate the performance of sensors in specific scenarios. However, this solution does not consider the performance impact of the sensor simulation front end. At the same time, it does not consider the synchronization and latency issues between multiple sensor simulation signals. The fusion of multiple sensors in autonomous driving requires that the perception of the same scene by different sensors can be synchronously simulated during scene simulation. At the same time, in order to ensure that the hardware closed-loop system can be consistent with the behavior of real road tests, the scene simulation system must be able to compensate for the additional delay generated in the scene simulation.

[0071] Based on the above problems, an embodiment of the present application proposes a simulation testing method, which can effectively solve the delay problem caused by the simulation sensors in the digital scene simulation test, thereby accurately and synchronously simulating the behavior and performance of multiple sensors in the digital scene simulator.

[0072] Now combined Figure 1-Figure 3 The simulation test method provided in the embodiment of the present application is described as follows: Figure 1 The system architecture provided by the embodiment of the present application is shown in FIG. Figure 1The above system architecture includes a virtual scene simulator 100, an input signal simulator 200, a sensor simulator 300, a digital simulator 400, an assisted driving and automatic driving system 500 (for the convenience of explanation, the "assisted driving and automatic driving system" is referred to as the "driving system" below), a power system simulator 600 and a system synchronization module 700. Among them,

[0073] The virtual scene simulator 100 is used to construct a virtual scene and send the virtual scene information to the input signal simulator 200. In the specific implementation, the virtual scene simulator 100 can be a computer or other types of computing devices, which is not specifically limited in this application. In the virtual scene simulator 100, a high-speed Ethernet network card supporting the IEEE1588 protocol, one or more central processing units (CPU) or graphics processing units (GPU) can be included to ensure the real-time performance of the simulation. Among them, the construction of the virtual scene can be realized by the scene simulation software installed in the virtual scene simulator 100, which can be a commercial 3D simulation software or an open source 3D simulation software, which is not specifically limited in this embodiment of the present application.

[0074] The virtual scene may include a virtual scene built using a virtual three-dimensional model, and the virtual three-dimensional model simulates real objects in the real world. For example, real objects may include cars, people, animals, trees, roads, and buildings, etc., and may also include other real objects. The embodiments of the present application are not particularly limited to this. It can be understood that the three-dimensional model can be regarded as a virtual object. In the specific implementation, the user usually creates a virtual object in the virtual scene and tests the virtual object as a test object. Therefore, the test object can be a first virtual object, and virtual objects other than the first virtual object in the virtual scene can be regarded as second virtual objects. Exemplarily, the first virtual object can be a car, thereby testing the automatic driving and assisted driving performance of the car. For example, while the car is driving, the second virtual object around the car will cause obstacles to the car, thereby affecting the driving of the car, and then affecting the driving decision of the car, such as acceleration, deceleration, turning and parking. Through the above-mentioned scene simulation software, the virtual scene simulator 100 can send the information of the virtual object in the virtual scene to the input signal simulator 200, and the information of the virtual object may include the coordinates, materials, lighting and other information of the virtual object. The transmission of information between the virtual scene simulator 100 and the input signal simulator 200 adopts a standardized interface to ensure that the above system architecture does not depend on specific scene simulation software.

[0075] It is understandable that the above-mentioned first virtual object may also be a drone or other types of autonomous driving equipment, and the embodiments of the present application do not specifically limit this.

[0076] In addition, for each real object, when signals such as visible light, millimeter wave radar, lidar and infrared radar are irradiated on the surface of the real object, physical phenomena such as reflection, scattering and diffraction will occur. Therefore, the virtual scene simulator 100 can establish a corresponding physical model for the virtual object based on the above physical phenomena. At the same time, the virtual scene simulator 100 can also establish model parameters corresponding to the material of the virtual object, wherein the material of the virtual object is consistent with the material used for the real object.

[0077] In order to test the first virtual object, the virtual scene simulator 100 can also arrange a virtual sensor on the first virtual object, and the virtual sensor is used to simulate the input signal obtained in the virtual environment. The above physical model and model parameters can simulate the input signals obtained by various virtual sensors in the virtual environment.

[0078] The input signal simulator 200 is used to simulate the input signals obtained by various virtual sensors in the virtual environment, and send the input signals to the sensor simulator 300 for processing. Among them, the input signal can be simulated based on the driving state and virtual scene information of the first virtual object. The above-mentioned virtual sensors may include millimeter wave radar virtual sensors, laser radar virtual sensors, infrared virtual sensors and camera virtual sensors, and may also include other types of virtual sensors, which are not specifically limited in the embodiments of the present application. In the specific implementation, the input signal simulator 200 can be a computer or other types of computing devices, which are not specifically limited in the present application. In the input signal simulator 200, a high-speed Ethernet network card supporting the IEEE1588 protocol and one or more GPUs may be included. In addition, the simulation process of acquiring the input signal can be implemented by input signal simulation software. It is understandable that in the input signal simulation software, the format, type, and related parameters of the virtual sensor can be configured, the information of the virtual object sent from the virtual scene simulator 100, and the material parameters of the related virtual object are received, and the ray tracing algorithm is used to simulate different types of virtual sensors based on the driving state of the first virtual object (for example, the position, speed and acceleration of the first virtual object), the above-mentioned physical model and model parameters. For example, the physical effect of each real sensor on the surface of the virtual object is simulated and calculated (for example, including but not limited to reflection, scattering, diffraction and other effects), thereby simulating the acquisition of the input signal in the virtual environment, for example, the transmitter of the virtual sensor transmits a signal, and the transmitted signal is returned to the receiver of the virtual sensor after the physical effect occurs on the surface of the virtual object, thereby obtaining the input signal of the virtual sensor. And the simulation of the above-mentioned input signal can be processed by the above-mentioned GPU to ensure the real-time performance of the simulation. Exemplarily, the input signal simulator 200 can simulate the shooting of images in the virtual environment by a virtual camera, or simulate the detection of virtual objects in the virtual environment by a millimeter wave radar.

[0079] The sensor simulator 300 is used to simulate the behavior and performance of each virtual sensor. For example, the sensor simulator 300 can receive the input signal obtained by the input signal simulator 200, and process the input signal by simulating the processing method of the real sensor, thereby obtaining the output signal, and the output signal can be sent to the digital simulator 400 for processing. In the specific implementation, the sensor simulator 300 can be a computer or other types of computing devices, which is not specifically limited in this application. In the sensor simulator 300, a high-speed Ethernet network card supporting the IEEE1588 protocol, one or more field programmable gate array (FPGA) acceleration cards or GPUs can be included. Exemplarily, the sensor simulator 300 can simulate the performance of the front end of each virtual sensor and the algorithm of each virtual sensor. Among them, the simulation of the performance of the front end of each virtual sensor can be carried out by modeling. Exemplarily, the model of each virtual sensor front end can be pre-constructed as: Y=G*X+N+I, where X is the input signal of the sensor, Y is the output signal of the sensor front end, G is the gain of the sensor front end, N is the noise of the sensor front end, and I is the interference introduced by the sensor front end. The output signal Y can be obtained by processing the preset algorithm of the virtual sensor.

[0080] It can be understood that the model of the virtual sensor front end is built based on the real sensor. Therefore, by using the virtual sensor to simulate the front end of the real sensor through modeling, it can be ensured that the performance of the virtual sensor front end is consistent with that of the real sensor front end in terms of statistical characteristics.

[0081] In addition, for the algorithm of the virtual sensor, the algorithm of the real sensor can be directly used, which will not be repeated here. For the algorithms of different real sensors, the sensor simulator 300 can use the corresponding FPGA acceleration card or GPU card for simulation to ensure the real-time requirements and computing power requirements of the simulation.

[0082] The digital simulator 400 is used to receive the output signal sent by the sensor simulator 300, and can send the output signal to the driving system 500. In a specific implementation, the digital simulator 400 can be a computer or other types of computing devices, which is not specifically limited in this application. The digital simulator 400 can include a high-speed Ethernet network card supporting the IEEE1588 protocol, one or more FPGA acceleration cards, and multiple interfaces.

[0083] The interface includes a physical interface and a digital interface between the digital simulator 400 and the real sensor. The digital simulator 400 can use the physical interface and the digital interface to receive real data collected by different real sensors, and perform testing by replaying these real collected data. The physical interface may include but is not limited to a controller area network (CAN), a mobile industry processor interface (MIPI), an Ethernet, a Gigabit Multimedia Serial Link (GSML), a flat panel display link (FPLINK), a local interconnect network (LIN), and a 100M-T1 interface. The digital interface may include but is not limited to a high-speed peripheral component interconnect Express (PCIE) interface, a serial advanced technology attachment (SATA) interface, a high-speed Ethernet interface, and a digital optical fiber interface.

[0084] In addition, the above-mentioned interface may also include a physical interface between the digital simulator 400 and the driving system 500, and a digital interface between the digital simulator 400 and the power system simulator 600. Among them, the physical interface may include a CAN interface. For example, through the CAN interface, the digital simulator 400 can be connected to the CAN bus of the driving system 500, so that the digital simulator 400 can send an output signal to the driving system 500, and then the driving system 500 can make a driving decision based on the output signal. The digital interface may include an Ethernet interface, which is used to transmit the driving state of the object to be tested (e.g., a vehicle) output by the power system simulator 600.

[0085] The driving system 500 is used to receive the output signal sent by the digital simulator 400, and make driving decisions based on the output signal. Among them, the driving system 500 can be located in a real object to be tested. For example, the driving system 500 can be located inside a real vehicle to be tested. At this time, the vehicle serves as the object to be tested. It can be understood that the driving system 500 can also be tested separately. For example, the driving system 500 can be taken out of the real vehicle, so that the driving system 500 can be used as the object to be tested. In addition, the driving decision can include operations such as acceleration, braking, deceleration, and turning, and can also include other operations, which are not particularly limited in this embodiment. Then, the driving system 500 can send the above-mentioned driving decision to the power system simulator 600, so that the power system simulator 600 can simulate based on the driving decision to update the driving status of the real object to be tested and the virtual object.

[0086] The power system simulator 600 is used to receive the driving decision sent by the driving system 500, simulate the dynamic characteristics of the real vehicle based on the driving decision, thereby outputting the driving state corresponding to the real vehicle, and feeding back the driving state to the first virtual object in the virtual scene, so that the first virtual object can be updated based on the driving state, thereby completing the simulation test. In the specific implementation, the power system simulator 600 can be a computer or other types of computing devices, and this application does not make special restrictions on this. In the power system simulator 600, multiple interfaces can be included. The interface can include a physical interface (for example, a CAN interface) between the power system simulator 600 and the driving system 500, and a digital interface (for example, an Ethernet interface) between the power system simulator 600 and the virtual scene simulator. Among them, through the physical interface, the power system simulator 600 can receive the driving decision sent by the driving system 500; through the digital interface, the power system simulator 600 can send the real vehicle state to the first virtual object in the virtual scene.

[0087] The system synchronization module 700 is used to provide a synchronous clock for the virtual scene simulator 100, the input signal simulator 200, the sensor simulator 300 and the digital simulator 400 to ensure the clock synchronization between the virtual scene simulator 100, the input signal simulator 200, the sensor simulator 300 and the digital simulator 400. In a specific implementation, the system synchronization module 700 can be, for example, a high-speed Ethernet switch supporting the 1588 synchronization protocol, or a dedicated synchronization module, which is not particularly limited in the embodiments of the present application.

[0088] It is understandable that data exchange can be performed between the virtual scene simulator 100, the input signal simulator 200, the sensor simulator 300, and the digital simulator 400 via a high-speed Ethernet switch or other high-speed data connection devices.

[0089] like Figure 2 The figure is a flow chart of an embodiment of the simulation test method provided by the embodiment of the present application, including:

[0090] Step 101 : construct a virtual scene using the virtual scene simulator 100 .

[0091] Specifically, a virtual scene can be constructed by a virtual scene simulator 100. The virtual scene can include a scene to be tested, and the scene to be tested can include a virtual object and a material corresponding to the virtual object. Exemplarily, the virtual object can include: a car, a person, an animal, a tree, a road, a building, etc., and can also include other objects, which are not particularly limited in the embodiments of the present application. In addition, a virtual object to be tested can also be determined in the virtual scene, for example, the virtual object to be tested can be a car.

[0092] In order to simulate the test of the virtual object, multiple virtual sensors can also be configured on the virtual object. Therefore, the first parameters of the virtual sensor can be configured through the input signal simulator 200 to simulate the acquisition of the input signal by the virtual sensor. Among them, when configuring the first parameters of the multiple virtual sensors, the first parameter information of the configuration may include: the number of sensors, the type of sensors, the assembly parameters of the sensors on the actual vehicle (for example, the installation height and angle of the sensors on the actual vehicle, etc.), the physical parameters of the sensors (for example, the number, position and direction of the transmitting and receiving antennas of the millimeter wave radar, the frequency, working mode and number of lines of the laser radar, the field of view angle and focal length of the camera, etc.).

[0093] It is understandable that in the constructed virtual scene, the virtual object to be tested can be regarded as the first virtual object, and all virtual objects in the virtual scene except the first virtual object can be regarded as the second virtual object. For example, if a car in the virtual scene is regarded as the first virtual object, that is, the virtual object to be tested, other virtual objects (e.g., cars, people, animals, trees, roads, buildings, etc.) can be regarded as the second virtual objects.

[0094] It should be noted that, in addition to configuring the parameters related to the virtual scene and the virtual object, the second parameters of the virtual sensor can also be configured in the sensor simulator 300. When configuring the second parameters of the multiple virtual sensors, the configured second parameter information may include: the number of sensors, the type of sensor, the front-end parameters of the sensor (for example, the front-end gain G, the front-end noise N and the interference I introduced by the front-end), the sensor processing delay and the sensor algorithm, etc.

[0095] Step 102 : the virtual scene simulator 100 sends the virtual scene information to the input signal simulator 200 .

[0096] Specifically, the virtual scene information may include relevant information of all virtual objects in the virtual scene, wherein the relevant information may include coordinate position, material information (for example, plastic, metal, etc.), lighting conditions, etc., and the embodiments of the present application do not specifically limit this.

[0097] Step 103: the input signal simulator 200 simulates and obtains multiple input signals of the first virtual object.

[0098] Specifically, when the first virtual object moves in the above-mentioned virtual scene, multiple input signals around the first virtual object can be obtained through the above-mentioned virtual sensor simulation in the above-mentioned input signal simulator 200. Among them, the virtual sensor may include: one or more of: millimeter wave radar virtual sensor, laser radar virtual sensor, infrared virtual sensor and camera virtual sensor. The input signal may include an echo signal and / or an image. Exemplarily, the echo signal can be obtained through the millimeter wave radar virtual sensor, the laser radar virtual sensor, and the infrared virtual sensor, and the captured image can also be obtained through the camera virtual sensor. The input signal can be based on the driving state S of the first virtual object. t , virtual scene information, and is obtained by using a ray tracing algorithm to calculate. It is understandable that other algorithms may also be used to calculate and obtain the input signal, and this embodiment of the present application does not specifically limit this.

[0099]

[0100] Xi is the position of the first virtual object, Vi is the speed of the first virtual object, and Ai is the acceleration of the first virtual object. It can be understood that the state of the first virtual object may also include other variables, which are not specifically limited in the present embodiment.

[0101] It is understandable that each input signal corresponds to each virtual sensor one by one. For example, the millimeter wave radar virtual sensor may obtain an A echo input signal, the laser radar virtual sensor may obtain a B echo input signal, and the camera virtual sensor may obtain a C image input signal.

[0102] In addition, when the multiple input signals are acquired through the input signal simulator 200, the multiple input signals may be synchronized to ensure that the input signal simulator 200 can simultaneously acquire input signals acquired by multiple virtual sensors for the same scene.

[0103] Furthermore, when the sensor simulator 300 processes the above-mentioned multiple input signals, a simulated processing time Tf will be generated. When the real sensor processes the input signal, a real processing time Tz will be generated. It can be understood that since the virtual sensor simulates the real sensor and is different from the real sensor, the simulated processing time Tf generated by the above-mentioned virtual sensor will be different from the real processing time Tz generated by the real sensor. For example, the above-mentioned simulated processing time Tf may be greater than or equal to the real processing time Tz, and the above-mentioned simulated processing time Tf may also be less than or equal to the real processing time Tz. When the above-mentioned simulated processing time Tf is inconsistent with the real processing time Tz, the above-mentioned input signal can be delayed compensated in the input signal simulator 200, so that the above-mentioned simulation test can better simulate the real scene. Among them, the delay refers to the difference between the above-mentioned simulated processing time Tf and the real processing time Tz.

[0104] Therefore, the input signal simulator 200 can also obtain the simulation processing time Tf of each virtual sensor in the sensor simulator 300. Among them, the simulation processing time Tf corresponds to the virtual sensor one by one. Exemplarily, the simulation processing time Tf1 of the millimeter wave radar virtual sensor, the simulation processing time Tf2 of the laser radar virtual sensor, and the simulation processing time Tf3 of the camera virtual sensor can be obtained. Then, the real processing time Tz of the real sensor corresponding to the virtual sensor can also be obtained. Among them, the real processing time Tz corresponds to the real sensor one by one. Exemplarily, the real processing time Tz1 of the millimeter wave radar real sensor, the real processing time Tz2 of the laser radar real sensor, and the real processing time Tz3 of the camera real sensor can be obtained.

[0105] Next, the input signal simulator 200 may compare the simulated processing time Tf of each virtual sensor with the corresponding real processing time Tz.

[0106] If the simulation processing time Tf < the real processing time Tz, the first input signal can be obtained based on the current state St of the first virtual object, and the first input signal corresponding to the virtual sensor can be sent with a delay, wherein the delay = Tz-Tf. Exemplarily, assuming that the simulation processing time Tf of the laser radar virtual sensor is 5ms and the real processing time Tz of the laser radar real sensor is 10ms, the first input signal corresponding to the laser radar virtual sensor can be sent with a delay of Tz-Tf=10-5=5ms, thereby matching the simulation processing time with the processing time of the real sensor, thereby improving the accuracy of the simulation test.

[0107] If the simulation processing time Tf> the real processing time Tz, that is, the virtual sensor processing time is longer than the real sensor processing time, resulting in a delay, and therefore, the delay needs to be compensated. In a specific implementation, the driving state of the first virtual object can be predicted, so that the input signal simulator 200 can simulate based on the predicted driving state, and then obtain the second input signal. Among them, the second input signal can be a predicted input signal after the Tf-Tz time period, and the prediction method can be through the Kalman filtering method, or other prediction methods can be used, and the embodiment of the present application does not specifically limit this.

[0108] Exemplarily, assuming that the driving state of the first virtual object at time t is S t ,in,

[0109]

[0110] Xi is the position of the first virtual object, Vi is the speed of the first virtual object, and Ai is the acceleration of the first virtual object. It can be understood that the state of the first virtual object may also include other variables, which are not specifically limited in the present embodiment. Then the predicted driving state of the first virtual object at time t+T is S t+T =Fi*S t +Bi*Ui+Ni, where,

[0111]

[0112]

[0113] Ui=ΔAi,

[0114] Ni is the state prediction noise. Thus, the driving state of the first virtual object at a future time can be predicted. Then, based on the driving state of the first virtual object at a future time, the input signal simulator 300 can be used to simulate the input signal, thereby obtaining a second input signal.

[0115] For example, assume that the simulation processing time of the millimeter-wave radar virtual sensor is Tf1=10ms, that is, the millimeter-wave radar virtual sensor needs to process the input signal for 10ms, thereby obtaining the first input signal of the millimeter-wave radar; the actual processing time of the millimeter-wave radar real sensor is Tz1=5ms, that is, the millimeter-wave radar real sensor needs to process the input signal for 5ms, thereby obtaining the first input signal of the millimeter-wave radar; the simulation processing time of the lidar virtual sensor is Tf2=12ms, that is, the lidar virtual sensor needs to process the input signal for 10ms, thereby obtaining the first input signal of the lidar; the actual processing time of the lidar real sensor is Tz2=8ms, that is, the lidar real sensor needs to process the input signal for 5ms, thereby obtaining the first input signal of the lidar. Since the simulation delay of the millimeter-wave radar virtual sensor is Ty1=Tf1-Tz1=10-5=5ms, and the simulation delay of the laser radar virtual sensor is Ty2=simulation processing time Tf2-Tz2=12-8=4ms, that is, Ty1>Ty2, at this time, the driving state of the first virtual object can be predicted based on the maximum simulation delay (for example, Ty1), and based on the predicted driving state, the millimeter-wave radar virtual sensor and the laser radar virtual sensor are used for simulation respectively to obtain the millimeter-wave radar second input signal and the laser radar second input signal, thereby ensuring that all first input signals can obtain effective delay compensation.

[0116] Now let’s take the millimeter wave radar virtual sensor as an example to explain. Figure 3 is a schematic diagram for predicting the second input signal. Figure 3 As shown, the driving state of the first virtual object at time t is S1, and the millimeter-wave radar virtual sensor on the first virtual object can obtain the input signal 101 based on the driving state S1 simulation at time t. Next, the driving state of the first virtual object at time t+T1 is predicted, thereby obtaining the driving state of the first virtual object at time t+T1 as S2, wherein T1 is the delay, that is, the difference between the simulated processing time Tf1 of the millimeter-wave radar virtual sensor and the real processing time Tz1 of the millimeter-wave radar real sensor. The millimeter-wave radar virtual sensor can obtain the second input signal 102 based on the driving state S2 simulation at time t+T1, thereby completing the prediction of the second input signal 102, and thus completing the delay compensation of the input signal.

[0117] It is understandable that the LiDAR virtual sensor can also be Figure 3 The input signal of the laser radar is predicted by the method in the above method, thereby obtaining the second input signal of the laser radar, which will not be repeated here.

[0118] Optionally, after the input signal simulator 200 completes the delay compensation of the input signal, the simulation delay of each virtual sensor can also be corrected. Taking the above-mentioned millimeter-wave radar virtual sensor and laser radar virtual sensor as an example, after the delay compensation of the millimeter-wave radar virtual sensor and the laser radar virtual sensor is completed, since the delay compensation time of the laser radar virtual sensor adopts the delay compensation time of the millimeter-wave radar virtual sensor, for example, the simulation delay Ty2 of the laser radar virtual sensor is Tf2-Tz2=12-8=4ms, but the delay compensation time of the laser radar virtual sensor is 5ms, therefore, the delay compensation time of the laser radar virtual sensor does not match the simulation delay Ty2 of the laser radar virtual sensor. At this time, the input signal simulator 200 can send the second input signal delay T2 predicted by the laser radar, thereby making the cumulative time of the delay T2 and the simulation processing time correspond to the predicted second input signal, thereby simulating the performance of the real sensor. Among them, T2 is the difference between the delay compensation time and the simulation delay. Exemplarily, assuming that the delay compensation time of the laser radar virtual sensor is 5ms and the simulation delay of the laser radar virtual sensor is 4ms, then T2 = delay compensation time - simulation delay = 5-4 = 1ms. In other words, the second output signal of the laser radar is sent with a delay of 1ms. The actual processing time of the laser radar real sensor Tz2 = 8ms, and the delay compensation time of the laser radar virtual sensor is 5ms, that is, the predicted second input signal is Tz2+5=8+5=13ms later; and the simulation processing time Tf2 of the laser radar virtual sensor is 12ms, therefore, the second input signal can be sent with a delay of 13-12=1ms, thereby ensuring that the predicted second input signal matches the simulation processing time of the sensor simulator 300.

[0119] In step 104 , the input signal simulator 200 sends a plurality of input signals to the sensor simulator 300 .

[0120] Specifically, after the input signal simulator 200 acquires a plurality of input signals of the first virtual object, the plurality of input signals may be sent to the sensor simulator 300 , wherein the input signal may include the first input signal and / or the second input signal.

[0121] Step 105 : The sensor simulator 300 processes the multiple input signals and outputs multiple output signals.

[0122] Specifically, after receiving the multiple input signals sent by the input signal simulator 200, the sensor simulator 300 can process the multiple input signals to obtain multiple output signals. The processing process can be performed according to the front-end model of the virtual sensor (for example, Y=G*X+N+I) and the preset algorithm of the virtual sensor.

[0123] It should be noted that each virtual sensor can adopt a different preset front-end model, and the embodiment of the present application does not specifically limit the implementation method of the specific front-end model.

[0124] It is understandable that each input signal corresponds to each output signal one-to-one. For example, the A echo input signal obtained by the millimeter-wave radar virtual sensor can be input into the front-end model of the millimeter-wave radar virtual sensor and processed using a preset millimeter-wave radar sensor algorithm to obtain the A echo output signal; or the B echo input signal obtained by the lidar virtual sensor can be input into the front-end model of the lidar virtual sensor and processed using a preset lidar sensor algorithm to obtain the B echo output signal; or the C image input signal obtained by the camera virtual sensor can be input into the front-end model of the camera virtual sensor and processed using a preset camera sensor algorithm to obtain the C image output signal.

[0125] Step 106 , sending the multiple output signals to the digital simulator 400 .

[0126] Specifically, after the sensor simulator 300 processes a plurality of input signals to obtain a plurality of output signals, the plurality of output signals may be sent to the digital simulator 400 .

[0127] Step 107 , the digital simulator 400 receives the multiple output signals for processing and sends the multiple output signals to the driving system 500 .

[0128] Specifically, the digital simulator 400 may receive the output signal corresponding to each virtual sensor sent by the sensor simulator 300. In order to test the driving performance of the real vehicle, the digital simulator 400 may also send the above-mentioned multiple output signals to the driving system 500 of the real vehicle.

[0129] In step 108 , the driving system 500 determines a driving decision based on the plurality of output signals.

[0130] Specifically, the driving system 500 makes a driving decision after receiving the above-mentioned multiple output signals sent by the digital simulator 400. The driving decision may include operations such as acceleration, deceleration, braking, and turning, and may also include other driving decisions, which are not specifically limited in the embodiments of the present application.

[0131] Step 109 , sending the driving decision to the power system simulator 600 .

[0132] In step 110 , the power system simulator 600 simulates the driving state St′ of the actual vehicle based on the driving decision.

[0133] Specifically, the driving state St' corresponds to the driving decision. For example, if the driving decision is acceleration, the driving state of the actual vehicle simulated by the power system simulator 600 is acceleration, and if the driving decision is braking, the driving state of the actual vehicle simulated by the power system simulator 600 is parking after braking.

[0134] Step 111 : Feedback the driving state St′ to the virtual scene simulator 100 , so that the virtual scene simulator 100 updates the driving state St of the first virtual object based on St′.

[0135] Specifically, after the power system simulator 600 obtains the driving state St', the driving state St' can be fed back to the virtual scene simulator 100, so that the virtual scene simulator 100 can update the driving state St of the first virtual object based on St'. Exemplarily, taking following a vehicle as an example, it is assumed that the test vehicle (first virtual object) follows another vehicle (second virtual object) to test the automatic driving performance of the first virtual object. When the second virtual object brakes, the distance between the first virtual object and the second virtual object becomes closer and closer. By analyzing the sensor data (for example, the second output signal) obtained by the virtual sensor on the first virtual object, the driving decision can be determined (for example, the braking is determined by the driving system 500), and the braking decision can be fed back to the first virtual object, thereby completing the simulation test of the entire system.

[0136] Step 112 : The virtual scene simulator 100 updates the driving state of the first virtual object based on the driving state St′.

[0137] Through the embodiments of the present application, the driving state of the object to be tested is predicted based on the delay between the simulated sensor and the real sensor, and the delay is compensated thereby, which can effectively solve the delay problem caused by the simulated sensor in the digital scene simulation test, and further accurately and synchronously simulate the behavior and performance of multiple sensors in the digital scene simulator.

[0138] Figure 4 This is a schematic diagram of the structure of an embodiment of the simulation test device of the present application, such as Figure 4As shown, the simulation test device 40 is applied to an input signal simulator, which is located in an autonomous driving test architecture. The autonomous driving test architecture also includes a virtual scene simulator and a sensor simulator. The virtual scene simulator is used to simulate a virtual scene. The virtual scene includes a virtual object to be tested. The virtual object to be tested includes a first driving state and a plurality of virtual sensors ... simulator is used to simulate a virtual scene. The virtual scene includes a virtual object to be tested. The virtual object to be tested includes a first driving state and a plurality of virtual sensors. The virtual scene simulator is used to simulate a virtual scene. The virtual scene simulator is used to simulate a virtual scene. The virtual scene includes a virtual object to be tested. The virtual object to be tested includes a first driving state and a plurality of virtual sensors. The virtual scene simulator is used to simulate a virtual scene. The virtual scene simulator is used to simulate a virtual scene. The virtual scene simulator is used to simulate a virtual scene. The virtual scene simulator is used to simulate a virtual scene. The virtual object to be tested includes a first driving state and a plurality of virtual sensors. The virtual scene simulator is used to simulate a virtual scene. The virtual scene simulator is used to simulate a virtual scene. The virtual scene simulator is used to simulate a virtual scene. The virtual object to be tested includes a first driving state and a

[0139] A receiving circuit 41, used to obtain a processing delay of each virtual sensor;

[0140] A prediction circuit 42, used to determine whether each processing delay satisfies a preset condition; if any processing delay satisfies the preset condition, the first driving state is predicted based on the processing delay to obtain a second driving state;

[0141] A first simulation circuit 43 is used to simulate based on each second driving state using a virtual sensor corresponding to the processing delay to obtain one or more first input signals, wherein each first input signal corresponds to each virtual sensor one by one;

[0142] The first sending circuit 44 is used to send one or more first input signals to the sensor simulator.

[0143] In one possible implementation, the first simulation circuit 43 is further configured to perform synchronous simulation using a plurality of virtual sensors corresponding to each processing delay, to obtain a plurality of first input signals.

[0144] In one possible implementation, the processing delay is determined by the difference between a first processing time and a second processing time, wherein the first processing time is the processing time of the virtual sensor in the sensor simulator, and the second processing time is the preset real processing time of the real sensor corresponding to the virtual sensor.

[0145] In one possible implementation, the device 40 further includes: a second analog circuit 45 and a second sending circuit 46;

[0146] A second simulation circuit 45 is used to simulate using a virtual sensor corresponding to the processing delay based on the first driving state to obtain a second input signal if any processing delay does not meet the preset condition;

[0147] The second sending circuit 46 is used to send one or more second input signals to the sensor simulator with a delay based on the processing delay.

[0148] In one possible implementation manner, the first input signal or the second input signal is obtained by an input signal simulator using a ray tracing algorithm through at least one GPU simulation.

[0149] In one possible implementation, the first driving state includes a first position, a first speed and a first acceleration of the virtual object to be tested at time t, and the prediction circuit is further used to predict the first driving state of the virtual object to be tested based on a processing delay and using a Kalman filtering method to obtain a second driving state, wherein the second driving state includes a second position, a second speed and a second acceleration of the virtual object to be tested at t+T, and T is the processing delay.

[0150] It should be understood that the above Figure 4 The division of the various modules of the simulation test device shown is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. For example, the detection module can be a separately established processing element, or it can be integrated in a chip of an electronic device. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. In the implementation process, each step of the above method or each of the above modules can be completed by an integrated logic circuit of hardware in the processor element or instructions in software form.

[0151] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more digital singnal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0152] Figure 5 FIG. 5 is a schematic diagram of a structure of an embodiment of an electronic device 50 of the present application; wherein the electronic device 50 may be the input signal simulator 200 described above. Figure 5 As shown, the electronic device 50 may be a data processing device or a circuit device built into the data processing device. The electronic device 50 may be used to execute the present application. Figure 1-Figure 3 The illustrated embodiments provide functions / steps in the method.

[0153] like Figure 5 As shown, electronic device 50 is embodied in the form of a general purpose computing device.

[0154] The electronic device 50 may include: one or more processors 510; a communication interface 520; a memory 530; a communication bus 540 connecting different system components (including the memory 530 and the processor 510), a database 550; and one or more computer programs.

[0155] The one or more computer programs are stored in the memory, and the one or more computer programs include instructions. When the instructions are executed by the electronic device, the electronic device performs the following steps:

[0156] Get the processing delay of each virtual sensor;

[0157] Determine whether each processing delay meets the preset conditions;

[0158] If any processing delay satisfies a preset condition, the first driving state is predicted based on the processing delay to obtain a second driving state;

[0159] Based on each second driving state, use a virtual sensor corresponding to the processing delay to perform simulation to obtain one or more first input signals, wherein each first input signal corresponds to each virtual sensor one by one;

[0160] One or more first input signals are sent to a sensor simulator.

[0161] In one possible implementation manner, when the instruction is executed by the electronic device, the electronic device performs a simulation using a virtual sensor corresponding to the processing delay to obtain one or more first input signals, including:

[0162] A plurality of virtual sensors corresponding to each of the processing delays are used to perform synchronous simulation to obtain a plurality of first input signals.

[0163] In one possible implementation, the processing delay is determined by the difference between a first processing time and a second processing time, wherein the first processing time is the processing time of the virtual sensor in the sensor simulator, and the second processing time is the preset real processing time of the real sensor corresponding to the virtual sensor.

[0164] In one possible implementation manner, when the instruction is executed by the electronic device, the electronic device further performs the following steps:

[0165] If any processing delay does not meet the preset condition, based on the first driving state, a virtual sensor corresponding to the processing delay is used for simulation to obtain a second input signal;

[0166] The one or more second input signals are sent to the sensor simulator with a delayed time based on the processing delay.

[0167] In one possible implementation, the sensor simulator is used to receive a first input signal or a second input signal, and perform calculations based on a preset front-end model and a preset algorithm of the virtual sensor to obtain an output signal. The preset front-end model of the virtual sensor is Y=G*X+N+I, wherein Y is the output signal of the front-end model, X is the first input signal or the second input signal, G is the gain of the front end of the virtual sensor, N is the noise of the front end of the virtual sensor, and I is the interference introduced by the front end of the virtual sensor.

[0168] In one possible implementation, the virtual scene is obtained by simulating a virtual scene simulator using at least one CPU and / or at least one GPU, and the first input signal or the second input signal is obtained by simulating an input signal simulator using a ray tracing algorithm through at least one GPU.

[0169] In one possible implementation, the first driving state includes a first position, a first speed, and a first acceleration of the virtual object to be measured at time t. When the instruction is executed by the electronic device, the electronic device predicts the first driving state based on the processing delay, and the step of obtaining the second driving state includes:

[0170] Based on the processing delay, the Kalman filtering method is used to predict the first driving state of the virtual object to be tested to obtain a second driving state, wherein the second driving state includes the second position, second speed and second acceleration of the virtual object to be tested at t+T, and T is the processing delay.

[0171] In one possible implementation, the autonomous driving test architecture also includes a digital simulator, a driving system and a power system simulator. The digital simulator is used to receive the output signal sent by the sensor simulator and send the output signal to the driving system. The driving system is used to determine the driving decision based on the output signal. The power system simulator is used to simulate the driving decision to obtain a third driving state, and the third driving state is fed back to the virtual scene simulator, so that the virtual object to be tested updates the first driving state based on the third driving state.

[0172] In one possible implementation, the virtual sensor includes at least one of a millimeter wave radar virtual sensor, a lidar virtual sensor, an infrared virtual sensor, and a camera virtual sensor.

[0173] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 50. In other embodiments of the present application, the electronic device 50 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0174] It is understandable that, in order to realize the above functions, the electronic device 50 includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present invention.

[0175] The embodiment of the present application can divide the functional modules of the above-mentioned electronic device etc. according to the above-mentioned method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present invention is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0176] Through the description of the above implementation methods, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0177] Each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0178] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as flash memory, mobile hard disk, read-only memory, random access memory, disk or optical disk.

[0179] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A simulation test method, It is characterized in that The method is applied to an input signal simulator, the input signal simulator is located in an autonomous driving test framework, the autonomous driving test framework also includes a virtual scene simulator and a sensor simulator, the virtual scene simulator is used to simulate a virtual scene, the virtual scene includes a virtual object to be tested, the virtual object to be tested includes a first driving state and a plurality of virtual sensors, the first driving state is the driving state of the virtual object to be tested at time t, and the method includes: Obtaining a processing delay of each of the virtual sensors, where the processing delay is determined by a difference between a first processing time and a second processing time, wherein the first processing time is a processing time of the virtual sensor in the sensor simulator, and the second processing time is a preset real processing time of a real sensor corresponding to the virtual sensor; Determining whether each of the processing delays satisfies a preset condition, wherein satisfying the preset condition is used to indicate that the first processing time is greater than the second processing time; If one or more of the processing delays meet the preset condition, predict the driving state of the virtual object to be tested at time t+T to obtain a second driving state, where T is the largest processing delay among the one or more processing delays; Based on each of the second driving states, use a virtual sensor corresponding to the processing delay to perform simulation to obtain one or more first input signals, wherein each of the first input signals corresponds to each of the virtual sensors one by one; sending one or more of the first input signals to the sensor simulator; The sending one or more of the first input signals to the sensor simulator comprises: Sending a first input signal corresponding to the target virtual sensor to the sensor simulator with a delay based on the target delay; The target virtual sensor is all virtual sensors except the virtual sensor corresponding to T, and the target delay is determined by the difference between the processing delay of the target virtual sensor and T.

2. The method according to claim 1, It is characterized in that The using a virtual sensor corresponding to the processing delay to perform simulation to obtain one or more first input signals comprises: A plurality of virtual sensors corresponding to each of the processing delays are used to perform synchronous simulation to obtain a plurality of first input signals.

3. The method according to claim 1 or 2, It is characterized in that The method further comprises: If any of the processing delays does not satisfy the preset condition, based on the first driving state, a virtual sensor corresponding to the processing delay is used for simulation to obtain a second input signal; One or more of the second input signals are sent to the sensor simulator with a delayed signal based on the processing delay.

4. The method according to claim 3, It is characterized in that The sensor simulator is used to receive the first input signal or the second input signal, and perform calculation based on the preset front-end model and preset algorithm of the virtual sensor to obtain an output signal. The preset front-end model of the virtual sensor is Y=G*X+N+I, wherein Y is the output signal of the front-end model, X is the first input signal or the second input signal, G is the gain of the front end of the virtual sensor, N is the noise of the front end of the virtual sensor, and I is the interference introduced by the front end of the virtual sensor.

5. The method according to claim 3, It is characterized in that The virtual scene is obtained by simulation by the virtual scene simulator using at least one CPU and / or at least one GPU, and the first input signal or the second input signal is obtained by simulation by the input signal simulator using a ray tracing algorithm through at least one GPU.

6. The method according to claim 1, It is characterized in that The first driving state includes a first position, a first speed and a first acceleration of the virtual object to be measured at the time t, and the second driving state includes a second position, a second speed and a second acceleration of the virtual object to be measured at the time t+T.

7. The method according to claim 1, It is characterized in that The autonomous driving test architecture also includes a digital simulator, a driving system and a power system simulator. The digital simulator is used to receive the output signal sent by the sensor simulator and send the output signal to the driving system. The driving system is used to determine the driving decision based on the output signal. The power system simulator is used to simulate the driving decision to obtain a third driving state, and feed the third driving state back to the virtual scene simulator, so that the virtual object to be tested updates the first driving state based on the third driving state.

8. The method according to claim 1, It is characterized in that The virtual sensor includes at least one of a millimeter wave radar virtual sensor, a laser radar virtual sensor, an infrared virtual sensor and a camera virtual sensor.

9. A simulation test device, It is characterized in that The method is applied to an input signal simulator, wherein the input signal simulator is located in an autonomous driving test framework, wherein the autonomous driving test framework further includes a virtual scene simulator and a sensor simulator, wherein the virtual scene simulator is used to simulate a virtual scene, wherein the virtual scene includes a virtual object to be tested, wherein the virtual object to be tested includes a first driving state and a plurality of virtual sensors, wherein the first driving state is a driving state of the virtual object to be tested at time t, and includes: A receiving circuit, configured to obtain a processing delay of each of the virtual sensors, wherein the processing delay is determined by a difference between a first processing time and a second processing time, wherein the first processing time is a processing time of the virtual sensor in the sensor simulator, and the second processing time is a preset real processing time of a real sensor corresponding to the virtual sensor; A prediction circuit, used for judging whether each of the processing delays satisfies a preset condition, wherein the satisfying of the preset condition is used to indicate that the first processing time is greater than the second processing time; if one or more of the processing delays satisfies the preset condition, predicting the driving state of the virtual object to be tested at time t+T to obtain a second driving state, wherein T is the largest processing delay among the one or more processing delays; a first simulation circuit, configured to perform simulation based on each of the second driving states using a virtual sensor corresponding to the processing delay to obtain one or more first input signals, wherein each of the first input signals corresponds one-to-one to each of the virtual sensors; a first sending circuit, configured to send one or more of the first input signals to the sensor simulator; The first sending circuit is further used to send the first input signal corresponding to the target virtual sensor to the sensor simulator with a delay based on the target delay; The target virtual sensor is all virtual sensors except the virtual sensor corresponding to T, and the target delay is determined by the difference between the processing delay of the target virtual sensor and T.

10. The device according to claim 9, It is characterized in that The first simulation circuit is also used to use multiple virtual sensors corresponding to each of the processing delays to perform synchronous simulation to obtain multiple first input signals.

11. The device according to claim 9 or 10, It is characterized in that The device also includes: a second simulation circuit, configured to perform simulation using a virtual sensor corresponding to the processing delay based on the first driving state to obtain a second input signal if any of the processing delays does not satisfy the preset condition; The second sending circuit is used to send one or more of the second input signals to the sensor simulator with a delay based on the processing delay.

12. The device according to claim 11, It is characterized in that The first input signal or the second input signal is obtained by the input signal simulator through at least one GPU simulation using a ray tracing algorithm.

13. The device according to claim 9, It is characterized in that The first driving state includes a first position, a first speed and a first acceleration of the virtual object to be measured at the time t, and the second driving state includes a second position, a second speed and a second acceleration of the virtual object to be measured at the time t+T.

14. A simulation test system, It is characterized in that include: Virtual scene simulator, input signal simulator, sensor simulator, digital simulator and system synchronization module; among them, The virtual scene simulator is used to simulate a virtual scene, the virtual scene includes a virtual object to be tested, the virtual object to be tested includes a first driving state and a plurality of virtual sensors, the first driving state is the driving state of the virtual object to be tested at time t; The input signal simulator is used to obtain the processing delay of each of the virtual sensors, and the processing delay is determined by the difference between the first processing time and the second processing time, wherein the first processing time is the processing time of the virtual sensor in the sensor simulator, and the second processing time is the preset real processing time of the real sensor corresponding to the virtual sensor; judging whether each of the processing delays satisfies a preset condition, wherein the satisfying of the preset condition is used to characterize that the first processing time is greater than the second processing time; if one or more of the processing delays satisfies the preset condition, predicting the driving state of the virtual object to be tested at time t+T to obtain a second driving state, wherein T is the largest processing delay among the one or more processing delays; based on each of the second driving states, using the virtual sensor corresponding to the processing delay for simulation to obtain one or more first input signals, wherein each of the first input signals corresponds one-to-one to each of the virtual sensors; sending one or more of the first input signals to the sensor simulator; The sensor simulator is used to receive the first input signal and perform calculation based on a preset front-end model and a preset algorithm of the virtual sensor to obtain an output signal; The digital simulator is used to receive the output signal sent by the sensor simulator; The system synchronization module is used to provide a synchronization clock to the virtual scene simulator, the input signal simulator, the sensor simulator and the digital simulator; The input signal simulator is also used to send the first input signal corresponding to the target virtual sensor to the sensor simulator with a delay based on the target delay; The target virtual sensor is all virtual sensors except the virtual sensor corresponding to T, and the target delay is determined by the difference between the processing delay of the target virtual sensor and T.

15. The system according to claim 14, It is characterized in that The input signal simulator is further used to perform synchronous simulation using a plurality of virtual sensors corresponding to each of the processing delays to obtain a plurality of first input signals.

16. The system according to claim 14 or 15, It is characterized in that The input signal simulator is also used to simulate, based on the first driving state, using a virtual sensor corresponding to the processing delay to obtain a second input signal if any of the processing delays does not meet the preset conditions; and send one or more of the second input signals to the sensor simulator with a delay based on the processing delay.

17. The system according to claim 16, It is characterized in that The sensor simulator is also used to receive a second input signal. The preset front-end model of the virtual sensor is Y=G*X+N+I, wherein Y is the output signal of the front-end model, X is the first input signal or the second input signal, G is the gain of the front end of the virtual sensor, N is the noise of the front end of the virtual sensor, and I is the interference introduced by the front end of the virtual sensor.

18. The system according to claim 16, It is characterized in that The virtual scene is obtained by simulation by the virtual scene simulator using at least one CPU and / or at least one GPU, and the first input signal or the second input signal is obtained by simulation by the input signal simulator using a ray tracing algorithm through at least one GPU.

19. The system according to claim 14, It is characterized in that The first driving state includes a first position, a first speed and a first acceleration of the virtual object to be measured at the time t, and the second driving state includes a second position, a second speed and a second acceleration of the virtual object to be measured at the time t+T.

20. The system according to claim 14, It is characterized in that It also includes driving system and power system simulators, including: The digital simulator is also used to send the output signal to the driving system; The driving system is used to determine a driving decision based on the output signal; The power system simulator is used to simulate the driving decision to obtain a third driving state, and feed the third driving state back to the virtual scene simulator, so that the virtual object to be tested updates the first driving state based on the third driving state.

21. The system according to claim 14, It is characterized in that The virtual sensor includes at least one of a millimeter wave radar virtual sensor, a laser radar virtual sensor, an infrared virtual sensor and a camera virtual sensor.

Images