Simulation test method, device and system

Abstract

The present embodiment provides a simulation testing method, apparatus, and system, which relate to the field of simulation testing. The method includes the steps of: acquiring a processing delay for each virtual sensor; determining whether each processing delay satisfies a preset condition; if any processing delay satisfies the preset condition, predicting a first operating state based on the processing delay to acquire a second operating state; performing emulation based on each second operating state by using a virtual sensor corresponding to the processing delay to acquire one or more first input signals, where each first input signal has a one-to-one correspondence with each virtual sensor; and transmitting the one or more first input signals to a sensor emulator. The method provided in the present embodiment can accurately simulate the behavior and performance of a sensor, improve the accuracy of sensor simulation, and improve the efficiency of simulation testing.

Description

[Technical field]

[0001] TECHNICAL FIELD Embodiments of the present application relate to the field of simulation testing, and in particular to a simulation testing method, apparatus, and system. [Background technology]

[0002] Assisted driving and autonomous driving technologies are rapidly developing and commercialized. The use of assisted driving and autonomous driving technologies will greatly change the way people transport and will have a great impact on people's work and life cycle. By using assisted driving and autonomous driving, the detection capabilities of sensors on vehicles and the autonomous driving capabilities of vehicles are important for safe driving in various scenarios. The relevant capabilities need to be tested in various scenarios to ensure that the capabilities are reliable. Current test methods usually include open road driving tests, closed field driving tests, and simulation tests. It is difficult to cover various test scenarios based on the road driving test mode. Analysis shows that more than 100 million kilometers of road tests are needed to ensure the coverage of various scenarios, but the efficiency is relatively low. In addition, due to safety considerations, it is difficult to repeat highly dangerous scenario tests in field tests. However, pure simulation tests such as Waymo's CarCraft cannot test the detection capabilities of sensors, the capabilities of the frame and power system, and the cooperation between the capabilities of the frame and power system and the autonomous driving algorithm. Therefore, the consistency between the simulation test and the road test cannot be effectively ensured. Summary of the Invention

[0003] SUMMARY OF THE DISCLOSURE The embodiments of the present application provide a simulation test method, apparatus, and system, providing a manner of performing simulation testing in a hypothetical scenario.

[0004] According to a first aspect, an embodiment of the present application provides a simulation testing method applied to an input signal emulator, the input signal emulator is located in an autonomous driving test architecture, the autonomous driving test architecture further includes a virtual scenario emulator and a sensor emulator, the virtual scenario emulator is configured to emulate a virtual scenario, the virtual scenario includes a virtual object to be tested, the virtual object to be tested includes a first driving state and a plurality of virtual sensors, and the method includes:

[0005] A processing delay for each virtual sensor is obtained. Specifically, the processing delay may be the difference between the processing time of the virtual sensor in the emulator and the processing time of the real sensor in the real environment.

[0006] It is determined whether each processing delay satisfies a preset condition. Specifically, the processing delay can be a positive value or a negative value. For example, if the processing time of the virtual sensor in the emulator is longer than the processing time of the real sensor in the real environment, the processing delay is a positive value. Or, if the processing time of the virtual sensor in the emulator is shorter than the processing time of the real sensor in the real environment, the processing delay is a negative value. Therefore, whether the processing delay satisfies the preset condition can be determined by determining whether the processing delay is a positive value or a negative value. Each virtual sensor corresponds to a preset front-end model and a preset algorithm of the virtual sensor. Therefore, the virtual sensor includes a processing delay of the virtual sensor. That is, it needs to be determined whether the processing delay of each virtual sensor satisfies the preset condition.

[0007] If any processing delay satisfies the preset condition, the first driving state is predicted based on the processing delay to obtain a second driving state. Specifically, the first driving state may include a position, a velocity, and an acceleration of the virtual object under test. Thus, when the processing delay of any virtual sensor satisfies the preset condition, e.g., the processing delay of the virtual sensor is a positive value, a prediction may be performed based on the processing delay of the virtual sensor to obtain a second driving state. The second driving state is a predicted state of the virtual sensor, and the second driving state includes a position, a velocity, and an acceleration at a future time point.

[0008] An emulation is performed based on each second operating state by using a virtual sensor corresponding to a processing delay to obtain one or more first input signals, each of the first input signals having a one-to-one correspondence with each virtual sensor. Specifically, the input signal emulator may perform an emulation based on the second operating state of each virtual sensor to obtain a first input signal.

[0009] The one or more first input signals are sent to a sensor emulator.

[0010] In this embodiment, the processing delay of the virtual sensor is determined, and delay compensation for the processing delay is performed, so that the behavior and performance of the sensor can be accurately simulated, the accuracy of the sensor simulation is improved, and the efficiency of the simulation test is improved.

[0011] In a possible implementation, the emulation is performed by using a virtual sensor corresponding to a processing delay, and the one or more first input signals are obtained, comprising:

[0012] A synchronous emulation is performed by using a plurality of virtual sensors corresponding to different processing delays, and a plurality of first input signals are obtained. Specifically, when a plurality of virtual sensors simultaneously perform emulation and obtain a first input signal, the signals obtained through emulation by each virtual sensor can be synchronized, and a plurality of synchronized first input signals are obtained. In this way, the signals can be synchronized, thereby improving the simulation test accuracy.

[0013] In this embodiment, by synchronizing the first input signal, the simulation test accuracy can be improved.

[0014] In a possible implementation, the processing delay is determined by the difference between a first processing time and a second processing time, where the first processing time is a processing time of a virtual sensor in the sensor emulator and the second processing time is a preset real processing time of a real sensor corresponding to the virtual sensor.

[0015] In this embodiment, the difference between the processing time of the virtual sensor in the sensor emulator and the preset real processing time of the real sensor is used as the processing delay, so that the performance of the real sensor can be accurately simulated.

[0016] In a possible implementation, the method includes the following.

[0017] If any processing delay does not satisfy the preset condition, emulation is performed based on the first operating state by using a virtual sensor corresponding to the processing delay to obtain a second input signal; and transmission of the one or more second input signals to the sensor emulator based on the processing delay is delayed. Specifically, if the processing delay of the virtual sensor does not satisfy the preset condition, for example, the processing delay is a negative value, i.e., the virtual sensor does not obtain the second predicted state, the input signal emulator may perform emulation based on the first operating state to obtain a second input signal, and transmission of the second input signal is delayed based on the processing delay to compensate for the processing delay.

[0018] In this embodiment, the processing delay may be compensated for by delaying the transmission of the second input signal, so that the performance of a real sensor may be accurately simulated.

[0019] In a possible implementation, the sensor emulator is configured to receive the first input signal or the second input signal, and perform a calculation based on a preset front-end model of the virtual sensor and a preset algorithm to obtain an output signal, where the preset front-end model of the virtual sensor is Y=G*X+N+I, where Y is the output signal of the front-end model, X is the first input signal or the second input signal, G is the gain of the front-end of the virtual sensor, N is the noise of the front-end of the virtual sensor, and I is the interference introduced by the front-end of the virtual sensor.

[0020] In this embodiment, a preset front-end model and a preset algorithm are configured in the virtual sensor, so that emulation is performed on the real sensor and the performance of the real sensor can be more accurately simulated.

[0021] In a possible implementation, the virtual scenario is obtained through emulation by a virtual scenario emulator by using at least one CPU and / or at least one GPU, and the first input signal or the second input signal is obtained through emulation by an input signal emulator according to a ray tracing algorithm by using at least one GPU.

[0022] In this embodiment, emulation is performed on the signals by using hardware units such as at least one CPU and / or at least one GPU, which increases the speed of signal emulation and thereby improves the efficiency of simulation testing.

[0023] In a possible implementation, the first driving state includes a first position, a first velocity, and a first acceleration of the test target virtual object at a time t, and predicting the first driving state based on a processing delay to obtain the second driving state includes:

[0024] The first driving state of the test subject virtual object is predicted based on the processing delay by using a Kalman filtering method to obtain a second driving state, the second driving state including a second position, a second velocity, and a second acceleration of the test subject virtual object at t+T, where T is the processing delay.

[0025] In this embodiment, processing delays can be effectively compensated for by performing predictions for position, velocity, and acceleration at future times, and as a result, simulation test accuracy can be improved.

[0026] In a possible implementation, the autonomous driving testing architecture further includes a digital emulator, a driving system, and a dynamics system emulator, where the digital emulator is configured to receive the output signal sent by the sensor emulator and send the output signal to the driving system, the driving system is configured to determine a driving decision based on the output signal, and the dynamics system emulator is configured to perform an emulation of the driving decision to obtain a third driving state, feed back the third driving state to the virtual scenario emulator, and enable the virtual object to be tested to update the first driving state based on the third driving state.

[0027] In this embodiment, by introducing a digital emulator, a driving system, and a power system emulator, the driving judgment can be obtained according to the output signal, and the driving state of the test target virtual object can be updated according to the driving judgment. Thus, a closed loop of the simulation test is formed, and the simulation test efficiency can be improved.

[0028] In a possible implementation, the virtual sensor includes at least one of a millimeter wave radar virtual sensor, a laser radar virtual sensor, an infrared virtual sensor, or a camera virtual sensor.

[0029] In this embodiment, the simulation test can be performed for different sensors by introducing multiple virtual sensors, so that the flexibility of the test can be improved and the simulation test efficiency can be further improved.

[0030] According to a second aspect, an embodiment of the present application provides a simulation test apparatus applied to an input signal emulator, the input signal emulator being located in an autonomous driving test architecture, the autonomous driving test architecture further including a virtual scenario emulator and a sensor emulator, the virtual scenario emulator being configured to emulate a virtual scenario, the virtual scenario including a test target virtual object, the test target virtual object including a first driving state and a plurality of virtual sensors, the apparatus comprising: a receiving circuit configured to obtain a processing delay of each virtual sensor; a predicting circuit configured to determine whether each processing delay satisfies a preset condition, and to predict the first driving state based on the processing delay to obtain a second driving state if any processing delay satisfies the preset condition; a first emulation circuit configured to perform emulation based on each second driving state by using a virtual sensor corresponding to the processing delay to obtain one or more first input signals, where each first input signal has a one-to-one correspondence with each virtual sensor; and a first transmitting circuit configured to transmit the one or more first input signals to the sensor emulator.

[0031] In a possible implementation, the first emulation circuit is further configured to perform synchronous emulation by using a plurality of virtual sensors corresponding separately to processing delays to obtain a plurality of first input signals.

[0032] In a possible implementation, the processing delay is determined by the difference between a first processing time and a second processing time, where the first processing time is a processing time of a virtual sensor in the sensor emulator and the second processing time is a preset real processing time of a real sensor corresponding to the virtual sensor.

[0033] In a possible implementation, the apparatus further includes a second emulation circuit configured to perform emulation based on the first operating condition by using a virtual sensor corresponding to the processing delay and obtain a second input signal if any processing delay does not satisfy a preset condition; and a second transmission circuit configured to delay transmission of one or more second input signals to the sensor emulator based on the processing delay.

[0034] In a possible implementation, the first input signal or the second input signal is obtained through emulation by an input signal emulator according to a ray tracing algorithm by using at least one GPU.

[0035] In a possible implementation, the first driving state includes a first position, a first velocity, and a first acceleration of the virtual object under test at a time t, and the prediction circuit is further configured to predict the first driving state of the virtual object under test based on the processing delay by using a method of Kalman filtering to obtain a second driving state, the second driving state including a second position, a second velocity, and a second acceleration of the virtual object under test at t+T, where T is the processing delay.

[0036] According to a third aspect, an embodiment of the present application provides a simulation test system, including a virtual scenario emulator, an input signal emulator, a sensor emulator, a digital emulator, and a system synchronization module.

[0037] The virtual scenario emulator is configured to emulate a virtual scenario, the virtual scenario including a virtual object to be tested, the virtual object to be tested including a first driving condition and a plurality of virtual sensors.

[0038] The input signal emulator is configured to obtain a processing delay for each virtual sensor; determine whether each processing delay satisfies a preset condition; if any processing delay satisfies the preset condition, predict a first operating state based on the processing delay to obtain a second operating state; perform emulation based on each second operating state by using a virtual sensor corresponding to the processing delay to obtain one or more first input signals, where each first input signal has a one-to-one correspondence with each virtual sensor; and send the one or more first input signals to the sensor emulator.

[0039] The sensor emulator is configured to receive a first input signal and perform a calculation based on a preset front-end model of the virtual sensor and a preset algorithm to obtain an output signal.

[0040] The digital emulator is configured to receive the output signal transmitted by the sensor emulator.

[0041] The system synchronization module is configured to provide a synchronized clock to each of the virtual scenario emulator, the input signal emulator, the sensor emulator, and the digital emulator.

[0042] In a possible implementation, the input signal emulator is further configured to perform synchronous emulation by using a plurality of virtual sensors corresponding separately to processing delays to obtain a plurality of first input signals.

[0043] In a possible implementation, the processing delay is determined by the difference between a first processing time and a second processing time, where the first processing time is a processing time of a virtual sensor in the sensor emulator and the second processing time is a preset real processing time of a real sensor corresponding to the virtual sensor.

[0044] In a possible implementation, the input signal emulator is further configured to: perform emulation based on the first operating condition by using a virtual sensor corresponding to the processing delay to obtain a second input signal if any processing delay does not satisfy a preset condition; and delay transmission of the one or more second input signals to the sensor emulator based on the processing delay.

[0045] In a possible implementation, the sensor emulator is further configured to receive a second input signal. The preset front-end model of the virtual sensor is Y=G*X+N+I, where Y is the output signal of the front-end model, X is the first input signal or the second input signal, G is the gain of the front-end of the virtual sensor, N is the noise of the front-end of the virtual sensor, and I is the interference introduced by the front-end of the virtual sensor.

[0046] In a possible implementation, the virtual scenario is obtained through emulation by a virtual scenario emulator by using at least one CPU and / or at least one GPU, and the first input signal or the second input signal is obtained through emulation by an input signal emulator according to a ray tracing algorithm by using at least one GPU.

[0047] In a possible implementation, the first driving state includes a first position, a first velocity, and a first acceleration of the virtual object under test at a time t, and the input signal emulator is further configured to predict the first driving state of the virtual object under test based on the processing delay by using a method of Kalman filtering to obtain a second driving state, the second driving state including a second position, a second velocity, and a second acceleration of the virtual object under test at t+T, where T is the processing delay.

[0048] In a possible implementation, the system further includes a driving system and a power system emulator.

[0049] The digital emulator is further configured to send an output signal to the driving system.

[0050] The driving system is configured to determine a driving decision based on the output signal.

[0051] The power system emulator is configured to perform emulation on the driving decisions to obtain a third driving state and feed back the third driving state to the virtual scenario emulator, enabling the tested virtual object to update the first driving state based on the third driving state.

[0052] In a possible implementation, the virtual sensor includes at least one of a millimeter wave radar virtual sensor, a laser radar virtual sensor, an infrared virtual sensor, or a camera virtual sensor.

[0053] According to a fourth aspect, an embodiment of the present application provides a computer readable storage medium storing a computer program which, when running on a computer, enables the computer to carry out the method according to the first aspect.

[0054] According to a fifth aspect, an embodiment of the present application provides a computer program, which, when executed by a computer, is used to perform the method according to the first aspect.

[0055] In a possible design, all or a part of the program in the fifth aspect may be stored in a storage medium enclosed with the processor, or all or a part of the program may be stored in memory not enclosed with the processor. [Brief description of the drawings]

[0056] [Figure 1] FIG. 1 is a schematic diagram of a system architecture according to an embodiment of the present application.

[0057] [Diagram 2] 1 is a schematic flow chart of a simulation test method according to an embodiment of the present application.

[0058] [Diagram 3] FIG. 2 is a schematic diagram of state prediction according to an embodiment of the present application;

[0059] [Figure 4] 1 is a schematic diagram of a structure of a simulation test device according to an embodiment of the present application;

[0060] [Diagram 5] FIG. 1 is a schematic diagram of a structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0061] The following describes the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means "or". For example, A / B may represent A or B. In this specification, "and / or" only describes the relationship of association to describe related objects, and represents that three relationships may exist. For example, A and / or B may represent the following three cases: only A exists, both A and B exist, and only B exists.

[0062] The terms "first" and "second" referred to below are intended merely for descriptive purposes and should not be understood as an indication or suggestion of the relative importance of the technical features depicted, or an implicit indication of their number. Thus, a feature qualified by "first" or "second" may explicitly or implicitly include one or more features. In the description of the embodiments of the present application, unless otherwise specified, "multiple of" means two or more than two.

[0063] Currently, in order to improve the efficiency of assisted driving test and autonomous driving test, and improve the consistency between simulation test and road test, a hardware closed-loop solution is needed that can emulate various scenarios in a laboratory and use autonomous driving software and algorithms to perform joint testing on sensor capabilities and the capabilities of the frame and power system of a test vehicle. Scenario emulation in a laboratory can be performed by an emulation interface-based method and a digital interface-based method. In the emulation interface-based method, the emulation signal received by each sensor is generated through scenario emulation, and the signal is sent to each sensor through an emulation interface. When the emulation interface method is used, there is a problem that the system is complex and the solution is immature. For example, in the emulated scenario, multiple target radar echoes can only emulate a small number of targets. This cannot meet the requirements. However, the system that uses a probe wall solution to emulate tens of target echoes is very expensive and complex, and there is no mature solution. In addition, there is no solution for the LiDAR echo signals of multiple targets in the emulated scenario. Therefore, a scenario emulation method of digital interface is commonly used. In this emulation method, the generated scenario emulation signal is directly sent to the processing system of assisted driving and autonomous driving through the digital interface without passing through the actual sensor. A common solution is to establish a behavioral or statistical model of each sensor to emulate the effect of the sensor's performance. This method cannot implement the synchronous emulation of multi-sensor behavior in the same scenario. In addition, the performance of a sensor in a specific scenario cannot be accurately described based on the behavioral or statistical model. Also, some solutions propose the use of a physical model combined with a sensor algorithm to emulate the sensor performance in a specific scenario.However, this solution does not consider the impact of the performance of the sensor emulation front end. In addition, the synchronization and delay between multiple sensor emulation signals are not considered. The fusion of multiple sensors in autonomous driving requires that the perceptions of different sensors for the same scenario can be emulated synchronously during scenario emulation. In addition, the excess delay generated in scenario emulation needs to be able to be compensated in the scenario emulation system to ensure that the hardware of the closed-loop system is consistent with the actual road test behavior.

[0064] In view of the above problems, the embodiments of the present application provide a simulation test method, which can effectively solve the delay problem caused by simulation sensors in digital scenario simulation testing, and thus the behaviors and performances of multiple sensors can be accurately and synchronously simulated in a digital scenario emulator.

[0065] 1-3, a simulation test method provided in one embodiment of the present application will be described. FIG. 1 shows a system architecture provided in this embodiment of the present application. Please refer to FIG. 1. The system architecture includes a virtual scenario emulator 100, an input signal emulator 200, a sensor emulator 300, a digital emulator 400, an assisted driving and autonomous driving system 500 (for ease of explanation, the "assisted driving and autonomous driving system" will be referred to as "driving system" below), a power system emulator 600, and a system synchronization module 700.

[0066] The virtual scenario emulator 100 is configured to build a virtual scenario and send virtual scenario information to the input signal emulator 200. In a specific implementation, the virtual scenario emulator 100 may be a computer or another type of computing device. This is not limited in the present application. The virtual scenario emulator 100 may include a high-speed Ethernet network adapter supporting the IEEE 1588 protocol, one or more central processing units (CPUs), or a graphics processing unit (GPU) to ensure real-time performance of the emulation. The construction of the virtual scenario may be implemented by using a scenario emulation software installed in the virtual scenario emulator 100. The scenario emulation software may be a commercial 3D emulation software or an open source 3D emulation software. This embodiment of the present application is not particularly limited thereto.

[0067] The virtual scenario may include a virtual scenario established by using a virtual three-dimensional model. The virtual three-dimensional model emulates a real object in the real world. For example, the real object may include a vehicle, a person, an animal, a tree, a road, a building, or the like, or may include another real object. This embodiment of the present application is not particularly limited in this respect. It can be understood that the three-dimensional model can be considered as a virtual object. In a specific implementation, a user usually creates a virtual object in a virtual scenario and uses the virtual object as a test object for testing. Thus, the test object may be a first virtual object, and all virtual objects except the first virtual object in the virtual scenario may be considered as a second virtual object. For example, the first virtual object may be a vehicle, and thus the autonomous driving and assisted driving performance of the vehicle may be tested. For example, in the traveling process of the vehicle, the second virtual object around the vehicle becomes an obstacle for the vehicle, which consequently affects the traveling of the vehicle, and further affects the driving judgment of the vehicle, such as acceleration, deceleration, turning, and stopping. By using the aforementioned scenario emulation software, the virtual scenario emulator 100 can send information about virtual objects in a virtual scenario to the input signal emulator 200. The information about the virtual objects can include information such as the coordinates, material, and lighting of the virtual objects. A standardized interface is used for information transmission between the virtual scenario emulator 100 and the input signal emulator 200, ensuring that the aforementioned system architecture is independent of a specific scenario emulation software.

[0068] It may be understood that the first virtual object may alternatively be an unmanned aerial vehicle or another type of autonomous driving device, although the present embodiment of the present application is not particularly limited in this regard.

[0069] In addition, for each real object, when a signal such as visible light, millimeter wave radar, laser radar, and infrared radar is irradiated on the surface of the real object, physical phenomena such as reflection, scattering, and diffraction occur. Therefore, the virtual scenario emulator 100 can establish a corresponding physical model for the virtual object based on the physical phenomenon. In addition, the virtual scenario emulator 100 can further establish model parameters corresponding to the material of the virtual object. The material of the virtual object matches the material used by the real object.

[0070] To test the first virtual object, the virtual scenario emulator 100 may further place virtual sensors on the first virtual object. The virtual sensors are configured to emulate acquiring input signals in the virtual environment. The input signals acquired by the various virtual sensors in the virtual environment may be emulated by using physical models and model parameters.

[0071] The input signal emulator 200 is configured to emulate input signals acquired by various virtual sensors in the virtual environment and send the input signals to the sensor emulator 300 for processing. The input signals can be emulated based on the driving state and virtual scenario information of the first virtual object. The virtual sensors can include a millimeter wave radar virtual sensor, a laser radar virtual sensor, an infrared virtual sensor, and a camera virtual sensor, or can include another type of virtual sensor. This embodiment of the present application is not particularly limited thereto. In a specific implementation, the input signal emulator 200 can be a computer or another type of computing device. This is not specifically limited thereto in the present application. The input signal emulator 200 can include a high-speed Ethernet network adapter that supports the IEEE1588 protocol and one or more GPUs. In addition, the emulation process of acquiring the input signal can be implemented by using input signal emulation software. It may be understood that in the input signal emulation software, the format, type, and related parameters of the virtual sensor may be configured, the information about the virtual object and the material parameters of the related virtual object transmitted from the virtual scenario emulator 100 are received, and emulation is performed for different types of virtual sensors based on the driving state (e.g., the position, velocity, and acceleration of the first virtual object), the physical model, and the model parameters according to the ray tracing algorithm. For example, emulation calculations (e.g., including but not limited to effects such as reflection, scattering, and diffraction) are performed by each real sensor on the physical effects occurring on the surface of the virtual object, so that the input signal in the virtual environment may be obtained through emulation. For example, the transmitter of the virtual sensor transmits a signal, and after the physical effects occur between the transmission signal and the surface of the virtual object, the transmission signal is returned to the receiver of the virtual sensor, and the input signal of the virtual sensor is obtained. In addition, the GPU may process the emulation of the input signal to ensure the real-time nature of the emulation.For example, the input signal emulator 200 may emulate taking images in a virtual environment by using a virtual camera, or emulate detecting virtual objects in a virtual environment by using millimeter wave radar.

[0072] The sensor emulator 300 is configured to emulate the behavior and performance of each virtual sensor. For example, the sensor emulator 300 may receive the input signal acquired by the input signal emulator 200 and process the input signal in a processing manner that emulates a real sensor, so as to acquire an output signal, which may be sent to the digital emulator 400 for processing. In a particular implementation, the sensor emulator 300 may be a computer or another type of computing device. This is not specifically limited in this application. The sensor emulator 300 may include a high-speed Ethernet network adapter that supports the IEEE 1588 protocol, one or more Field Programmable Gate Array (FPGA) acceleration cards, or a GPU. For example, the sensor emulator 300 may emulate the performance of the front end of each virtual sensor and the algorithm of each virtual sensor. The emulation of the performance of the front end of each virtual sensor may be performed in a modeling manner. For example, a model of each virtual sensor front-end may be pre-built as Y=G*X+N+I, where X is the input signal of the sensor, Y is the output signal of the sensor front-end, G is the gain of the sensor front-end, N is the noise of the sensor front-end, and I is the interference introduced by the sensor front-end. The output signal Y may be obtained through processing by using a preset algorithm of the virtual sensor.

[0073] It may be understood that a model of the virtual sensor front end is constructed based on a real sensor, and thus emulation may be performed on the real sensor front end by using the virtual sensor in a modeling manner, thereby ensuring that the performance of the virtual sensor front end matches the real sensor front end in terms of statistical properties.

[0074] In addition, the algorithm of the virtual sensor can be executed by directly using the algorithm of the real sensor, which will not be described again here. For different algorithms of the real sensor, the sensor emulator 300 can execute the emulation by using corresponding FPGA acceleration cards or GPU cards to ensure the real-time and computing power requirements of the emulation.

[0075] The digital emulator 400 is configured to receive the output signal sent by the sensor emulator 300 and can send the output signal to the driving system 500. In a particular implementation, the digital emulator 400 can be a computer or another type of computing device, which is not specifically limited in this application. The digital emulator 400 can include a high-speed Ethernet network adapter supporting the IEEE 1588 protocol, one or more FPGA acceleration cards, and multiple interfaces.

[0076] The interfaces include physical and digital interfaces between the digital emulator 400 and real sensors. The digital emulator 400 can use the physical and digital interfaces to receive real data collected by different real sensors and perform testing by replaying the real collected data. The physical interfaces can include, but are not limited to, a Controller Area Network (CAN) interface, a Mobile Industry Processor Interface (MIPI), an Ethernet interface, a Gigabit Multimedia Serial Link (GSML) interface, a Flat Panel Display Link (FPLINK) interface, a Local Interconnect Network (LIN) interface, a 100M-T1 interface, and the like. The digital interface may include, but is not limited to, a Peripheral Component Interconnect Express (PCIE) interface, a Serial Advanced Technology Attachment (SATA) interface, a Fast Ethernet interface, a digital fiber interface, and the like.

[0077] In addition, the interface may further include a physical interface between the digital emulator 400 and the driving system 500, and a digital interface between the digital emulator 400 and the power system emulator 600. The physical interface may include a CAN interface. For example, the digital emulator 400 may be connected to a CAN bus of the driving system 500 through the CAN interface, so that the digital emulator 400 can send output signals to the driving system 500, and the driving system 500 can perform driving decisions based on the output signals. The digital interface may include an Ethernet interface, which is configured to transmit the driving status of the object under test (e.g., a vehicle) output by the power system emulator 600.

[0078] The driving system 500 is configured to receive the output signal sent by the digital emulator 400 and perform driving judgment based on the output signal. The driving system 500 may be located in a real object to be tested. For example, the driving system 500 may be located in a real vehicle to be tested. In this case, the vehicle is used as the object to be tested. It may be understood that the driving system 500 may also be tested separately. For example, the driving system 500 may be taken out of the real vehicle, and thus the driving system 500 may be used as the object to be tested. In addition, the driving judgment may include operations such as acceleration, braking, deceleration, and turning, or may include other operations. This is not particularly limited in this embodiment. The driving system 500 may then send the driving judgment to the power system emulator 600, which may then perform emulation based on the driving judgment and update the driving states of the real object to be tested and the virtual object.

[0079] The power system emulator 600 is configured to receive the driving decision sent by the driving system 500, emulate the dynamic characteristics of the real vehicle based on the driving decision, output the driving state corresponding to the real vehicle, and feed back the driving state to the first virtual object in the virtual scenario, so that the first virtual object can be updated based on the driving state and the emulation test can be completed. In a specific implementation, the power system emulator 600 can be a computer or another type of computing device. This is not specifically limited in the present application. The power system emulator 600 can include multiple interfaces. The interfaces can include a physical interface (e.g., a CAN interface) between the power system emulator 600 and the driving system 500, and a digital interface (e.g., an Ethernet interface) between the power system emulator 600 and the virtual scenario simulator. The power system emulator 600 can receive the driving decision sent by the driving system 500 through the physical interface, and the power system emulator 600 can send the real vehicle state to the first virtual object in the virtual scenario through the digital interface.

[0080] The system synchronization module 700 is configured to provide a synchronous clock to each of the virtual scenario emulator 100, the input signal emulator 200, the sensor emulator 300, and the digital emulator 400, and ensure clock synchronization among the virtual scenario emulator 100, the input signal emulator 200, the sensor emulator 300, and the digital emulator 400. In a specific implementation, the system synchronization module 700 may be, for example, a high-speed Ethernet switch supporting the 1588 synchronization protocol, or may be an application-specific synchronization module. This embodiment of the present application is not particularly limited thereto.

[0081] It may be appreciated that the virtual scenario emulator 100, the input signal emulator 200, the sensor emulator 300, and the digital emulator 400 may exchange data with each other by using a high speed Ethernet switch or another high speed data connection device.

[0082] 2 is a schematic flow chart of an embodiment of a simulation test method according to an embodiment of the present application. The method includes the following steps:

[0083] Step 101 : Construct a virtual scenario by using a virtual scenario emulator 100 .

[0084] Specifically, the virtual scenario may be constructed by using the virtual scenario emulator 100. The virtual scenario may include a scenario to be tested, and the scenario to be tested may include a virtual object and a material corresponding to the virtual object. For example, the virtual object may include a vehicle, a person, an animal, a tree, a road, a building, or the like, or may include another object. This embodiment of the present application is not particularly limited in this respect. In addition, the virtual object to be tested may further be determined in the virtual scenario. For example, the virtual object to be tested may be a vehicle.

[0085] To perform emulation testing on the virtual object, a plurality of virtual sensors can be further configured for the virtual object. Thus, the first parameters of the virtual sensors can be configured by using the input signal emulator 200 to emulate the acquisition of input signals by the virtual sensors. When the first parameters of the plurality of virtual sensors are configured, the configured first parameter information can include the number of sensors, the type of sensors, the assembly parameters of the sensors on the real vehicle (e.g., the installation height and angle of the sensors on the real vehicle), and the physical parameters of the sensors (e.g., the number, position, and direction of antennas transmitted and received by the millimeter wave radar, the frequency, operation method, and number of lines of the laser radar, the field of view and focal length of the camera).

[0086] It may be understood that in the constructed virtual scenario, the virtual object to be tested may be considered as a first virtual object, and all virtual objects in the virtual scenario other than the first virtual object may be considered as second virtual objects. For example, if a vehicle in the virtual scenario is used as the first virtual object, i.e., the virtual object to be tested, another virtual object (e.g., a vehicle, a person, an animal, a tree, a road, or a building) may be used as the second virtual object.

[0087] It should be noted that in addition to configuring parameters for the virtual scenario and virtual objects, second parameters of the virtual sensors may also be configured in the sensor emulator 300. When second parameters of the multiple virtual sensors are configured, the configured second parameter information may include the number of sensors, the type of sensors, front-end parameters of the sensors (e.g., front-end gain G, front-end noise N, and interference introduced by the front-end I), sensor processing delay, sensor algorithms, and the like.

[0088] Step 102 : The virtual scenario emulator 100 sends the virtual scenario information to the input signal emulator 200 .

[0089] Specifically, the virtual scenario information may include related information of all virtual objects in the virtual scenario, and the related information may include coordinate positions, material information (e.g., plastic or metal), lighting conditions, and the like. In the present embodiment of the present application, there is no particular limitation thereon.

[0090] Step 103: The input signal emulator 200 emulates obtaining a plurality of input signals of a first virtual object.

[0091] Specifically, when the first virtual object moves in the virtual scenario, multiple input signals around the first virtual object may be obtained through emulation in the input signal emulator 200 by using virtual sensors. The virtual sensors may include one or more of a millimeter wave radar virtual sensor, a laser radar virtual sensor, an infrared virtual sensor, and a camera virtual sensor. The input signals may include echo signals and / or images. For example, an echo signal may be obtained by using a millimeter wave radar virtual sensor, a laser radar virtual sensor, or an infrared virtual sensor, or a photographed image may be obtained by using a camera virtual sensor. The input signals may be associated with the driving state S of the first virtual object and the virtual scenario information according to a ray tracing algorithm. t It can be understood that the input signal can also be obtained through calculation by using another algorithm. In the present embodiment of the present application, there is no particular limitation thereon.

number

[0092] Xi is the position of the first virtual object, Vi is the velocity of the first virtual object, and Ai is the acceleration of the first virtual object. It can be understood that the state of the first virtual object may further include other variables. In the present embodiment of the present application, there is no particular limitation thereon.

[0093] It can be understood that each input signal has a one-to-one correspondence with each virtual sensor. For example, the echo input signal A can be obtained by using a millimeter wave radar virtual sensor, the echo input signal B can be obtained by using a laser radar virtual sensor, and the image input signal C can be obtained by using a camera virtual sensor.

[0094] In addition, when multiple input signals are acquired by using the input signal emulator 200, the multiple input signals can be further synchronized to ensure that the input signal emulator 200 can simultaneously acquire input signals acquired by multiple virtual sensors for the same scenario.

[0095] Furthermore, when the sensor emulator 300 processes multiple input signals, a simulation processing time Tf is generated. When the real sensor processes the input signals, a real processing time Tz is generated. Since the virtual sensor performs emulation on the real sensor and is different from the real sensor, it can be understood that the simulation processing time Tf generated by the virtual sensor is different from the real processing time Tz generated by the real sensor. For example, the simulation processing time Tf can be greater than or equal to the real processing time Tz, or the simulation processing time Tf can also be less than or equal to the real processing time Tz. When the simulation processing time Tf does not match the real processing time Tz, a delay compensation can be performed on the input signals in the input signal emulator 200, so that the simulation test can better simulate the real scenario. The delay refers to the difference between the simulation processing time Tf and the real processing time Tz.

[0096] Therefore, the input signal emulator 200 may further obtain a simulation processing time Tf of each virtual sensor in the sensor emulator 300. The simulation processing time Tf has a one-to-one correspondence with the virtual sensor. For example, a simulation processing time Tf1 of the millimeter wave radar virtual sensor, a simulation processing time Tf2 of the laser radar virtual sensor, a simulation processing time Tf3 of the camera virtual sensor, and the like may be obtained. Next, a real processing time Tz of the real sensor corresponding to the virtual sensor may further be obtained. The real processing time Tz has a one-to-one correspondence with the real sensor. For example, a real processing time Tz1 of the millimeter wave radar real sensor, a real processing time Tz2 of the laser radar real sensor, a real processing time Tz3 of the camera real sensor, and the like may be obtained.

[0097] The input signal emulator 200 may then compare the simulated processing time Tf of each virtual sensor with the corresponding real processing time Tz.

[0098] If the simulation processing time Tf is smaller than the real processing time Tz, the first input signal may be obtained through emulation based on the current state St of the first virtual object, and the transmission of the first input signal corresponding to the virtual sensor may be delayed, where delay=Tz-Tf. For example, the simulation processing time Tf of the laser radar virtual sensor is assumed to be 5 ms, and the real processing time Tz of the laser radar real sensor is 10 ms. In this case, the first input signal corresponding to the laser radar virtual sensor may be transmitted after a delay of Tz-Tf=10-5=5 ms. Thus, the simulation processing time may be matched with the processing time of the real sensor, thereby improving the simulation test accuracy.

[0099] When the simulation processing time Tf is greater than the real processing time Tz, i.e., when the processing time of the virtual sensor is longer than the processing time of the real sensor, a delay is caused. Therefore, the delay needs to be compensated. During a specific implementation, a prediction is performed for the driving state of the first virtual object, so that the input signal emulator 200 can perform emulation based on the predicted driving state and obtain a second input signal. The second input signal can be a predicted input signal obtained after a Tf-Tz period. The prediction can be performed by using a Kalman filtering method, or can be performed by using another prediction method. In the present embodiment of the present application, there is no particular limitation on this.

[0100] For example, the driving state of the first virtual object at time t is S t where:

number

[0101] Xi is the position of the first virtual object, Vi is the velocity of the first virtual object, and Ai is the acceleration of the first virtual object. It can be understood that the state of the first virtual object may further include other variables. In the present embodiment of the present application, there is no particular limitation on this. In this case, the predicted driving state of the first virtual object at time t+T is S t+T =Fi*S t +Bi*Ui+Ni, where

number

number

number

[0102] Ni is the state prediction noise. In this way, the driving state of the first virtual object at a future time point can be predicted. Then, based on the driving state of the first virtual object at a future time point, the input signal emulator 200 can perform emulation on the input signal to obtain a second input signal.

[0103] For example, it is assumed that the simulation processing time of the millimeter wave radar virtual sensor is Tf1=10 ms, i.e., the millimeter wave radar virtual sensor needs to process the input signal for 10 ms to obtain the first input signal of the millimeter wave radar; or the real processing time of the millimeter wave radar real sensor is Tz1=5 ms, i.e., the millimeter wave radar real sensor needs to process the input signal for 5 ms to obtain the first input signal of the millimeter wave radar. The simulation processing time of the laser radar virtual sensor is Tf2=12 ms, i.e., the laser radar virtual sensor needs to process the input signal for 10 ms to obtain the first input signal of the laser radar; or the real processing time of the laser radar real sensor is Tz2=8 ms, i.e., the laser radar real sensor needs to process the input signal for 5 ms to obtain the first input signal of the laser radar. Since the simulation delay Ty1 of the millimeter wave radar virtual sensor is Tf1-Tz1=10-5=5 ms, and the simulation delay Ty2 of the laser radar virtual sensor is the simulation processing time Tf2-Tz2=12-8=4 ms, i.e., Ty1>Ty2, a prediction can be performed for the driving state of the first virtual object based on the maximum simulation delay (e.g., Ty1). Based on the predicted driving state, emulation is performed separately by using the millimeter wave radar virtual sensor and the laser radar virtual sensor, and the second input signal of the millimeter wave radar and the second input signal of the laser radar are obtained. In this way, it can be ensured that effective delay compensation can be obtained for all the first input signals.

[0104] The millimeter wave radar virtual sensor is used as an example for explanation. Figure 3 is a schematic diagram of prediction of the second input signal. As shown in Figure 3, the driving state of the first virtual object at time t is S1, and the millimeter wave radar virtual sensor on the first virtual object can obtain the input signal 101 through emulation at time t based on the driving state S1. Then, prediction is performed for the driving state of the first virtual object at time t+T1, so that the driving state of the first virtual object at time t+T1 is S2, where T1 is a delay, i.e., the difference between the simulation processing time Tf1 of the millimeter wave radar virtual sensor and the real processing time Tz1 of the millimeter wave radar real sensor. The millimeter wave radar virtual sensor can obtain the second input signal 102 through emulation at time t+T1 based on the driving state S2, so that the prediction of the second input signal 102 can be completed and the delay compensation for the input signal can be completed.

[0105] It can be understood that the laser radar virtual sensor can also predict the input signal of the laser radar in the manner of Fig. 3 and obtain the second input signal of the laser radar, which will not be described in detail again here.

[0106] Optionally, after completing the delay compensation for the input signal, the input signal emulator 200 may further correct the simulation delay of each virtual sensor. The millimeter wave radar virtual sensor and the laser radar virtual sensor are used as examples. After the delays of the millimeter wave radar virtual sensor and the laser radar virtual sensor are compensated, the delay compensation time of the laser radar virtual sensor is the delay compensation time of the millimeter wave radar virtual sensor, for example, the simulation delay Ty2 of the laser radar virtual sensor=Tf2-Tz2=12-8=4 ms, but the delay compensation time of the laser radar virtual sensor is 5 ms, so that the delay compensation time of the laser radar virtual sensor does not match the simulation delay Ty2 of the laser radar virtual sensor. In this case, the input signal emulator 200 may send a second input signal predicted by the laser radar after the delay T2, so that the cumulative time of the delay T2 and the simulation processing time corresponds to the predicted second input signal, thereby emulating the performance of the real sensor. T2 is the difference between the delay compensation time and the simulation delay. For example, it is assumed that the delay compensation time of the laser radar virtual sensor is 5 ms, and the simulation delay of the laser radar virtual sensor is 4 ms. In this case, T2 = delay compensation time - simulation delay = 5-4 = 1 ms. That is, the second output signal of the laser radar is transmitted after a delay of 1 ms. The real processing time of the real sensor of the laser radar is Tz2 = 8 ms, and the delay compensation time of the laser radar virtual sensor is 5 ms, that is, the second input signal is predicted to be obtained after Tz2 + 5 = 8 + 5 = 13 ms. However, the simulation processing time of the laser radar virtual sensor is Tf2 = 12 ms. Therefore, the second input signal can be transmitted after a delay of 13-12 = 1 ms, so as to ensure that the predicted second input signal is consistent with the simulation processing time of the sensor emulator 300.

[0107] Step 104 : The input signal emulator 200 sends a plurality of input signals to the sensor emulator 300 .

[0108] Specifically, after obtaining the multiple input signals of the first virtual object, the input signal emulator 200 may send the multiple input signals to the sensor emulator 300. The input signals may include a first input signal and / or a second input signal.

[0109] Step 105: The sensor emulator 300 processes a plurality of input signals and outputs a plurality of output signals.

[0110] Specifically, after receiving the multiple input signals sent by the input signal emulator 200, the sensor emulator 300 may process the multiple input signals to obtain multiple output signals. The processing process may be performed based on a front-end model of the virtual sensor (e.g., Y=G*X+N+I) and a preset algorithm of the virtual sensor.

[0111] It should be noted that each virtual sensor may use a different preset front-end model. The specific implementation of the front-end model is not particularly limited in this embodiment of the present application.

[0112] It can be understood that each input signal has a one-to-one correspondence with each output signal. For example, an echo input signal A acquired by a millimeter wave radar virtual sensor can be input to a front-end model of a millimeter wave radar virtual sensor for processing by using a preset millimeter wave radar sensor algorithm, and an echo output signal A can be obtained. Alternatively, an echo input signal B acquired by a laser radar virtual sensor can be input to a front-end model of a laser radar virtual sensor for processing by using a preset laser radar sensor algorithm, and an echo output signal B can be obtained. Alternatively, an image input signal C acquired by a camera virtual sensor can be input to a front-end model of a camera virtual sensor for processing by using a preset camera sensor algorithm, and an image output signal can be obtained.

[0113] Step 106 : Send the plurality of output signals to the digital emulator 400 .

[0114] Specifically, after processing the multiple input signals to obtain multiple output signals, the sensor emulator 300 may send the multiple output signals to the digital emulator 400 .

[0115] Step 107: The digital emulator 400 receives the plurality of output signals for processing and transmits the plurality of output signals to the operating system 500.

[0116] Specifically, the digital emulator 400 may receive the output signals corresponding to each virtual sensor and sent by the sensor emulator 300. To test the driving performance of a real vehicle, the digital emulator 400 may further send the aforementioned output signals to a driving system 500 of the real vehicle.

[0117] Step 108: The driving system 500 determines a driving decision based on the plurality of output signals.

[0118] Specifically, after receiving the multiple output signals sent by the digital emulator 400, the driving system 500 makes a driving decision. The driving decision may include operations such as accelerating, decelerating, braking, and turning, or may include other driving decisions. In the present embodiment of the present application, there is no particular limitation thereon.

[0119] Step 109: Send driving decision to power system emulator 600.

[0120] Step 110: The power system emulator 600 emulates the driving state St' of a real vehicle based on the driving decisions.

[0121] Specifically, the driving state St' corresponds to a driving decision. For example, when the driving decision is acceleration, the driving state of the real vehicle emulated by the power system emulator 600 is acceleration driving; or when the driving decision is braking, the driving state of the real vehicle emulated by the power system emulator 600 is a stopping state after braking.

[0122] Step 111: Feed back the driving state St' to the virtual scenario emulator 100, allowing the virtual scenario emulator 100 to update the driving state St of the first virtual object based on St'.

[0123] Specifically, after obtaining the driving state St', the power system emulator 600 may feed back the driving state St' to the virtual scenario emulator 100, allowing the virtual scenario emulator 100 to update the driving state St of the first virtual object based on St'. For example, the following vehicle is used as an example. It is assumed that a test vehicle (first virtual object) follows another vehicle (second virtual object) to test the autonomous driving performance of the first virtual object. After the second virtual object brakes, the distance between the first virtual object and the second virtual object becomes closer. By analyzing the sensor data (e.g., the second output signal) acquired by the virtual sensor on the first virtual object, a driving decision (e.g., the driving system 500 decides to brake) can be determined, and the decision to brake can be fed back to the first virtual object, thereby completing the emulation test of the whole system.

[0124] Step 112: The virtual scenario emulator 100 updates the driving state of the first virtual object based on the driving state St′.

[0125] In this embodiment of the present application, a prediction is performed on the operating state of the object under test based on the delay between the simulation sensor and the real sensor, and thus the delay is compensated, which can effectively solve the delay problem caused by the simulation sensor in the digital scenario simulation test, and the behavior and performance of multiple sensors can be accurately and synchronously simulated in the digital scenario emulator.

[0126] Fig. 4 is a schematic diagram of the structure of one embodiment of a simulation test apparatus according to the present application. As shown in Fig. 4, the simulation test apparatus 40 is applied to an input signal emulator. The input signal emulator is located in an autonomous driving test architecture, the autonomous driving test architecture further includes a virtual scenario emulator and a sensor emulator, the virtual scenario emulator is configured to emulate a virtual scenario, the virtual scenario includes a test target virtual object, the test target virtual object includes a first driving state and a plurality of virtual sensors, and the apparatus may include a receiving circuit 41, a predicting circuit 42, a first emulation circuit 43, and a first transmitting circuit 44.

[0127] The receiver circuit 41 is configured to obtain the processing delay of each virtual sensor.

[0128] The prediction circuit 42 is configured to determine whether each processing delay satisfies a preset condition, and if any processing delay satisfies the preset condition, predict the first operating state based on the processing delay to obtain a second operating state.

[0129] The first emulation circuit 43 is configured to perform emulation based on each second operating condition by using a virtual sensor corresponding to a processing delay to obtain one or more first input signals, each of which has a one-to-one correspondence with each virtual sensor.

[0130] The first transmitter circuit 44 is configured to transmit one or more first input signals to the sensor emulator.

[0131] In a possible implementation, the first emulation circuit 43 is further configured to perform synchronous emulation by using a plurality of virtual sensors corresponding separately to processing delays to obtain a plurality of first input signals.

[0132] In a possible implementation, the processing delay is determined by the difference between a first processing time and a second processing time, where the first processing time is a processing time of a virtual sensor in the sensor emulator and the second processing time is a preset real processing time of a real sensor corresponding to the virtual sensor.

[0133] In a possible implementation, the device 40 further comprises a second emulation circuit 45 and a second transmission circuit 46 .

[0134] The second emulation circuit 45 is configured to perform emulation based on the first operating condition by using a virtual sensor corresponding to the processing delay and obtain a second input signal if any processing delay does not satisfy the preset condition.

[0135] The second transmit circuit 46 is configured to delay transmitting the one or more second input signals to the sensor emulator based on a processing delay.

[0136] In a possible implementation, the first input signal or the second input signal is obtained through emulation by an input signal emulator according to a ray tracing algorithm by using at least one GPU.

[0137] In a possible implementation, the first driving state includes a first position, a first velocity, and a first acceleration of the virtual object under test at a time t, and the prediction circuit is further configured to predict the first driving state of the virtual object under test based on the processing delay by using a method of Kalman filtering to obtain a second driving state, the second driving state including a second position, a second velocity, and a second acceleration of the virtual object under test at t+T, where T is the processing delay.

[0138] It should be understood that the division of modules of the simulation test apparatus shown in FIG. 4 is merely a logical functional division. In actual implementation, the modules may be all or partly integrated into a physical entity or may be physically separated. In addition, all of the modules may be implemented in the form of software called by a processing element, or may be implemented in the form of hardware. Alternatively, some of the modules may be implemented in the form of software called by a processing element, and another part of the modules may be implemented in the form of hardware. For example, the detection module may be a processing element that is separately located, or may be integrated and implemented in a chip of an electronic device. The implementation of other modules is similar to that of the detection module. In addition, all or part of the modules may be integrated together or implemented independently. In the implementation process, the steps in the above-mentioned method or the above-mentioned modules may be implemented by using a hardware integrated logic circuit in a processing element, or by using instructions in the form of software.

[0139] For example, the modules may be one or more integrated circuits configured to implement the methods described above, such as one or more Application Specific Integrated Circuits (ASICs), one or more Digital Signal Processors (DSPs), or one or more Field Programmable Gate Arrays. In another example, the modules may be integrated together and implemented in the form of a System on a Chip (SOC).

[0140] 5 is a schematic diagram of the structure of an embodiment of an electronic device 50 according to the present application. The electronic device 50 may be the input signal emulator 200 described above. As shown in FIG. 5, the electronic device 50 may be a data processing device, or may be a data processing device built into a circuit device. The electronic device 50 may be configured to perform functions / steps in the methods provided in the embodiments shown in FIG. 1 to FIG. 3 of the present application.

[0141] As shown in FIG. 5, electronic device 50 is represented in the form of a general purpose computing device.

[0142] The electronic device 50 may include one or more processors 510, a communication interface 520, a memory 530, a communication bus 540 connecting different system components (including the memory 530 and the processor 510), a database 550, and one or more computer programs.

[0143] The one or more computer programs are stored in the memory, and the one or more computer programs include instructions, and when the instructions are executed by the electronic device, the electronic device is capable of performing the following steps: obtaining a processing delay for each virtual sensor; determining whether each processing delay satisfies a preset condition; if any processing delay satisfies the preset condition, predicting a first operating state based on the processing delay and obtaining a second operating state; performing emulation based on each second operating state by using a virtual sensor corresponding to the processing delay, and obtaining one or more first input signals, where each first input signal has a one-to-one correspondence with each virtual sensor; and sending the one or more first input signals to a sensor emulator.

[0144] In a possible implementation, when the instructions are executed by the electronic device, the step of the electronic device performing emulation to obtain one or more first input signals by using a virtual sensor corresponding to a processing delay includes performing synchronous emulation by using multiple virtual sensors separately corresponding to the processing delays to obtain multiple first input signals.

[0145] In a possible implementation, the processing delay is determined by the difference between a first processing time and a second processing time, where the first processing time is a processing time of a virtual sensor in the sensor emulator and the second processing time is a preset real processing time of a real sensor corresponding to the virtual sensor.

[0146] In a possible implementation, when the instructions are executed by the electronic device, the electronic device may further perform the following steps: if any processing delay does not satisfy a preset condition, performing emulation based on the first operating state by using a virtual sensor corresponding to the processing delay and obtaining a second input signal; and delaying transmission of one or more second input signals to the sensor emulator based on the processing delay.

[0147] In a possible implementation, the sensor emulator is configured to receive the first input signal or the second input signal, and perform a calculation based on a preset front-end model of the virtual sensor and a preset algorithm to obtain an output signal, where the preset front-end model of the virtual sensor is Y=G*X+N+I, where Y is the output signal of the front-end model, X is the first input signal or the second input signal, G is the gain of the front-end of the virtual sensor, N is the noise of the front-end of the virtual sensor, and I is the interference introduced by the front-end of the virtual sensor.

[0148] In a possible implementation, the virtual scenario is obtained through emulation by a virtual scenario emulator by using at least one CPU and / or at least one GPU, and the first input signal or the second input signal is obtained through emulation by an input signal emulator according to a ray tracing algorithm by using at least one GPU.

[0149] In a possible implementation, the first driving state includes a first position, a first velocity, and a first acceleration of the test virtual object at a time t, and when the instructions are executed by the electronic device, the electronic device is enabled to perform a step of predicting the first driving state based on a processing delay to obtain a second driving state including predicting the first driving state of the test virtual object based on the processing delay and obtaining the second driving state by using a Kalman filtering method, where the second driving state includes a second position, a second velocity, and a second acceleration of the test virtual object at t+T, where T is the processing delay.

[0150] In a possible implementation, the autonomous driving testing architecture further includes a digital emulator, a driving system, and a dynamics system emulator, where the digital emulator is configured to receive the output signal sent by the sensor emulator and send the output signal to the driving system, the driving system is configured to determine a driving decision based on the output signal, and the dynamics system emulator is configured to perform an emulation of the driving decision to obtain a third driving state, feed back the third driving state to the virtual scenario emulator, and enable the virtual object to be tested to update the first driving state based on the third driving state.

[0151] In a possible implementation, the virtual sensor includes at least one of a millimeter wave radar virtual sensor, a laser radar virtual sensor, an infrared virtual sensor, or a camera virtual sensor.

[0152] It may be understood that the interfacing relationships between modules shown in this embodiment of the present invention are merely illustrative examples and do not constitute limitations on the structure of the electronic device 50. In some other embodiments of the present application, the electronic device 50 may alternatively use a different interfacing scheme than the aforementioned embodiment or may use a combination of interfacing schemes.

[0153] To implement the aforementioned functions, the electronic device 50 may be understood to include corresponding hardware structures and / or software modules for performing the functions. In combination with the examples described in the embodiments disclosed herein, those skilled in the art should easily understand that the units, algorithms, and steps may be implemented by hardware, or a combination of hardware, and computer software in the embodiments of the present application. Whether a function is performed by hardware or by hardware running computer software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but the implementation should not be considered to go beyond the scope of the embodiments of the present invention.

[0154] The division of the functional modules can be performed for the electronic device based on the above-mentioned method example in the embodiment of the present application. For example, the functional modules can be obtained through division of the corresponding functions, or two or more functions can be integrated into one processing module. The integrated module can be implemented in the form of hardware, or in the form of a software function module. It should be noted that in the embodiment of the present invention, the module division is an example, and is merely a logical function division. During actual implementation, another division method can be used.

[0155] The above description of implementation allows those skilled in the art to understand that for the purpose of simple and concise description, the division of the above-mentioned functional modules is taken as an example for explanation. In practical application, the above-mentioned functions can be allocated and implemented into multiple different modules based on requirements, that is, the internal structure of the device is divided into multiple different functional modules to implement all or part of the above-mentioned functions. For detailed operation processes of the above-mentioned system, device and unit, please refer to the corresponding processes in the above-mentioned method embodiments. Here, details will not be described again.

[0156] The functional units in the embodiments of the present application may be integrated into one processing unit, each of these units may exist physically alone, or two or more units may be integrated into one unit. The integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0157] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, the integrated unit may be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application may be essentially implemented in the form of a software product, or a part of the contribution to the prior art, or all or a part of the technical solutions. The computer software product is stored in a storage medium and includes a plurality of instructions for instructing a computer device (which may be a personal computer, a server, a network device, or the like) or a processor to execute all or a part of the steps of the method described in the embodiments of the present application. The aforementioned storage medium includes any medium capable of storing program code, such as a flash memory, a removable hard disk, a read-only memory, a random access memory, a magnetic disk, or an optical disk.

[0158] The above description is merely a specific implementation of the present application, and is not intended to limit the scope of protection of the present application. Any variations or replacements within the technical scope disclosed in the present application shall be included in the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the scope of protection of the claims. Other possible items (Item 1) 1. A simulation testing method applied to an input signal emulator, the input signal emulator being located in an autonomous driving test architecture, the autonomous driving test architecture further comprising a virtual scenario emulator and a sensor emulator, the virtual scenario emulator being configured to emulate a virtual scenario, the virtual scenario including a virtual object to be tested, the virtual object to be tested including a first driving state and a plurality of virtual sensors, the method comprising: obtaining a processing delay for each virtual sensor; determining whether each processing delay satisfies a preset condition; predicting the first operating state based on the processing delay to obtain a second operating state if any processing delay satisfies the preset condition; performing an emulation based on each second operating condition by using a virtual sensor corresponding to the processing delay to obtain one or more first input signals, where each first input signal has a one-to-one correspondence with each virtual sensor; and transmitting the one or more first input signals to the sensor emulator; A method for providing the above. (Item 2) performing emulation by using a virtual sensor corresponding to the processing delay to obtain one or more first input signals, performing synchronous emulation by using the plurality of virtual sensors corresponding to the processing delays separately to obtain the plurality of first input signals; The method according to item 1, comprising: (Item 3) 3. The method of claim 1 or 2, wherein the processing delay is determined by a difference between a first processing time and a second processing time, the first processing time being a processing time of the virtual sensor in the sensor emulator, and the second processing time being a preset real processing time of a real sensor corresponding to the virtual sensor. (Item 4) The method further comprises: if any processing delay does not satisfy the preset condition, performing emulation based on the first operating condition by using the virtual sensor corresponding to the processing delay to obtain a second input signal; and delaying the transmission of one or more second input signals to the sensor emulator based on the processing delay. 4. The method according to any one of items 1 to 3, comprising: (Item 5) 5. The method of claim 4, wherein the sensor emulator is configured to receive the first input signal or the second input signal, perform calculations based on a preset front-end model of the virtual sensor and a preset algorithm to obtain an output signal, and the preset front-end model of the virtual sensor is Y=G*X+N+I, where Y is the output signal of the front-end model, X is the first input signal or the second input signal, G is a gain of the front-end of the virtual sensor, N is the noise of the front-end of the virtual sensor, and I is the interference introduced by the front-end of the virtual sensor. (Item 6) 5. The method according to claim 4, wherein the virtual scenario is obtained through emulation by the virtual scenario emulator by using at least one CPU and / or at least one GPU, and the first input signal or the second input signal is obtained through emulation by the input signal emulator according to a ray tracing algorithm by using at least one GPU. (Item 7) The first driving state includes a first position, a first velocity, and a first acceleration of the test target virtual object at a time t, and the step of predicting the first driving state based on the processing delay to obtain a second driving state includes: predicting the first driving state of the test subject virtual object based on the processing delay by using a Kalman filtering method to obtain the second driving state, where the second driving state includes a second position, a second velocity, and a second acceleration of the test subject virtual object at time t+T, where T is the processing delay; The method according to item 1, comprising: (Item 8) Item 1, wherein the autonomous driving test architecture further includes a digital emulator, a driving system, and a power system emulator, the digital emulator configured to receive an output signal transmitted by the sensor emulator and transmit the output signal to the driving system, the driving system configured to determine a driving decision based on the output signal, and the power system emulator configured to perform emulation on the driving decision to obtain a third driving state and feed back the third driving state to the virtual scenario emulator to enable the test subject virtual object to update the first driving state based on the third driving state. (Item 9) 9. The method according to any one of claims 1 to 8, wherein the virtual sensor includes at least one of a millimeter wave radar virtual sensor, a laser radar virtual sensor, an infrared virtual sensor, or a camera virtual sensor. (Item 10) A simulation test apparatus applied to an input signal emulator, the input signal emulator being located in an autonomous driving test architecture, the autonomous driving test architecture further comprising a virtual scenario emulator and a sensor emulator, the virtual scenario emulator being configured to emulate a virtual scenario, the virtual scenario including a test target virtual object, the test target virtual object including a first driving state and a plurality of virtual sensors, the apparatus comprising: a receiving circuit configured to obtain a processing delay for each virtual sensor; a prediction circuit configured to determine whether each processing delay satisfies a preset condition, and if any processing delay satisfies the preset condition, predict the first operating state based on the processing delay and obtain a second operating state; a first emulation circuit configured to perform emulation based on each second operating condition by using a virtual sensor corresponding to the processing delay to obtain one or more first input signals, where each first input signal has a one-to-one correspondence with each virtual sensor; and a first transmitting circuit configured to transmit the one or more first input signals to the sensor emulator; An apparatus comprising: (Item 11) 11. The apparatus of claim 10, wherein the first emulation circuit is further configured to perform synchronous emulation by using the plurality of virtual sensors that separately correspond to the processing delays to obtain the plurality of first input signals. (Item 12) 12. The apparatus of claim 10 or 11, wherein the processing delay is determined by a difference between a first processing time and a second processing time, the first processing time being a processing time of the virtual sensor in the sensor emulator, and the second processing time being a preset real processing time of a real sensor corresponding to the virtual sensor. (Item 13) The apparatus further comprises: a second emulation circuit configured to, if any processing delay does not satisfy the preset condition, perform emulation based on the first operating condition by using the virtual sensor corresponding to the processing delay to obtain a second input signal; and a second transmitting circuit configured to delay the transmitting of one or more second input signals to the sensor emulator based on the processing delay. 13. The apparatus according to any one of items 10 to 12, comprising: (Item 14) Item 14. The apparatus of item 13, wherein the first input signal or the second input signal is obtained through emulation by the input signal emulator according to a ray tracing algorithm by using at least one GPU. (Item 15) 11. The apparatus of claim 10, wherein the first driving state includes a first position, a first velocity, and a first acceleration of the test target virtual object at a time t, and the prediction circuit is further configured to predict the first driving state of the test target virtual object based on the processing delay by using a Kalman filtering method to obtain the second driving state, wherein the second driving state includes a second position, a second velocity, and a second acceleration of the test target virtual object at t+T, where T is the processing delay. (Item 16) A simulation test system comprising a virtual scenario emulator, an input signal emulator, a sensor emulator, a digital emulator, and a system synchronization module, the virtual scenario emulator is configured to emulate a virtual scenario, the virtual scenario including a virtual object to be tested, the virtual object to be tested including a first operating condition and a plurality of virtual sensors; The input signal emulator is configured to obtain a processing delay for each virtual sensor; determine whether each processing delay satisfies a preset condition; if any processing delay satisfies the preset condition, predict the first operating state based on the processing delay to obtain a second operating state; perform emulation based on each second operating state by using a virtual sensor corresponding to the processing delay to obtain one or more first input signals, where each first input signal has a one-to-one correspondence with a respective virtual sensor; and transmit the one or more first input signals to the sensor emulator; The sensor emulator is configured to receive the first input signal, perform a calculation based on a preset front-end model of the virtual sensor and a preset algorithm, and obtain an output signal; the digital emulator configured to receive the output signal transmitted by the sensor emulator; the system synchronization module is configured to provide a synchronized clock to each of the hypothetical scenario emulator, the input signal emulator, the sensor emulator, and the digital emulator; Simulation test system. (Item 17) 17. The system of claim 16, wherein the input signal emulator is further configured to perform synchronous emulation by using the virtual sensors that correspond separately to the processing delays to obtain the first plurality of input signals. (Item 18) Item 16 or 17, wherein the processing delay is determined by a difference between a first processing time and a second processing time, the first processing time being a processing time of the virtual sensor in the sensor emulator, and the second processing time being a preset real processing time of a real sensor corresponding to the virtual sensor. (Item 19) 19. The system of any one of items 16 to 18, wherein the input signal emulator is further configured to: perform emulation based on the first operating condition by using the virtual sensor corresponding to the processing delay to obtain a second input signal if any processing delay does not satisfy the preset condition; and delay transmission of the one or more second input signals to the sensor emulator based on the processing delay. (Item 20) 20. The system of claim 19, wherein the sensor emulator is further configured to receive the second input signal, and the preset front-end model of the virtual sensor is Y=G*X+N+I, where Y is an output signal of the front-end model, X is the first input signal or the second input signal, G is a gain of the front-end of the virtual sensor, N is noise of the front-end of the virtual sensor, and I is interference introduced by the front-end of the virtual sensor. (Item 21) 20. The system of claim 19, wherein the virtual scenario is obtained through emulation by the virtual scenario emulator by using at least one CPU and / or at least one GPU, and the first input signal or the second input signal is obtained through emulation by the input signal emulator according to a ray tracing algorithm by using at least one GPU. (Item 22) 17. The system of claim 16, wherein the first driving state includes a first position, a first velocity, and a first acceleration of the test target virtual object at a time t, and the input signal emulator is further configured to predict the first driving state of the test target virtual object based on the processing delay by using a Kalman filtering method to obtain the second driving state, wherein the second driving state includes a second position, a second velocity, and a second acceleration of the test target virtual object at t+T, where T is the processing delay. (Item 23) Further comprising a driving system and a power system emulator; The digital emulator is further configured to transmit the output signal to the driving system; The driving system is configured to determine a driving decision based on the output signal; the power system emulator is configured to perform an emulation on the driving decision to obtain a third driving state and feed back the third driving state to the virtual scenario emulator, thereby enabling the virtual object under test to update the first driving state based on the third driving state. Item 17. The system according to item 16. (Item 24) 24. The system of any one of items 16 to 23, wherein the virtual sensor includes at least one of a millimeter wave radar virtual sensor, a laser radar virtual sensor, an infrared virtual sensor, or a camera virtual sensor.

Claims

1. 1. A method of simulation testing applied to an input signal emulator, the input signal emulator being located in an autonomous driving test architecture, the autonomous driving test architecture further comprising a virtual scenario emulator and a sensor emulator, the virtual scenario emulator being configured to emulate a virtual scenario, the virtual scenario including a virtual object to be tested, the virtual object to be tested including a first driving state and a plurality of virtual sensors, the method comprising: obtaining a processing delay for each virtual sensor; determining whether each processing delay satisfies a preset condition; predicting the first operating state based on the processing delay to obtain a second operating state if any processing delay satisfies the preset condition; performing an emulation based on each second operating condition by using a virtual sensor corresponding to the processing delay to obtain one or more first input signals, where each first input signal has a one-to-one correspondence with each virtual sensor; and transmitting the one or more first input signals to the sensor emulator; A method for providing the above.

2. performing emulation by using a virtual sensor corresponding to the processing delay to obtain one or more first input signals, performing synchronous emulation by using the virtual sensors that correspond separately to the processing delays to obtain the first input signals; The method of claim 1 , comprising:

3. 3. The method of claim 1 or 2, wherein the processing delay is determined by a difference between a first processing time and a second processing time, the first processing time being a processing time of the virtual sensor in the sensor emulator, and the second processing time being a preset real processing time of a real sensor corresponding to the virtual sensor.

4. The method further comprises: if any processing delay does not satisfy the preset condition, performing emulation based on the first operating condition by using the virtual sensor corresponding to the processing delay to obtain a second input signal; and delaying the transmission of one or more second input signals to the sensor emulator based on the processing delay. The method according to claim 1 , comprising:

5. 5. The method of claim 4, wherein the sensor emulator is configured to receive the first input signal or the second input signal, perform calculations based on a preset front-end model of the virtual sensor and a preset algorithm to obtain an output signal, and the preset front-end model of the virtual sensor is Y=G*X+N+I, where Y is the output signal of the front-end model, X is the first input signal or the second input signal, G is a gain of a front-end of the virtual sensor, N is noise of the front-end of the virtual sensor, and I is interference introduced by the front-end of the virtual sensor.

6. 5. The method of claim 4, wherein the virtual scenario is obtained through emulation by the virtual scenario emulator by using at least one CPU and / or at least one GPU, and the first input signal or the second input signal is obtained through emulation by the input signal emulator according to a ray tracing algorithm by using at least one GPU.

7. The first driving state includes a first position, a first velocity, and a first acceleration of the test target virtual object at a time t, and the step of predicting the first driving state based on the processing delay to obtain a second driving state includes: predicting the first driving state of the test subject virtual object based on the processing delay by using a Kalman filtering method to obtain the second driving state, where the second driving state includes a second position, a second velocity, and a second acceleration of the test subject virtual object at time t+T, where T is the processing delay; The method of claim 1 , comprising:

8. 2. The method of claim 1, wherein the autonomous driving test architecture further includes a digital emulator, a driving system, and a powered system emulator, the digital emulator configured to receive an output signal transmitted by the sensor emulator and transmit the output signal to the driving system, the driving system configured to determine a driving decision based on the output signal, and the powered system emulator configured to perform emulation on the driving decision to obtain a third driving state and feed back the third driving state to the virtual scenario emulator to enable the test subject virtual object to update the first driving state based on the third driving state.

9. The method of claim 1 , wherein the virtual sensor includes at least one of a millimeter wave radar virtual sensor, a laser radar virtual sensor, an infrared virtual sensor, or a camera virtual sensor.

10. An apparatus for simulation testing applied to an input signal emulator, the input signal emulator being located in an autonomous driving test architecture, the autonomous driving test architecture further comprising a virtual scenario emulator and a sensor emulator, the virtual scenario emulator being configured to emulate a virtual scenario, the virtual scenario including a test target virtual object, the test target virtual object including a first driving state and a plurality of virtual sensors, the apparatus comprising: a receiving circuit configured to obtain a processing delay for each virtual sensor; a prediction circuit configured to determine whether each processing delay satisfies a preset condition, and if any processing delay satisfies the preset condition, to predict the first operating state based on the processing delay and to obtain a second operating state; a first emulation circuit configured to perform an emulation based on each second operating condition by using a virtual sensor corresponding to the processing delay to obtain one or more first input signals, where each first input signal has a one-to-one correspondence with each virtual sensor; and a first transmitting circuit configured to transmit the one or more first input signals to the sensor emulator; An apparatus comprising:

11. 11. The apparatus of claim 10, wherein the processing delay is determined by a difference between a first processing time and a second processing time, the first processing time being a processing time of the virtual sensor in the sensor emulator, and the second processing time being a preset real processing time of a real sensor corresponding to the virtual sensor.

12. The apparatus further comprises: a second emulation circuit configured to, if any processing delay does not satisfy the preset condition, perform emulation based on the first operating condition by using the virtual sensor corresponding to the processing delay to obtain a second input signal; and a second transmitting circuit configured to delay the transmission of the one or more second input signals to the sensor emulator based on the processing delay.

12. The apparatus according to claim 10 or 11, comprising:

13. The apparatus of claim 12 , wherein the first input signal or the second input signal is obtained through emulation by the input signal emulator according to a ray tracing algorithm by using at least one GPU.

14. 11. The apparatus of claim 10, wherein the first driving state includes a first position, a first velocity, and a first acceleration of the test target virtual object at a time t, and the prediction circuitry is further configured to predict the first driving state of the test target virtual object based on the processing delay by using a Kalman filtering method to obtain the second driving state, wherein the second driving state includes a second position, a second velocity, and a second acceleration of the test target virtual object at t+T, where T is the processing delay.

15. A simulation test system comprising: a virtual scenario emulator, an input signal emulator, a sensor emulator, a digital emulator, and a system synchronization module, the virtual scenario emulator configured to emulate a virtual scenario, the virtual scenario including a virtual object to be tested, the virtual object to be tested including a first operating state and a plurality of virtual sensors; The input signal emulator is configured to perform the method of simulation testing provided by any one of claims 1 to 9; The sensor emulator is configured to receive the first input signal, perform a calculation based on a preset front-end model of the virtual sensor and a preset algorithm, and obtain an output signal; the digital emulator configured to receive the output signal transmitted by the sensor emulator; the system synchronization module is configured to provide a synchronized clock to each of the hypothetical scenario emulator, the input signal emulator, the sensor emulator, and the digital emulator; Simulation test system.

Images