Method, device and equipment for vehicle testing, vehicle and medium

Abstract

The embodiment of the invention relates to a method, device and equipment for vehicle testing, a vehicle and a medium. The method includes receiving test data including object test data and vehicle test data, inputting the test data to a camera simulator for a camera to generate first simulation data. The method includes inputting test data to the radar simulator for the radar to generate second analog data according to a processing delay between the camera simulator and the radar simulator for the radar, and testing the system in the vehicle based on the first analog data and the second analog data. According to the method disclosed by the embodiment of the invention, the synchronization of the analog signals of the plurality of sensors can be determined more quickly and conveniently, and the test efficiency is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure generally relate to the field of data processing, and in particular to methods, devices, equipment, vehicles, and media for vehicle testing. Background Art

[0002] With the development of technology, the electronic systems in automobiles are constantly evolving, and these systems play an important role in modern automobiles. Electronic systems in automobiles can include engine control units (ECUs) for monitoring and controlling engine performance, electronic stability control systems (ESCs) for maintaining vehicle stability, anti-lock braking systems (ABSs) for preventing wheel locks, and advanced driver assistance systems (ADASs).

[0003] ADAS is a set of advanced automotive technologies and systems designed to improve driving safety, efficiency and convenience. The ADAS system can use a variety of built-in sensors (such as ultrasonic sensors, millimeter-wave radars, lidars, cameras, antennas, etc.) to sense changes in the surrounding environment in real time during driving, collect data, and detect, identify and track static and dynamic objects. Combined with electronic map data, the ADAS system can perform calculations and analysis to issue warnings of potential dangers to the driver, improving the driving experience.

[0004] In order to make ADAS systems achieve more accurate and timely predictions, many technologies for testing and verifying ADAS systems have emerged. They can use various simulators, sensors and other equipment and algorithms to determine whether ADAS systems can perform the operations and behaviors expected by people. However, there are still many technical problems that need to be solved in the technology of testing and verifying ADAS systems. Summary of the invention

[0005] Embodiments of the present disclosure provide a method, an apparatus, a device, a vehicle, and a medium for a vehicle testing environment.

[0006] According to a first aspect of the present disclosure, a method for vehicle testing is provided, the method comprising receiving test data including object test data and vehicle test data, inputting the test data into a camera simulator for a camera to generate first simulation data, the method comprising inputting the test data into a radar simulator for a radar to generate second simulation data according to a processing delay between the camera simulator and a radar simulator for the radar, and testing a system in the vehicle based on the first simulation data and the second simulation data.

[0007] According to a second aspect of the present disclosure, a vehicle testing device is provided, the device comprising a receiving unit configured to receive test data including object test data and vehicle test data, a first generating unit configured to input the test data into a camera simulator for a camera to generate first simulation data. The device also comprises a second generating unit configured to input the test data into a radar simulator for a radar to generate second simulation data according to a processing delay between the camera simulator and the radar simulator for the radar, and a testing unit configured to test a system in a vehicle based on the first simulation data and the second simulation data.

[0008] According to a third aspect of the present disclosure, an electronic device is provided. The electronic device includes at least one processor; and a memory coupled to the at least one processor and having instructions stored thereon, which, when executed by the at least one processor, causes the device to perform the steps of the method in the first aspect of the present disclosure.

[0009] According to a fourth aspect of the present disclosure, a vehicle is provided. The vehicle includes the electronic device in the third aspect of the present disclosure.

[0010] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided, wherein computer-executable instructions are stored on the computer-readable storage medium, wherein the computer-executable instructions are executed by a processor to implement the steps of the method in the first aspect of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other objects, features and advantages of the present disclosure will become more apparent through a more detailed description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present disclosure.

[0012] Figure 1 A schematic diagram illustrating an example environment in which devices and / or methods according to embodiments of the present disclosure may be implemented;

[0013] Figure 2 A flow chart of a method for vehicle testing according to an embodiment of the present disclosure is illustrated;

[0014] Figure 3A A schematic diagram of a framework for vehicle testing according to an embodiment of the present disclosure is illustrated;

[0015] Figure 3B A schematic diagram illustrating a framework for another vehicle test according to an embodiment of the present disclosure

[0016] Figure 4 A schematic diagram illustrating another example framework for vehicle testing according to an embodiment of the present disclosure is illustrated;

[0017] Figure 5 A schematic diagram illustrating a process of synchronizing received signals of a sensor based on processing delays is shown;

[0018] Figure 6 A schematic diagram illustrating an apparatus for vehicle testing according to an embodiment of the present disclosure; and

[0019] Figure 7 A schematic block diagram of an example device for implementing an embodiment of the present disclosure is illustrated. DETAILED DESCRIPTION

[0020] The embodiments of the present disclosure described below with reference to the accompanying drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure. In addition, before using the technical solutions disclosed in the embodiments of the present disclosure, the type, scope of use, and usage scenarios of the personal information involved in the present disclosure shall be informed to the user in an appropriate manner in accordance with relevant laws and regulations and the user's authorization shall be obtained.

[0021] As described above, in order to make various systems in vehicles such as ADAS systems achieve more accurate and timely predictions as much as possible, many technologies for testing and verifying systems in vehicles have emerged. For example, there are multiple sensors and simulators in ADAS systems, such as sensors for lidar, sensors for cameras, and sensors for satellites, as well as radar simulators for simulating lidar signals, camera simulators for simulating real-time road condition signals, etc.

[0022] Due to the different types of these signals, the speed of generating simulated signals may be different. For example, due to the need for GPU rendering, a camera simulator that simulates real-time road condition signals may take more time to simulate and generate image data signals than a radar simulator that simulates lidar signals. These processes may cause the time and process of generating simulated signals by different simulators to be out of sync.

[0023] Therefore, when testing ADAS systems, it is necessary to fuse different data so that they can be output synchronously. Usually, the synchronization of multiple types of signals or data is achieved by printing timestamps. This process is not only costly, but also has low test efficiency.

[0024] At least to address the above and other potential problems, an embodiment of the present disclosure provides a method for adjusting data in a vehicle test environment. In the method, test data including object test data and vehicle test data is first received, and the test data is input into a camera simulator for a camera to generate first simulation data. The method also includes inputting the test data into a radar simulator for a radar to generate second simulation data based on a processing delay between the camera simulator and a radar simulator for a radar, and testing the system in the vehicle based on the first simulation data and the second simulation data. Through the method implemented by the present disclosure, data synchronization can be determined more quickly and conveniently, test costs can be reduced, test coverage and efficiency can be improved, and multi-ECU interactions can be verified. In addition, through the method implemented by the present disclosure, the posture and position of each sensor and object can be changed in real time, and the sensor signal deviation can be adjusted according to the test requirements, thereby realizing customized testing of the sensor fusion algorithm.

[0025] The embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings. Figure 1 An example test system is shown in which the apparatus and / or methods of embodiments of the present disclosure may be implemented.

[0026] like Figure 1 As shown, the example test system 100 may include a simulator 102. The simulator 102 may simulate various situations and scenarios during the driving process of the car to test, verify and optimize the performance of the ADAS system 112. The simulator 102 is generally used to develop and test new ADAS functions to ensure that they can operate stably under different traffic conditions and situations while shortening the development cycle. The simulator 102 may also include one or more different types of simulators, such as a camera simulator 104, a radar simulator 106, and an ultrasonic simulator. It should be understood that in the context of the present disclosure, the simulator 102 may also be referred to as an emulator 102, the camera simulator 104 may also be referred to as a virtual camera model 104, and the radar simulator 106 may also be referred to as a virtual radar model 106, and the present disclosure does not impose any restrictions on this.

[0027] refer to Figure 1 These simulators can independently output data to the test ADAS system 112 to test whether the ADAS system 112 will perform the expected action or behavior when receiving a single signal. Additionally or alternatively, these simulators can also interact and influence each other to test whether the ADAS system 112 will perform the expected action or behavior when receiving multiple signals at the same time.

[0028] For example, in some embodiments, the simulator 102 can implement the generation of simulation scenarios 108 for creating various traffic scenarios, road conditions, and weather conditions to test the response of the ADAS system 112. The simulator 102 can also help the vehicle under test to be extensively tested in a controlled environment without relying on actual road testing. For example, the simulation scenarios 108 can include different types of roads such as city streets, highways, mountain roads, and various traffic conditions, such as other vehicles, pedestrians, bicycles, etc. These simulation scenarios 108 can simulate various driving situations, including daily driving, emergency situations, bad weather, etc.

[0029] The camera simulator 104 can receive signals or data from the simulated scene 108 and process virtual images based on the data to simulate the field of view and image data of the camera. For example, the camera simulator 104 can process traffic signs, obstacles, and lanes in the simulated scene 108. The camera simulator 104 can help evaluate the performance of the ADAS system 112 in different viewing angles, lighting conditions, and environments.

[0030] In some embodiments, the simulator 102 may also control and simulate data associated with the vehicle dynamics model 110, such as simulating the dynamic characteristics and driving control system of the vehicle to evaluate the impact of the ADAS system 112 function on the vehicle behavior. As an example, the vehicle dynamics model 110 may include, but is not limited to, a set of functions and data associated with the vehicle's motion and handling characteristics, such as the vehicle's speed, acceleration, braking force, steering force and other physical motion characteristics, vehicle stability control for preventing unexpected situations such as skidding or rollover, shock absorption effect, traction control, suspension control, etc. The safety and performance of the ADAS system 112 may be tested by conducting experiments on the vehicle dynamics model 110.

[0031] In some embodiments, the data associated with the vehicle dynamics model 110 may then be input into the radar simulator 106, and the radar simulator 106 may generate corresponding microwave or millimeter wave signals and echo signals based on the data, etc. Additionally or alternatively, in some embodiments, the data of the simulation scene 108 may be consistent with or correspond to the data associated with the vehicle dynamics model 110. Subsequently, the data or signals generated by the camera simulator 104 may be input into the camera sensor 114 of the vehicle under test, and the data or signals generated by the radar simulator 106 may be input into the radar sensor 116 of the vehicle under test.

[0032] The ADAS system 112 can then take corresponding actions based on the data received from the camera sensor 114 and the radar sensor 116. For example, the ADAS system 112 can perform algorithms such as path planning, object detection and tracking to test the decision-making and control capabilities of the ADAS system 112, as well as perform data recording and analysis, and can also record data during the simulation process to analyze and evaluate the performance of the system. Additionally or alternatively, in some embodiments, the ADAS system 112 or the simulator 102 can be implemented by any suitable computing device, including but not limited to a personal computer, a handheld or laptop device, a mobile device, a multi-processor system, a consumer electronic product, a small computer including a distributed computing environment of any of the above systems or devices, etc.

[0033] Combined with the above Figure 1 A block diagram of an example test system 100 in which embodiments of the present disclosure can be implemented is described below. Figure 2 A flowchart of a method 200 for collecting training data according to an embodiment of the present disclosure is described. The method 200 may be Figure 1 The simulator 102 in the system is executed at any suitable computing device.

[0034] At block 202, test data including object test data and vehicle test data is received. As an example, the simulator 102 in the example test system 100 may receive object test data collected from a real road. These object test data may include information about vehicles and pedestrians around the vehicle during driving, road conditions, data associated with traffic lights and signs, and various signal data, etc. The object test data may also be generated by an object model based on pedestrians or vehicles in the experiment.

[0035] In some embodiments, the simulator 102 may receive vehicle test data associated with the test vehicle itself during the vehicle driving process. For example, the vehicle test data may include the speed, engine speed, acceleration, braking force, steering force, etc. of the test vehicle. In some embodiments, these data may then be input into each simulator of the simulator 102 as sample data or test data to generate simulation data for various types of sensors. The above examples are only used to describe the present disclosure, and are not specific limitations of the present disclosure.

[0036] At block 204, the test data is input into a camera simulator for the camera to generate first simulation data. As an example, the example test system 100 may input test data including object test data and vehicle test data into the camera simulator 104 to generate camera simulation data. For example, the example test system 100 may model, generate, or render the virtual scene 108 based on the test data and using a graphics processor (GPU), such as by a 3D modeling technique of point cloud processing or map generation, and the present disclosure does not make any limitation to this.

[0037] Additionally or alternatively, in some embodiments, the virtual scene 108 may be annotated or modified by using computer vision technology to better simulate roads and environments. For example, various parameters such as road sections, vehicle density, traffic lights, signs, buildings, obstacles, etc. may be added or reduced in the virtual scene according to different needs. In some embodiments, the virtual scene 108 may be expanded by collecting test data under different weather conditions, simulating different road conditions, or introducing new traffic scenarios.

[0038] The camera simulator 104 can then process the virtual scene 108 to generate camera simulation data. The camera simulator 104 can simulate the perception performance of different types of cameras to generate data similar to the output of the camera sensor 114, including but not limited to images, depth information, lighting conditions, etc. The parameters that can be adjusted by the camera simulator 104 may include, but are not limited to, field of view angle, resolution, distortion, etc. The camera simulation data can then be input into the camera sensor 114 of the vehicle to be tested for testing the perception and decision-making functions of the ADAS system.

[0039] At block 206, the test data is input to the radar simulator for radar to generate second simulation data according to the processing delay between the camera simulator and the radar simulator for radar. Since different simulators need to call different processing hardware to process the same test data, there may be a processing delay for the same test data.

[0040] For example, when using the camera simulator 104 to process test data, it is necessary to call a graphics processor to model and render the test input data. The whole process may consume a lot of time and system resources, so it takes a long time. When using the radar simulator 106 to process test data, various signals reflected by radar waves after encountering objects or pedestrians can be extracted, or radar data can be directly collected in the process of collecting test data without further processing. The radar simulator 106 does not require rendering, only simple algorithm calculations, so the processing time is shorter.

[0041] Therefore, the process using the radar simulator 106 may consume less time and system resources and take less time. Since the processing time between different simulators is different, the output data of different simulators can be synchronized by adjusting the output time of different simulators. In some embodiments, the method for determining the processing delay between different simulators can be to input the predetermined test data into the camera simulator 104 before starting the vehicle test to generate the predetermined camera simulation data, and input the predetermined test data into the radar simulator 106 to generate the predetermined radar simulation data.

[0042] The predetermined camera simulation data can be processed by the camera sensor 114 and fed into the ADAS system 112, and the predetermined radar simulation data can be processed by the radar sensor 116 and fed into the ADAS system 112. The ADAS system 112 can then fuse the two processed data and make a judgment. If the fusion is successful, that is, the predetermined camera simulation data is consistent with the predetermined radar simulation data, it means that there is no processing delay. On the contrary, the fusion is unsuccessful, that is, the predetermined camera simulation data is inconsistent with the predetermined radar simulation data, which means that there is a processing delay. The fusion algorithm can integrate data from multiple sensors to provide and verify consistent and complete environmental information, such as through accurate data calibration.

[0043] Additionally or alternatively, in some embodiments, the inconsistency between the camera simulation data and the radar simulation data may include, but is not limited to: inconsistency between camera space data in the camera simulation data and radar space data in the radar simulation data, inconsistency between camera time data in the camera simulation data and radar time data in the radar simulation data, and the like.

[0044] In some embodiments, the processing delay between the camera simulator and the radar simulator can also be determined based on these fusion results. Specifically, in one embodiment, the ADAS-based algorithm can output object recognition and fusion results, and the delay time of a certain signal can be determined based on the recognition effect. For example, the ADAS can receive signal inputs from the camera sensor 114 and the radar sensor 116, but the fusion fails according to the object fusion algorithm because the data generated by the camera simulator 104 is output 10 milliseconds later than the data generated by the radar simulator 106.

[0045] Therefore, when the radar simulator 106 performs simulation, by adding a 10 millisecond delay output to the radar simulation data, the camera simulation data and the radar simulation data can be simultaneously input into the ADAS system 112 for calculation to meet the object fusion requirements.

[0046] In some embodiments, after determining the processing delay between the camera simulator 104 and the radar simulator 106, the camera simulation data may be synchronized by adjusting the camera simulation data based on the processing delay to generate radar simulation data. For example, after determining that the camera simulator 104 outputs camera simulation data 15 milliseconds slower than the radar simulator 106, the example test system 100 may delay the input of test data that should have been directly input to the radar simulator 106 by 15 milliseconds.

[0047] Specifically, in some embodiments, in an example of synchronization based on the x coordinate on the XY plane, the simulation times may be t0, t1, and t2 in order, and x1=x t0 ,x2=x t0 , where x1 represents the coordinate system data for the radar simulator 106, and x2 represents the coordinate system data for the camera simulator 104. At time t1, x1=x t1 ,x2=x t0 Due to the delay caused by the calculation time of the processing data corresponding to x2, when x1 has already been t1 The data is processed, but for x2, x t0 Data processing is completed. Therefore, a certain processing delay Δ can be made on the data of x1. The process of generating radar simulation data based on the processing delay and the time data in the camera simulation data can be expressed as:

[0048] x1= x t(1-Δ) (1)

[0049] Where Δ can be 1, so that the final output can be adjusted to make the inputs of x1 and x2 consistent at the same time.

[0050] In some embodiments, after determining the processing delay between the camera simulator 104 and the radar simulator 106, the radar simulation data may also be generated based on the camera space data in the camera simulation data. The camera space data may include one or more of the spatial coordinate system data, heading angle, pitch angle, and roll angle, and the present disclosure does not impose any limitation on this.

[0051] Since the coordinate data is constantly changing according to the operation cycle in the simulator 102, delaying the coordinate change (difference) of a certain simulator may include making it equal to the original data of the previous cycle, so the signal may be characterized as a phenomenon of causing a time delay of one cycle. However, the delay method may be achieved by adjusting the spatial difference of the coordinates in different simulators.

[0052] For example, in one embodiment, the operating cycle of the raw data such as the vehicle dynamics model 110 is 1 ms, which is the fastest in the entire test system 100 and the minimum time scale for adjustable delay. In addition, the signals of the camera simulator 104 and the radar simulator 106 are delayed by 10 ms in time. Therefore, the camera space data of the camera simulator 104 can be used as the original coordinate value, and the coordinates obtained by the radar simulator 106 are set to the coordinate values ​​10 cycles ago. In this way, when the camera simulator 104 and the radar simulator 106 complete the calculation and output the signal, the extra 10 ms of the camera simulator 104 calculation can be added to achieve that the results of the camera simulator 104 and the radar simulator 106 input to the ADAS system 112 at the same time come from the original coordinate values ​​at the same time, thereby achieving a synchronization effect.

[0053] Specifically, in the example of coordinate transformation in the XY plane, it is assumed that there are n objects in the camera space data of the camera simulator 104 that need to be transformed or offset. Then the process of generating radar simulation data based on processing delay and spatial data in the camera simulation data can be expressed as:

[0054]

[0055]

[0056] where x o is the original coordinate for the camera simulator, x t is the converted coordinate for the radar simulator, θ is the roll angle of the coordinate on the XY plane, Δx is the translation in the x-axis direction, and Δy is the translation in the y-axis direction. o ,y o Projecting onto the rotated coordinate axis and superimposing the displacement, we get the transformed coordinate x t ,y t . This formula can also be applied to the XZ and YZ planes, etc.

[0057] The coordinate transformation of the heading angle or heading angle can be expressed as:

[0058] H t =H o +θ (4)

[0059] The orientation angle H t is the original orientation angle H o The superimposed rotation angle θ. This formula can also be applied to one or more of the pitch angle and the roll angle, and the present disclosure does not impose any limitation on this.

[0060] At box 208, based on the first simulation data and the second simulation data, the system in the vehicle is tested. In some embodiments, the system may be an ADAS-based system, which may also be an autonomous driving system (ADS), etc. The example test system 100 may simultaneously input the camera simulation data and the radar simulation data into the tested camera sensor 104 and the radar simulator 106 to test whether the system can make the desired action based on the input signal. For example, when encountering an obstacle, whether the system can simultaneously make a braking action based on the signals of different sensors. The above examples are only used to describe the present disclosure, and are not specific limitations of the present disclosure.

[0061] In some embodiments, due to possible errors in the adjustment process, the adjusted camera simulation data and radar simulation data may be input into the ADAS system 102 for re-fusion, and it is determined again whether fusion is possible. If fusion is still not possible, the camera simulation data may be automatically and dynamically adjusted or manually adjusted to generate radar simulation data.

[0062] Additionally or alternatively, in some embodiments, the example test system 100 may also receive any processing delay from user input, and adjust one or more of the camera simulation data and the radar simulation data based on the user input processing delay, thereby achieving customized adjustment of the test data.

[0063] In some embodiments, the example test system 100 can also adjust the delay between different simulators, not limited to the above camera simulator and radar simulator. For example, the example test system 100 can adjust the camera simulation data and the ultrasonic simulation data based on the difference in hardware configuration and processing delay between the camera simulator and the ultrasonic simulator, and feed these data to the ADAS system 112. In some embodiments, the example test system 100 can also adjust the millimeter wave radar simulation data and the laser radar simulation data based on the difference in hardware configuration and processing delay between the millimeter wave radar simulator and the laser radar simulator.

[0064] In this way, the present disclosure outputs the original coordinates or test data of the vehicle and the object to different sensor models through delay, offset, rotation and other operations based on different coordinate systems, thereby achieving synchronization of different sensor signals, definable deviations and the like.

[0065] Combined with the above Figure 2 A flow chart of a method 200 for vehicle testing according to an embodiment of the present disclosure is described. Figure 3A A schematic diagram of a framework 300 a for vehicle testing according to an embodiment of the present disclosure is depicted.

[0066] like Figure 3AAs shown, at box 302, the simulation or emulation process can begin. At box 304, test data including, but not limited to, posture and coordinate information data such as outputs of a vehicle dynamics model and an object model can be received. The posture and coordinate information data list can be expressed as [x, y, z, h, p, r], where x, y, z can represent the spatial coordinate values ​​in the x, y, and z axis directions of the test vehicle or test object, and h, p, r can represent the heading angle, pitch angle, and roll angle of the test vehicle or test object, respectively. Additionally or alternatively, in an embodiment of the present disclosure, the test data may also include any data characterizing any time parameter or space parameter.

[0067] At block 306, it may be determined whether the synchronization mode is enabled. If the synchronization mode is enabled, then at block 308, the synchronization mode may be enabled by using Figure 2 The method for vehicle testing described in the specification adds processing delays such as time delays or spatial delays to radar simulators or other types of simulators to synchronize signals between sensors. For example, the test system can delay the coordinate signal sent to each sensor as required by the object fusion algorithm to eliminate differences in processing and differences in the sending time between different simulated signals. And the test system can ensure that the signal received by the ADAS controller or system meets the synchronization requirements.

[0068] If it is confirmed that the synchronization mode is not turned on, then at box 310, the deviation control mode can be turned on. In the deviation control mode, according to the input instructions, deviations such as displacement or angle can be added to the coordinate and attitude signals of the vehicle and the object sent to different sensors, the difference of the sensor signals can be manually created, and the object fusion algorithm can be tested. At box 312, the transformed attitude and coordinate information data can be sent to multiple different types of sensors, for example, the transformed radar simulation data n[x, y, z, h, p, r]1 is sent to the radar sensor, the transformed ultrasonic simulation data n[x, y, z, h, p, r]2 is sent to the ultrasonic sensor, and so on.

[0069] At block 322, it is determined whether the above test process needs to be repeated cyclically, if so, return to block 304, if not, proceed to block 320. At block 320, in response to the completion of the test goal, the user can stop the entire test process.

[0070] For ease of understanding, blocks 312 to 318, 316, 314 to block 304 describe the flow of data. At block 318, different sensor models may receive corresponding transformed data and perform corresponding processing. For example, a radar sensor may receive transformed radar simulation data and perform corresponding processing such as denoising, feature extraction, etc. An ultrasonic sensor may receive transformed ultrasonic simulation data and perform corresponding processing, etc.

[0071] At box 316, the different sensors can input the processed data into the ADAS system to test whether the vehicle can perform the desired behavior based on the input signals, such as braking, changing lanes. At box 314, the ADAS system generated data (i.e., the desired behavior) can then be input into the vehicle model and the object model of the test system to generate new coordinate data. These new coordinate data can then be input into box 304 for the next round of testing and changes.

[0072] In the prior art, new coordinate data is generated at box 314 and directly input into box 318 for corresponding processing. Since it is not processed by boxes 304, 306, 308, 310, 312, etc. of the method of the present invention, the time and process of generating simulation signals by different simulators may be out of sync.

[0073] The following combination Figure 3B A schematic diagram of another framework 300b for vehicle testing according to an embodiment of the present disclosure is described. Figure 3B As shown, at block 303, the simulation process begins. At block 305, raw simulation data n[x, y, z, h, p, r] from the target object model and the vehicle model may be sent to the test system. At block 307, the test system may receive n[x, y, z, h, p, r] as the original coordinate system.

[0074] At block 309, it may be determined whether the synchronization mode is enabled. If the synchronization mode is enabled, then at block 311, the synchronization mode may be enabled by using Figure 2 In the method for vehicle testing described in , a processing delay such as a time delay or a spatial delay is added to a radar simulator or other type of simulator to synchronize the signals between sensors. If it is confirmed that the synchronization mode is not turned on, at box 313, the simulated data n[x, y, z, h, p, r] is coordinate transformed based on factors such as user input and sensor type.

[0075] The transformed simulation data may be input into different types of sensor models, such as the camera model 104, the radar model 106. For example, at box 315, the transformed radar simulation data n[x, y, z, h, p, r]1 may be sent to a radar sensor. At box 317, the transformed ultrasonic simulation data n[x, y, z, h, p, r]2 may be sent to an ultrasonic sensor. At box 319, the transformed velocity simulation data n[x, y, z, h, p, r]3 may be sent to an inertial measurement unit (IMU), and at box 321, the transformed simulation data n[x, y, z, h, p, r]n may be sent to another sensor, and so on.

[0076] Combined with the above Figure 3A and Figure 3B A schematic diagram of a framework 300 for vehicle testing according to an embodiment of the present disclosure is described below. Figure 4 A schematic diagram depicts another example framework 400 for vehicle testing according to an embodiment of the present disclosure.

[0077] like Figure 4 As shown, data 402 associated with vehicle test dynamics and data 404 associated with object test dynamics can be input to a simulator 406 to generate pose and coordinate data 408. Two sets of coordinate and pose information are generated through different coordinate transformations, such as camera coordinate and pose information 410 for a camera simulator or virtual camera model 416, and radar coordinate and pose information 412 for a radar simulator or virtual radar model 420. These data can ultimately be sent to different sensor models for simulation signal calculation.

[0078] The camera coordinate and pose information 410 can be input into the scene module 414 in the simulator 406 for scene modeling and rendering, which may take a long time. The camera simulator or virtual camera model 416 can then process the generated scene to generate camera simulation data. The camera simulation data can be input into the camera sensor 418 of the test vehicle.

[0079] The radar coordinates and attitude information 412 may be input into the radar simulator or virtual radar model 420 in the simulator 406. Since the simulation processing time of the camera simulator 416 is relatively long, the time of the radar simulation data sent to the radar simulator may be appropriately delayed to synchronize the signal reception of real sensors such as the camera sensor 418 and the radar sensor 422.

[0080] Finally, the data received by the camera sensor 418 and the radar sensor 422 can be synchronously input into the ADAS system 430 to verify whether the ADAS system 430 can make the expected behavior according to the input. Similarly, the test parameter adjustment range can be expanded to [x, y, z, h, p, r], the coordinate signals sent to different sensors can be fully customized, and simulation results can be generated as needed to verify the fusion algorithm.

[0081] Figure 5 A schematic diagram of a signal process 500 for synchronizing the reception of sensors based on processing delays is shown. Figure 5As shown, time series 502 represents the benchmark truth time of the real world, which may include several moments from t0 to t4. Time series 504 may represent the time series for a sensor, such as the time series for a radar sensor. Since the radar simulator generates simulation data at a fast speed, a time delay, such as 504-a, may be added to synchronize it with the time when other sensors receive simulation data.

[0082] Time series 506 may represent a time series for another sensor, such as a time series for a camera sensor. Since the camera simulator generates simulation data at a slow speed, a small time delay, such as time delay 506-a, may be added to synchronize it with the time when other sensors receive simulation data. Coordinate system data 512 and 514 represent coordinate data for different sensors, respectively.

[0083] Additionally or alternatively, in some embodiments, different time delays, such as time delay 510-a, may be added to time series of different types of sensors, such as time series 508 and 510, to synchronize the moments when different sensors receive simulated data. Ultimately, data synchronization between multiple sensors is achieved by adding time delays, distance transformation, angle transformation, etc., and sensor failures, position changes between sensors, etc. may be simulated through various transformations.

[0084] Figure 6 A schematic diagram of a device for vehicle testing according to an embodiment of the present disclosure is further shown. The device 600 can be applied to the electronic device 100, which can include multiple modules for performing the following steps: Figure 2 The corresponding steps in the process 200 discussed in Figure 6 As shown, the device 600 includes: a receiving unit 602, configured to receive test data including object test data and vehicle test data; a first generating unit 604, configured to input the test data into a camera simulator for a camera to generate first simulation data; a second generating unit 606, configured to input the test data into a radar simulator for a radar to generate second simulation data based on a processing delay between the camera simulator and the radar simulator for the radar; and a testing unit 608, configured to test a system in a vehicle based on the first simulation data and the second simulation data.

[0085] In some embodiments, the device 600 also includes a processing delay determination unit, which is configured to determine the processing delay between the camera simulator and the radar simulator for the radar, including: a fusion unit, which is configured to fuse predetermined camera simulation data generated by the camera simulator with predetermined radar simulation data generated by the radar simulator to generate a fusion result; and a determination unit, which is configured to determine the processing delay between the camera simulator and the radar simulator based on the fusion result, wherein the fusion result indicates that the predetermined camera simulation data is consistent with the predetermined radar simulation data or that the predetermined camera data is inconsistent with the predetermined radar simulation data.

[0086] In some embodiments, the second generation unit 606 includes: a determination unit, configured to determine first spatial data in the first simulation data, wherein the first spatial data includes one or more of spatial coordinate system data, a heading angle, a pitch angle, and a roll angle; an adjustment unit, configured to adjust the first spatial data in the first simulation data based on processing delay; and a generation unit, configured to generate second spatial data in the second simulation data based on the adjusted first spatial data and test data.

[0087] In some embodiments, the second generation unit 606 includes: a determination unit configured to determine first time data in the first simulation data; an adjustment unit configured to adjust the first time data of the first simulation data based on processing delay; and a generation unit configured to generate second time data in the second simulation data based on the adjusted first time data and test data.

[0088] In some embodiments, the device 600 further includes: a fusion unit configured to re-fuse the adjusted first simulation data with the second simulation data; and a dynamic adjustment unit configured to dynamically adjust the first simulation data in response to the inconsistency between the adjusted first simulation data and the second simulation data.

[0089] In some embodiments, the device 600 also includes: a determination unit configured to determine that the first spatial data of the first simulation data is inconsistent with the second spatial data of the second simulation data; and a determination unit configured to determine that the first time data of the first simulation data is inconsistent with the second time data of the second simulation data.

[0090] In some embodiments, the apparatus 600 further includes: a receiving unit configured to receive a processing delay from an input; and an adjusting unit configured to adjust the first analog data or the second analog data based on the processing delay of the input.

[0091] In some embodiments, the apparatus 600 further includes: a generating unit configured to input the test data into the ultrasonic simulator for ultrasonic waves to generate third simulation data according to a processing delay between the camera simulator and the ultrasonic simulator for ultrasonic waves; and a testing unit configured to test the system in the vehicle based on the first simulation data and the third simulation data. In some embodiments, the apparatus 600 further includes an ADAS-based system unit.

[0092] Figure 7 A schematic block diagram of an example device 700 that may be used to implement embodiments of the present disclosure is shown. Figure 1 The electronic device 100 in the embodiment can be implemented by using the device 700. As shown in the figure, the device 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to computer program instructions stored in a read-only memory (ROM) 702 or computer program instructions loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0093] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a memory 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0094] The various processes and processing described above, such as method 200, may be performed by processing unit 701. For example, in some embodiments, method 200 may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by CPU 701, one or more actions of method 200 described above may be performed.

[0095] The present disclosure may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present disclosure.

[0096] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (non-exhaustive list) of computer readable storage medium include: portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, such as a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination of the above. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (e.g., a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0097] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0098] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0099] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0100] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0101] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0102] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0103] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for vehicle testing, the method comprising: receiving test data including object test data and vehicle test data; Inputting the test data into a camera simulator for a camera to generate first simulation data; inputting the test data into the radar simulator for the radar to generate second simulation data according to a processing delay between the camera simulator and the radar simulator for the radar; as well as A system in the vehicle is tested based on the first simulation data and the second simulation data.

2. The method according to claim 1, further comprising: fusing predetermined camera simulation data generated by the camera simulator with predetermined radar simulation data generated by the radar simulator to generate a fusion result; as well as The processing delay between the camera simulator and the radar simulator is determined based on the fusion result, wherein the fusion result indicates that the predetermined camera simulation data is consistent with the predetermined radar simulation data or that the predetermined camera data is inconsistent with the predetermined radar simulation data.

3. The method according to claim 1 , wherein inputting the test data into the radar simulator for the radar to generate the second simulation data according to the processing delay between the camera simulator and the radar simulator for the radar comprises: Determine first spatial data in the first simulation data, wherein the first spatial data includes one or more of spatial coordinate system data, heading angle, pitch angle, and roll angle; adjusting first spatial data in the first simulation data based on the processing delay; as well as Second spatial data in the second simulation data is generated based on the adjusted first spatial data and the test data.

4. The method according to claim 1, wherein according to the processing delay between the camera simulator and the radar simulator for the radar, inputting the test data into the radar simulator for the radar to generate the second simulation data further comprises: determining first time data in the first simulation data; adjusting the first time data of the first simulation data based on the processing delay; as well as Second time data in the second simulation data is generated based on the adjusted first time data and the test data.

5. The method of claim 2, wherein adjusting the first analog data based on the processing delay further comprises: Re-integrating the adjusted first simulation data with the second simulation data; as well as In response to determining that the adjusted first analog data is inconsistent with the second analog data, the first analog data is dynamically adjusted.

6. The method according to claim 5, wherein determining that the first simulation data is inconsistent with the second simulation data comprises: Determining that first spatial data of the first simulation data is inconsistent with second spatial data of the second simulation data; and / or It is determined that the first time data of the first simulation data is inconsistent with the second time data of the second simulation data.

7. The method according to claim 1, further comprising: receiving said processing delay from an input; as well as The first analog data or the second analog data is adjusted based on the processing delay of the input.

8. The method according to claim 1, further comprising: inputting the test data to the ultrasonic simulator for ultrasonic waves to generate third simulation data according to a processing delay between the camera simulator and the ultrasonic simulator for ultrasonic waves; and A system in the vehicle is tested based on the first simulation data and the third simulation data.

9. The method of claim 1, wherein the system in the vehicle is an advanced driver assistance system (ADAS).

10. A device for vehicle testing, the device comprising: a receiving unit configured to receive test data including object test data and vehicle test data; A first generating unit, configured to input the test data into a camera simulator for a camera to generate first simulation data; a second generating unit configured to input the test data into the radar simulator for the radar to generate second simulation data according to a processing delay between the camera simulator and the radar simulator for the radar; as well as A test unit is configured to test a system in the vehicle based on the first simulation data and the second simulation data.

11. An electronic device, comprising: at least one processor; as well as A memory coupled to the at least one processor and having instructions stored thereon, the instructions, when executed by the at least one processor, causing the apparatus to perform the method according to any one of claims 1-9.

12. A vehicle comprising the electronic device according to claim 11.

13. A computer-readable storage medium having computer-executable instructions stored thereon, wherein the computer-executable instructions are executed by a processor to implement the method according to any one of claims 1 to 9.

Images