Automatic driving non-intrusive test system and method

Abstract

The invention provides an automatic driving non-intrusive test system and method. The system comprises an external VR module, a radio frequency signal simulator, a multi-mode sensing module, an edge calculation module, a dynamic scene generation module, a virtual-real fusion engine, external mechanical equipment and a closed-loop test verification module. The system injects a virtual signal into the sensor of the to-be-detected vehicle through optical projection and a radio frequency echo signal, so that intervention on an internal system of the vehicle is avoided, and the system has high compatibility and high safety. And the dynamic scene generation module matches an extreme test scene from an AI scene library in combination with vehicle real-time state information, so that the comprehensiveness and the high efficiency of the test are ensured. The virtual-real fusion engine accurately controls the synchronization of virtual and real scenes, and ensures the authenticity and accuracy of the test. According to the testing method, a reliable, flexible and comprehensive automatic driving testing tool can be provided under the condition that a vehicle system is not intervened, and safety and compliance verification of the automatic driving technology is promoted.

Description

Technical Field

[0001] The present invention belongs to the technical field of autonomous driving testing, and specifically relates to a non-intrusive testing system and method for autonomous driving. Background Art

[0002] With the continuous development of autonomous driving technology, intelligent connected vehicles are gradually entering the market and becoming an integral part of the modern transportation system. To ensure the safety and reliability of autonomous driving systems in various road and traffic environments, they require comprehensive and accurate testing and verification. However, a series of urgent challenges remain in the field of autonomous driving technology testing. These challenges not only affect the efficiency and accuracy of testing, but also hinder the rapid development and application of autonomous driving technology.

[0003] First, existing autonomous driving testing technologies generally suffer from insufficient scenario coverage. Traditional physical testing scenarios often only cover a small number of potentially dangerous scenarios, and the testing process is relatively simple, failing to fully simulate extreme operating conditions found in complex real-world environments, such as heavy rain or pedestrians crossing the road at night. Furthermore, the high cost of reproducing extreme operating scenarios significantly limits the comprehensiveness and depth of testing.

[0004] In the prior art, Chinese patent CN118376420A discloses a method and system for testing autonomous vehicles, which falls within the field of autonomous driving testing. In this method, a simulation platform transmits virtual scene information and virtual sensor information to the test vehicle based on a virtual test scenario. The autonomous driving controller then controls the test vehicle for autonomous driving based on this information. The simulation platform then transmits motion state information and related control signals during the autonomous driving process to the simulation platform, which then determines the test results. This method simulates the autonomous driving environment to a certain extent through a virtual scenario, addressing the difficulty of covering extreme operating conditions in traditional physical testing. However, it still has some significant shortcomings and limitations.

[0005] First, this method relies on the access to the sensors and control systems of the test vehicle and needs to receive virtual signals through existing sensors and controllers. This means that there may be compatibility issues with the integration and communication protocols between the test system and the vehicle, and it may be necessary to adapt to different types of vehicles. This not only increases the complexity of the test system but also may interfere with or modify the original vehicle system, thereby affecting the normal functions and performance of the vehicle. In addition, the communication interaction between the simulation platform and the test vehicle requires a certain degree of openness and modification of the existing vehicle system, which may pose certain technical and safety risks in practical applications. Second, the existing integration of virtual scenarios and real environments mainly relies on AR displays to present images, without clearly mentioning how to achieve high-precision spatio-temporal alignment between virtual scenarios and real environments. Since the construction of virtual scenarios during the test process may not be fully synchronized with the sensor data of the real environment, it may lead to spatio-temporal inconsistencies between virtual obstacles and obstacles in the sensor data, thus affecting the accuracy and reliability of the test results. The invasive risk is another major drawback of traditional test methods. Most traditional test methods require modifying the vehicle system through physical connections or software interfaces, which not only increases the complexity of vehicle testing but also may introduce potential safety hazards and affect the accuracy and reliability of test results. Therefore, how to conduct comprehensive testing without directly intervening in the vehicle control system has become an urgent problem in the industry. Summary of the Invention

[0006] The purpose of the present invention is to overcome the defects of the above-mentioned existing technologies and provide a non-intrusive test system and method for autonomous driving.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] On the one hand, the present invention provides a non-intrusive test system for autonomous driving, including:

[0009] An external VR module for injecting virtual visual signals into the in-vehicle camera of the vehicle to be tested through optical projection technology;

[0010] A radio frequency signal simulator for generating virtual echo signals for the lidar of the vehicle to be tested;

[0011] A multi-modal perception module including an inertial measurement unit (IMU) and a high-precision positioning module for real-time acquisition of multi-sensor data of the vehicle to be tested;

[0012] An edge computing module for obtaining multi-sensor data of the vehicle to be tested from the multi-modal perception module and calculating and obtaining the real-time state information of the vehicle to be tested based on multi-sensor data fusion technology;

[0013] A dynamic scenario generation module, which is used to match the extreme test scenario data suitable for the current state from the AI scenario library according to the real-time state information of the vehicle to be tested transmitted by the edge computing unit;

[0014] A virtual-real fusion engine, which is used to receive the extreme test scenario data generated by the dynamic scenario generation module and the real-time state information of the vehicle to be tested obtained from the edge computing unit, and control the output synchronization of the external VR module and the radio frequency signal simulator according to the real-time state information of the vehicle to be tested and the extreme test scenario;

[0015] External mechanical equipment, which is used to adapt the cameras and lidars of different vehicles to be tested, and fix the external VR module and the radio frequency signal simulator in front of the cameras and lidars of the vehicle to be tested;

[0016] A closed-loop test verification module, which is used to record the real-time state information of the vehicle to be tested generated by the edge computing module and the extreme test scenario generated by the dynamic scenario generation module, and obtain the vehicle obstacle avoidance trajectory and braking response delay according to the real-time state information and the extreme test scenario, and generate a quantitative test report based on the vehicle obstacle avoidance trajectory and braking response delay.

[0017] Furthermore, the external VR module includes a high-resolution VR display device, which is fixed in front of the in-vehicle camera of the vehicle to be tested through external mechanical equipment, and is used to project virtual visual signals to the in-vehicle camera. The virtual visual signals are virtual images or video signals generated by optical projection technology, including obstacles, extreme weather or other potential dangerous scenarios, where the potential dangerous scenarios include sudden pedestrians crossing the road and sudden braking of the vehicle in front.

[0018] Furthermore, the radio frequency signal simulator is fixed in front of the lidar of the vehicle to be tested through external mechanical equipment, and provides virtual echo signals to the lidar to simulate the reflection characteristics of different environments and obstacles. The virtual echo signals include virtual lidar echo data generated by radio frequency signal modulation technology, simulating information such as the distance, speed, and reflectivity of obstacles.

[0019] Furthermore, the multi-sensor data includes vehicle acceleration and angular velocity data collected by an inertial measurement unit (IMU) and the real-time position information of the vehicle collected by a high-precision positioning module. The real-time state information of the vehicle to be tested includes the speed, position, and attitude of the vehicle.

[0020] Furthermore, the extreme test scenario data includes virtual images or video signals under obstacles, extreme weather or other potential dangerous scenarios and their corresponding virtual echo signals.

[0021] On the other hand, the present invention provides an autonomous driving non-intrusive test method, including:

[0022] The multi - sensor data of the vehicle to be tested is obtained through a multi - modal perception module, including the vehicle acceleration and angular velocity data collected by an inertial measurement unit (IMU), and the real - time position information of the vehicle collected by a high - precision positioning module;

[0023] The edge computing module performs fusion processing on the collected multi - sensor data, calculates and obtains the real - time state information of the vehicle to be tested, including the vehicle speed, position, and attitude;

[0024] According to the real - time state information of the vehicle to be tested, extreme test scenario data suitable for the current vehicle state is matched from a preset AI scenario library;

[0025] The virtual - real fusion engine calculates the virtual echo signal delay according to the real - time state information of the vehicle to be tested and the extreme test scenario data, and synchronously controls the outputs of the external VR module and the radio frequency signal simulator based on the virtual echo signal delay;

[0026] The external VR module projects virtual visual signals to the in - vehicle camera of the vehicle to be tested through optical projection technology, and the radio frequency signal simulator provides virtual echo signals to the lidar of the vehicle to be tested to simulate extreme test scenarios;

[0027] The closed - loop test verification module records the real - time state information of the vehicle to be tested generated by the edge computing module and the extreme test scenario data generated by the dynamic scenario generation module, and generates a quantitative test report based on the vehicle obstacle avoidance trajectory and braking response delay during the test.

[0028] Further, the matching of extreme test scenario data suitable for the current vehicle state from a preset AI scenario library according to the real - time state information of the vehicle to be tested includes:

[0029] When the speed in the real - time state information of the vehicle to be tested is greater than the first preset value, extreme test scenario data related to high - speed driving scenarios is selected from the preset AI scenario library, including fast - moving obstacles, sudden braking of the vehicle ahead, and pedestrians crossing the road;

[0030] When the attitude change in the real - time state information of the vehicle to be tested is large, extreme test scenario data suitable for scenarios with turning is selected from the AI scenario library, including oncoming vehicles on the opposite road.

[0031] Further, the virtual - real fusion engine calculates the virtual echo signal delay according to the real - time state information of the vehicle to be tested and the extreme test scenario data, and synchronously controls the outputs of the external VR module and the radio frequency signal simulator based on the virtual echo signal delay, specifically including:

[0032] Construct a cross-modal delay benchmark template for the vehicle under different extreme test scenarios. The cross-modal delay benchmark template is the virtual echo signal delay at different speeds and postures of the vehicle under different extreme test scenarios. The virtual echo signal delay is the time difference between the image captured by the on-vehicle camera and the echo signal received by the lidar at the same moment.

[0033] Based on the real-time state information of the vehicle, i.e., speed, posture, and extreme test scenario data, obtain the corresponding virtual echo signal delay through the cross-modal delay benchmark template.

[0034] Transmit the virtual image or video signal in the extreme test scenario data to the external VR module. The external VR module projects the virtual image or video signal onto the on-vehicle camera of the vehicle to be tested based on optical projection technology.

[0035] After waiting for the obtained virtual echo signal delay, transmit the virtual echo signal in the extreme test scenario data to the RF signal simulator. The RF signal simulator sends the virtual echo signal to the lidar of the vehicle to be tested.

[0036] Further, the process of constructing the cross-modal delay benchmark template is as follows:

[0037] Synchronously collect multi-modal data of typical traffic participants through the real on-vehicle sensor array, including the on-vehicle camera and lidar. Record the perception data of each sensor at different speeds and postures of the vehicle, and obtain the timestamp t of the image captured by the on-vehicle camera vision and the timestamp t of the echo signal received by the lidar r ;

[0038] Based on the collected multi-modal data, annotate the delay information of each sensor in the hardware trigger, signal processing, data transmission, and data generation links, and calculate the time difference between the image captured by the on-vehicle camera and the echo signal received by the lidar at the same moment as the virtual echo signal delay:

[0039] Δt = t r -t vision

[0040] where Δt is the virtual echo signal delay;

[0041] Utilize the virtual echo signal delays at different speeds v and yaw angles θ collected to establish a cross-modal delay benchmark template, and fit the delay characteristics through a non-linear regression model. The model expression is:

[0042] Δt = a0 + a1v + a2θ + a3vθ + ε

[0043] Where a0, a1, a2, and a3 are the regression coefficients of the model, which are used to describe the impact of vehicle speed and posture on delay. ε represents the error not explained by the model, which covers factors such as sensor noise, environmental interference, and model simplification error. The fitting parameters are optimized by the least squares method to enable the model to accurately describe the delay characteristics of the multimodal sensor. The optimization calculation formula is:

[0044]

[0045] Where N is the number of data samples; Δt i is the actual observed virtual echo signal delay of the i-th sample; v i is the vehicle speed of the i-th sample; θ i is the vehicle yaw angle of the i-th sample.

[0046] Furthermore, the updating process of the cross-modal delay reference template includes:

[0047] Get the current real-time status information of the vehicle, including the vehicle speed v t and yaw angle θ t and the corresponding extreme test scenario data, and based on this information, query the cross-modal delay benchmark template to obtain the model regression coefficients a0, a1, a2, and a3 of the current virtual echo signal delay;

[0048] Based on the multi-sensor data obtained by the multimodal perception module at the current moment and the preset AI scenario library, the real-time vehicle status information and extreme test scenario data at the next moment are predicted;

[0049] Based on the predicted vehicle status information and extreme test scenario data, the cross-modal delay benchmark template is used to calculate the predicted virtual echo signal delay at the next moment:

[0050] Δt predicted =a0+a1v t+1 +a2λ t+1 +a3v t+1 θ t+1 +ε

[0051] Where Δt predicted is the virtual echo signal delay at the next moment predicted based on the current model; v t+1 is the predicted vehicle speed at the next moment; θ t+1 is the predicted vehicle yaw angle at the next moment;

[0052] Based on the current moment's delay control, the plug-in VR module and the RF signal simulator generate corresponding extreme test scenario data;

[0053] At the next moment, collect the actual real-time state information of the vehicle through the multi-modal perception module, and calculate and obtain the true virtual echo signal time delay Δt according to the actual vehicle state at the next moment. actual :

[0054] Δt actual = t r - t visi0n

[0055] Compare the predicted time delay Δt predicted with the actually measured time delay Δt actual Calculate the time delay error and record the error of the vehicle at each moment to generate a time delay error time series:

[0056] ∈ = Δt actual - Δt predicted

[0057] where ε represents the error unexplained by the model, i.e., the time delay error;

[0058] Use the recursive least squares (RLS) method to dynamically adjust the time delay model parameters to adapt to the time delay changes. The calculation formula for updating the model parameters is as follows:

[0059] Compare the predicted time delay at the next moment with the actual time delay at the next moment, calculate the time delay error, record the time delay error of the vehicle at each moment, and generate a time delay error time series;

[0060] Filter and correct the time delay error time series to reduce sudden interference and errors, and obtain a corrected time delay error time series, including:

[0061] Filter and correct the time delay error time series, and use a low-pass filtering method to smooth the time delay error:

[0062] Δt filtered = α∈ + (1 - α)Δt filtered-prev

[0063] where ∈ is the difference between the actually observed time delay and the model-predicted time delay at the current moment; Δt filtered-prev is the filtered time delay error at the previous moment; α is the filtering coefficient, and Δt filtered is the filtered time delay error at the current moment;

[0064] Update the cross-modal time delay reference template according to the corrected time delay error time series. The formula is:

[0065]

[0066] θ k = θ k-1 + K k∈

[0067]

[0068] where θ k = [a0, a1, a2, a3] T is the time-delay model parameter vector; H k = [1, v, θ, vθ] is the observation matrix; K k is the gain matrix; P k is the covariance matrix; λ is the forgetting factor, which controls the influence of historical data on model update.

[0069] Compared with the prior art, the present invention has the following advantages:

[0070] (1) The present invention adopts an external VR module and a radio frequency signal simulator, and injects virtual visual signals into the vehicle-mounted camera and provides virtual echo signals to the lidar through optical projection technology respectively, without accessing the in-vehicle system of the vehicle to be tested or modifying the vehicle's software and hardware. This technical means realizes a completely non-intrusive test, avoids invasive modification of the vehicle control system, ensures the safety, compatibility and flexibility during the test, can adapt to various vehicle models on the market, and reduces the test complexity and cost.

[0071] (2) Through the virtual-real fusion engine, the present invention uses the calculated time delay of the virtual echo signal to synchronously control the outputs of the external VR module and the radio frequency signal simulator, and realizes high-precision spatio-temporal alignment of the virtual scene and the real environment through space mapping technology, ensuring that the virtual obstacles are consistent with the real obstacles in the camera image and the lidar point cloud. The precise alignment of the virtual scene and the real environment at the spatio-temporal and sensor data levels greatly improves the authenticity of the test scene, ensures the reliability of the test results, especially the verification effect in complex scenes (such as low visibility environments), and increases the comprehensiveness of the test.

[0072] (3) The present invention integrates an AI scene library, dynamically generates extreme test cases (such as heavy rain + pedestrian crossing at night, etc.) according to the real-time state of the vehicle (such as speed, position), and supports real-time adjustment of scene parameters. By dynamically generating extreme test scenarios, the test coverage rate is greatly improved, the dependence on manually designed scenarios is reduced, the test efficiency and flexibility are enhanced, and complex real-world scenarios can be more comprehensively simulated.

[0073] (4) By constructing a cross-modal time-delay benchmark template, based on the real-time state information of the vehicle and the extreme test scenario data, the present invention accurately calculates the time delay of the virtual echo signal, and corrects the template according to the time delay error to optimize the test results. The synchronization accuracy between the virtual signal and the sensor data is improved, the interference caused by the time delay error is reduced, and the test accuracy and reliability are further enhanced.

[0074] (5) The present invention predicts the time delay of the virtual echo signal by constructing a cross-modal time delay reference template, combining real-time vehicle state information and extreme test scenario data, and dynamically adjusts the output timing of the virtual signal by accurately calculating the time delay error. This time delay prediction and compensation mechanism can significantly improve the synchronization accuracy between the virtual signal and the sensor data, ensure the spatio-temporal consistency between the virtual scenario and the sensor feedback data in the test, and thus enhance the authenticity and reliability of the test.

[0075] (6) When the simulation parameters change or a new extreme interference mode is added, the present invention triggers the model update mechanism, readjusts the time delay predictor, and continuously improves the time delay compensation ability of the model through the gradient optimization method. By continuously optimizing the model, the time delay prediction becomes more accurate, adapting to different test environments and vehicle states, enhancing the system's ability to cope with changing scenarios, and thus improving the applicability and reliability of the test system.

[0076] (7) The present invention establishes a cross-modal time delay reference template, fits the perception time delay characteristics of the camera and lidar at different vehicle speeds and attitudes based on a non-linear regression model, and dynamically updates the model parameters using the recursive least squares method (RLS) to ensure the accuracy of time delay compensation. At the same time, combining the multi-modal perception module and the AI scenario library, it predicts the vehicle motion state at the next moment and adjusts the time delay of the virtual echo signal in the simulation test to achieve precise spatio-temporal alignment of the sensor data, solving the problem of error accumulation in the existing test system in a dynamic environment and improving the reliability of autonomous driving tests.

[0077] (8) The present invention uses low-pass filtering to correct the time delay error time series, reduce noise interference, and combines an external VR module and a radio frequency signal simulator to ensure the synchronization between the virtual simulation environment and the real vehicle perception time through time delay control. This method effectively improves the accuracy and stability of non-intrusive testing, compensates for the time deviation between the existing virtual test platform and the real vehicle test data, and makes the simulation test results of the autonomous driving system more credible. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 is the functional schematic diagram of the non-intrusive test system of the present invention;

[0079] Figure 2 is the test flow chart of the non-intrusive test system of the present invention;

[0080] Figure 3 is the external installation schematic diagram of the non-intrusive test system of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0081] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0082] This embodiment provides a non-intrusive testing system for autonomous driving, which mainly performs non-intrusive testing on the vehicle to be tested through external devices and multi-modal perception modules. The system realizes the synchronization of virtual signals and vehicle sensor data through means such as optical projection and radio frequency signal simulation, and simulates complex test scenarios with high precision to ensure the comprehensiveness and accuracy of the test.

[0083] A non-intrusive testing system for autonomous driving, as Figure 1 shown, includes three parts: an external perception and signal injection module, an edge computing and dynamic scenario generation module, and a standardization verification module. Specifically, the system includes:

[0084] The external perception and signal injection module includes:

[0085] An external VR module for injecting virtual visual signals into the on-vehicle camera of the vehicle to be tested through optical projection technology;

[0086] A radio frequency signal simulator for generating virtual echo signals for the lidar of the vehicle to be tested;

[0087] The multi-modal perception module includes an inertial measurement unit (IMU) and a high-precision positioning module for real-time acquisition of multi-sensor data of the vehicle to be tested;

[0088] The edge computing and dynamic scenario generation module includes:

[0089] An edge computing module for obtaining multi-sensor data of the vehicle to be tested from the multi-modal perception module and calculating and obtaining the real-time state information of the vehicle to be tested based on multi-sensor data fusion technology;

[0090] A dynamic scenario generation module for matching extreme test scenario data suitable for the current state from the AI scenario library according to the real-time state information of the vehicle to be tested transmitted by the edge computing unit;

[0091] A virtual-real fusion engine for receiving the extreme test scenario data generated by the dynamic scenario generation module and the real-time state information of the vehicle to be tested obtained from the edge computing unit, and controlling the output synchronization of the external VR module and the radio frequency signal simulator according to the real-time state information of the vehicle to be tested and the extreme test scenario;

[0092] The standardization verification module includes:

[0093] External mechanical equipment is used to adapt cameras and lidars for different vehicles to be tested, and fixedly mount an external VR module and a radio frequency signal simulator in front of the cameras and lidars of the vehicle to be tested.

[0094] A closed-loop test verification module is used to record the real-time status information of the vehicle to be tested generated by the edge computing module and the extreme test scenarios generated by the dynamic scenario generation module, and based on the real-time status information and extreme test scenarios, obtain the vehicle obstacle avoidance trajectory and braking response delay, and generate a quantitative test report based on the vehicle obstacle avoidance trajectory and braking response delay.

[0095] The external VR module injects virtual visual signals into the in-vehicle camera of the vehicle to be tested through optical projection technology. The virtual visual signals are fixed in front of the camera of the vehicle to be tested through external mechanical equipment, without relying on the vehicle internal data interface or system modification, thus avoiding direct intervention in the vehicle system. Through this technical feature, the present invention avoids the invasive risk in traditional test methods, ensures the safety of the test process and the authenticity of test data. At the same time, the external VR module is used to adapt to the sensor layouts of different vehicle models, significantly improving the system compatibility and deployment efficiency, and reducing the test cost.

[0096] The radio frequency signal simulator injects virtual echo signals into the lidar of the vehicle to be tested, which is used to simulate the echo signals received by the lidar in extreme weather or complex traffic scenarios. The radio frequency signal simulator is fixed in front of the lidar of the vehicle to be tested through external mechanical equipment, and provides virtual echo signals to the lidar, and these signals simulate the reflection characteristics of different environments and obstacles. Through this technical feature, the system can simulate complex environments and obstacle reflection situations without changing the vehicle hardware, thus providing more comprehensive and diverse test scenarios, and improving the complexity and depth of the test.

[0097] The multi-modal perception module consists of an inertial measurement unit (IMU) and a high-precision positioning module, and collects multi-sensor data of the vehicle to be tested in real time, including the vehicle's acceleration, angular velocity and real-time position information. The edge computing module is responsible for fusing these data and calculating the status information of the vehicle to be tested in real time, such as speed, position and attitude. This module provides support for the dynamic scenario generation module by perceiving the vehicle status information in real time. Through this technical feature, the system can efficiently obtain the dynamic status of the vehicle to be tested in real time, and provide an accurate basis for subsequent test data processing, thus ensuring the timeliness and accuracy of various data during the test process.

[0098] The dynamic scenario generation module matches the real-time status information of the vehicle under test transmitted by the edge computing module to the extreme test scenario data suitable for the current status from the AI scenario library. Through the analysis of the vehicle's real-time status, this module dynamically generates virtual test data including obstacles, extreme weather, or other potential dangerous scenarios. With this technical feature, the present invention can dynamically generate more complex and extreme test scenarios according to the real-time status of the vehicle, covering more potential dangerous situations, improving the coverage rate of test scenarios and the comprehensiveness of testing, and reducing the dependence on manually designed scenarios.

[0099] The virtual-real fusion engine controls the output synchronization of the external VR module and the radio frequency signal simulator according to the real-time status information of the vehicle under test and the generated extreme test scenario data. Through this precise time delay control mechanism, this technical feature ensures that the virtual visual signal and the lidar echo signal can be synchronized with the signals received by the vehicle sensors, thereby achieving high-precision alignment between the virtual environment and the real world. With this technical feature, the system can ensure the high-precision simulation of test scenarios, improving the reliability and authenticity of test results, especially the scenario verification ability in low visibility and complex environments.

[0100] The external mechanical equipment is used to adapt the cameras and lidars of different vehicles under test, and fix the external VR module and the radio frequency signal simulator in front of the cameras and lidars of the vehicle under test to ensure the stability and consistency of signal injection. This technical feature enables the system to adapt to different types of vehicles and sensor layouts, improving the universality and adaptability of the system. With this technical feature, the system can be widely applied to various autonomous driving vehicles on the market, further enhancing the compatibility of the test tool chain.

[0101] The closed-loop test verification module records the real-time status information of the vehicle under test generated by the edge computing module and the extreme test scenario data generated by the dynamic scenario generation module, and generates a quantitative test report based on the vehicle's obstacle avoidance trajectory and braking response delay during the test. This test report is automatically generated and meets the quantitative compliance requirements of industry standards such as ISO 21448. With this technical feature, the present invention can achieve standardized verification in autonomous driving tests, automatically generate compliance reports that meet the regulations, improving the credibility, traceability, and legal effect of test results, and providing strong support for the access certification of autonomous driving technology.

[0102] Through the autonomous driving non-intrusive testing system provided by the present invention, the testing process avoids the invasive modification of vehicle hardware existing in traditional testing, reducing safety risks and interference to the vehicle system. The system can comprehensively improve the coverage and complexity of the testing scenario through high-precision virtual scenario simulation, dynamic scenario generation and real-time data fusion. Through multi-sensor synchronization and high-precision alignment of virtual and real environments, the test results are more reliable, providing a more rigorous and efficient verification tool for the safety and feasibility of autonomous driving technology.

[0103] This embodiment also provides an autonomous driving non-intrusive testing method, which comprehensively and accurately evaluates the autonomous driving performance by combining the interaction between the virtual scenario and the actual vehicle. The method includes multiple key steps to ensure that complex scenarios can be efficiently simulated and the response behavior of the vehicle can be accurately recorded during the testing process. As Figure 2 shown, it includes:

[0104] Obtain multi-sensor data of the vehicle to be tested through the multi-modal perception module, including vehicle acceleration and angular velocity data collected by the inertial measurement unit IMU, and real-time position information of the vehicle collected by the high-precision positioning module;

[0105] The edge computing module performs fusion processing on the collected multi-sensor data, calculates and obtains the real-time state information of the vehicle to be tested, including the speed, position, and attitude of the vehicle;

[0106] According to the real-time state information of the vehicle to be tested, match the extreme test scenario data suitable for the current vehicle state from the preset AI scenario library;

[0107] The virtual-real fusion engine calculates the virtual echo signal delay according to the real-time state information of the vehicle to be tested and the extreme test scenario data, and synchronously controls the output of the external VR module and the radio frequency signal simulator based on the virtual echo signal delay;

[0108] The external VR module projects virtual visual signals to the in-vehicle camera of the vehicle to be tested through optical projection technology, and the radio frequency signal simulator provides virtual echo signals to the lidar of the vehicle to be tested to simulate extreme test scenarios;

[0109] The closed-loop test verification module records the real-time state information of the vehicle to be tested generated by the edge computing module and the extreme test scenario data generated by the dynamic scenario generation module, and generates a quantitative test report based on the vehicle obstacle avoidance trajectory and braking response delay during the testing process.

[0110] According to the real-time state information of the vehicle to be tested, match the extreme test scenario data suitable for the current vehicle state from the preset AI scenario library, including:

[0111] When the speed in the real-time status information of the vehicle to be tested is greater than the first preset value, select extreme test scenario data related to the high-speed driving scenario from the preset AI scenario library, including fast-moving obstacles, sudden braking of the vehicle ahead, and pedestrians crossing the road.

[0112] When the attitude change in the real-time status information of the vehicle to be tested is large, select extreme test scenario data suitable for extreme test scenarios with turning from the AI scenario library, including oncoming vehicles on the opposite road.

[0113] Further, the virtual-real fusion engine calculates the virtual echo signal delay based on the real-time status information of the vehicle to be tested and the extreme test scenario data, and synchronously controls the output of the external VR module and the radio frequency signal simulator based on the virtual echo signal delay. Specifically, it includes:

[0114] Construct a cross-modal delay benchmark template for the vehicle under different extreme test scenarios. The cross-modal delay benchmark template is the virtual echo signal delay of the vehicle under different speeds and attitudes in different extreme test scenarios. The virtual echo signal delay is the time difference between the image captured by the on-vehicle camera and the echo signal received by the lidar at the same moment.

[0115] Based on the speed, attitude of the real-time status information of the vehicle and the extreme test scenario data, obtain the corresponding virtual echo signal delay through the cross-modal delay benchmark template.

[0116] Transmit the virtual image or video signal in the extreme test scenario data to the external VR module, and the external VR module projects the virtual image or video signal onto the on-vehicle camera of the vehicle to be tested based on the optical projection technology.

[0117] After waiting for the obtained virtual echo signal delay, transmit the virtual echo signal in the extreme test scenario data to the radio frequency signal simulator, and the radio frequency signal simulator sends the virtual echo signal to the lidar of the vehicle to be tested.

[0118] The process of constructing the cross-modal delay benchmark template is as follows:

[0119] Synchronously collect multi-modal data of typical traffic participants through a real vehicle sensor array including an on-vehicle camera and a lidar, record the perception data of each sensor at different speeds and attitudes of the vehicle, and obtain the timestamp of the image captured by the on-vehicle camera and the timestamp of the echo signal received by the lidar.

[0120] Based on the collected multi-modal data, annotate the delay information of each sensor in the hardware trigger, signal processing, data transmission, and data generation links, and calculate the time difference between the image captured by the on-vehicle camera and the echo signal received by the lidar at the same moment as the virtual echo signal delay.

[0121] Using the time delays of virtual echo signals at different speeds and attitudes collected, a cross-modal time delay reference template is formed.

[0122] Furthermore, the process of constructing the cross-modal time delay reference template is as follows:

[0123] Through a real vehicle-mounted sensor array including vehicle-mounted cameras and lidar, multi-modal data of typical traffic participants are synchronously collected, the perception data of each sensor at different speeds and attitudes of the vehicle are recorded, and the timestamp of the image collected by the vehicle-mounted camera and the timestamp of the echo signal received by the lidar are obtained;

[0124] Based on the collected multi-modal data, the time delay information of each sensor in the hardware trigger, signal processing, data transmission, and data generation links is marked, and the time difference between the image collected by the vehicle-mounted camera and the echo signal received by the lidar at the same moment is calculated as the time delay of the virtual echo signal;

[0125] Using the time delays of virtual echo signals at different speeds and attitudes collected, a cross-modal time delay reference template is formed. The cross-modal time delay reference template is constructed by fitting a non-linear regression model with experimental data, and the formula is:

[0126] Δt = a0 + a1v + a2θ + a3vθ + ε

[0127] Where, Δt = t r -t vision is the time difference between the signal transmission time t r of the lidar or millimeter-wave radar and the image transmission time t vision of the camera; v is the vehicle speed; θ is the vehicle yaw angle; ε represents the error not explained by the model, that is, the error between the actual observed time delay value and the model predicted value, including the following potential factors: sensor noise: measurement errors of IMU, camera, and lidar; environmental interference: electromagnetic interference, influence of temperature and humidity changes on signal transmission; model simplification error: complex time delay characteristics not fully covered by the non-linear regression model, such as random fluctuations in sensor hardware delays; a0, a1, a2, a3 are fitting coefficients, which can be solved by the least squares method:

[0128]

[0129] Furthermore, the update process of the cross-modal time delay reference template includes:

[0130] Obtain the real-time vehicle state information and corresponding extreme test scenario data at the current moment. Through the real-time vehicle state information (vehicle speed v t ; vehicle yaw angle θ t)Query and obtain the virtual echo signal delay and related parameters a0, a1, a2, a3 corresponding to the cross-modal delay benchmark template from the extreme test scenario data;

[0131] Based on the multi-sensor data obtained by the multi-modal perception module at the current moment and the preset AI scenario library, predict the real-time vehicle state information at the next moment (vehicle speed v t+1 ; vehicle yaw angle θ t+1 ) and the extreme test scenario data;

[0132] According to the predicted real-time vehicle state information at the next moment and the extreme test scenario data, use the cross-modal delay benchmark template to predict the virtual echo signal delay at the next moment:

[0133] Δt predicted = a0 + a1v t+1 + a2θ t+1 + a3v t+1 θ t+1 + ε

[0134] Based on the delay control plug-in VR module and the radio frequency signal simulator at the current moment, generate the corresponding extreme test scenario data;

[0135] At the next moment, collect the actual real-time state information of the vehicle through the multi-modal perception module. According to the actual real-time state information of the vehicle at the next moment, calculate and obtain the actual virtual echo signal delay Δt actual = t r - t vision ;

[0136] Compare the predicted delay at the next moment with the actual delay at the next moment, calculate the delay error, record the delay error of the vehicle at each moment, and generate a delay error time series:

[0137] ∈ = Δt actual - Δt predicted

[0138] Dynamically adjust the delay model coefficients to adapt to the delay change through a recursive algorithm (including but not limited to various filtering methods or Bayesian methods). Taking the recursive least squares (RLS) method as an example, where the parameter vector θ of the delay model k = [a0, a1, a2, a3] T ; the observation matrix is H k = [1, v, θ, vθ]; forgetting factor λ; the initial covariance matrix is P0 = 10 3 I (I is the identity matrix), and the update formula of the delay model is:

[0139]

[0140] θ k = θ k-1 + K k ∈

[0141]

[0142] Through continuous iteration, the model coefficients (a0, a1, a2, a3) gradually approximate the true time-delay characteristics.

[0143] Filter and correct the time-delay error time series to reduce burst interference and errors, and obtain the corrected time-delay error time series. To suppress noise interference, perform low-pass filtering on the error sequence, which can be specifically expressed as:

[0144] Δt filtered = αΔt raw +(1 - α)Δt filtered-prev

[0145] Where, Δt raw represents the difference between the actual observed time-delay value at the current moment and the model prediction value; Δt filtered-prev represents the time-delay error after low-pass filtering at the previous moment, which is the result of the previous iteration in the recursive filtering process; α is the filtering coefficient, corresponding to the corresponding cut-off frequency and the applicable sampling period. If the filtered error exceeds the threshold, the system alarm is triggered. It is necessary to verify the model accuracy regularly to ensure that the time-delay prediction error is less than the threshold.

[0146] A non-intrusive testing method for autonomous driving in this embodiment specifically includes:

[0147] Initialize the attitude information and positioning of the virtual vehicle in the virtual scene to ensure that the state of the virtual vehicle is the same as that of the vehicle to be tested at the start of the test. This initialization process provides a benchmark for the subsequent scenario simulation and signal injection, ensuring the docking accuracy between the virtual and actual environments at the initial stage of the test.

[0148] Immediately afterwards, the dynamic scenario trigger module selects extreme test scenario data suitable for the current situation from the AI scenario library according to the real-time state information of the vehicle to be tested. These data include obstacles, extreme weather, or other potential dangerous scenarios, aiming to simulate various complex road and environmental conditions. At this time, signals are injected into the vehicle under test. Specifically, virtual visual signals are injected into the on-vehicle camera through an external VR module, and virtual echo signals are injected into the lidar through a radio frequency signal simulator. The signal injection process fully simulates various obstacles and extreme weather conditions in the real environment, and examines the adaptability of the autonomous driving system to complex situations.

[0149] After signal injection, the system will monitor the response behavior of the actual vehicle under test in real time, including obstacle avoidance trajectories, braking responses, etc. The vehicle under test makes autonomous driving decisions based on dynamic obstacles and weather changes in the virtual environment. The system updates its state information, including speed, position, and attitude, by obtaining the multi-sensor data of the vehicle in real time (such as cameras, radars, lidars, IMUs, etc.). Through multi-sensor synchronization, the consistency and accuracy of all perception data in space and time are ensured, providing high-precision data support for subsequent test analysis.

[0150] With the update of the virtual vehicle attitude information and positioning in the virtual scene, the autonomous driving performance response test of the actual vehicle is triggered. The system determines whether the vehicle has completed the test based on the test content (such as obstacle avoidance, emergency braking, lane keeping, etc.), and determines whether the autonomous driving system meets the safety margin requirements. If the test content is not completed or the safety margin does not meet the standard, the system will continue with the corresponding tests to ensure the reliability and safety of the autonomous driving system under extreme conditions.

[0151] After all autonomous driving performance tests are completed, the system will automatically generate a result analysis and compliance report. The report includes a quantitative evaluation of various data during the test, such as the obstacle avoidance trajectory of the vehicle, braking response delay, perception accuracy, etc., and meets the compliance requirements of industry regulations such as ISO 21448. This report provides a standardized basis for the certification of autonomous driving technology.

[0152] Finally, the test process ends, and all test data, reports, and analysis results will be stored and archived for subsequent technical review and safety certification.

[0153] Through the test method of this embodiment, it is possible to efficiently and comprehensively test the autonomous driving system without changing the vehicle hardware. The method utilizes a test mechanism that combines virtual and real scenarios, which not only ensures the safety and reliability during the test process but also can simulate extreme environments and complex scenarios, providing strong support for the verification of the safety and reliability of autonomous driving technology.

[0154] The installation schematic diagram of each module in this embodiment is as Figure 3 shown. Among them, the external VR module is installed in front of the in-vehicle camera of the vehicle under test, the radio frequency signal simulator is fixed in front of the vehicle lidar, the multi-modal perception module is installed on the top of the vehicle, and the edge computing and dynamic scene generation module is installed at the rear of the vehicle.

[0155] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs.

[0156] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. An autonomous driving non-intrusive testing system, characterized in that, Including: An external VR module for injecting virtual visual signals into the in-vehicle camera of the vehicle under test through optical projection technology; A radio frequency signal simulator for generating virtual echo signals for the lidar of the vehicle under test; A multi-modal perception module including an inertial measurement unit (IMU) and a high-precision positioning module for real-time acquisition of multi-sensor data of the vehicle under test; An edge computing module for obtaining multi-sensor data of the vehicle under test from the multi-modal perception module and calculating and obtaining real-time state information of the vehicle under test based on multi-sensor data fusion technology; A dynamic scenario generation module for matching extreme test scenario data suitable for the current state from an AI scenario library according to the real-time state information of the vehicle under test transmitted by the edge computing unit; A virtual-real fusion engine for receiving the extreme test scenario data generated by the dynamic scenario generation module and the real-time state information of the vehicle under test obtained from the edge computing unit, and controlling the synchronization of the outputs of the external VR module and the radio frequency signal simulator according to the real-time state information of the vehicle under test and the extreme test scenario; An external mechanical device for adapting to the cameras and lidars of different vehicles under test and fixing the external VR module and the radio frequency signal simulator in front of the cameras and lidars of the vehicle under test; A closed-loop test verification module for recording the real-time state information of the vehicle under test generated by the edge computing module and the extreme test scenarios generated by the dynamic scenario generation module, and obtaining the vehicle obstacle avoidance trajectory and braking response delay according to the real-time state information and the extreme test scenarios, and generating a quantitative test report based on the vehicle obstacle avoidance trajectory and braking response delay.

2. The non-intrusive test system for autonomous driving according to claim 1, characterized in that, The external VR module includes a high-resolution VR display device, which is fixed in front of the in-vehicle camera of the vehicle under test through an external mechanical device and is used for projecting virtual visual signals onto the in-vehicle camera. The virtual visual signals are virtual images or video signals generated through optical projection technology, including obstacles, extreme weather, or other potential dangerous scenarios, where the potential dangerous scenarios include sudden pedestrians crossing the road and sudden braking of the vehicle in front.

3. The non-intrusive test system for autonomous driving according to claim 1, characterized in that The radio frequency signal simulator is fixed in front of the lidar of the vehicle under test through an external mechanical device and provides virtual echo signals to the lidar, simulating the reflection characteristics of different environments and obstacles. The virtual echo signals include virtual lidar echo data generated through radio frequency signal modulation technology, simulating information such as the distance, speed, and reflectivity of obstacles.

4. The non-intrusive test system for autonomous driving according to claim 1, wherein, The multi-sensor data includes vehicle acceleration and angular velocity data collected by the inertial measurement unit (IMU) and the real-time position information of the vehicle collected by the high-precision positioning module. The real-time state information of the vehicle under test includes the speed, position, and attitude of the vehicle.

5. The non-intrusive test system for autonomous driving according to claim 1, characterized in that The extreme test scenario data includes virtual images or video signals and their corresponding virtual echo signals under obstacles, extreme weather, or other potential dangerous scenarios.

6. A non-intrusive testing method for autonomous driving based on the system according to any one of claims 1-5, characterized in that Including: Obtaining multi-sensor data of the vehicle under test through the multi-modal perception module, including vehicle acceleration and angular velocity data collected by the inertial measurement unit (IMU) and the real-time position information of the vehicle collected by the high-precision positioning module; The edge computing module performs fusion processing on the multi-sensor data collected, calculates and obtains the real-time state information of the vehicle to be tested, including the speed, position, and attitude of the vehicle; According to the real-time state information of the vehicle to be tested, match the extreme test scenario data suitable for the current vehicle state from the preset AI scenario library; The virtual-real fusion engine calculates the virtual echo signal delay according to the real-time state information of the vehicle to be tested and the extreme test scenario data, and synchronously controls the outputs of the external VR module and the radio frequency signal simulator based on the virtual echo signal delay; The external VR module projects virtual visual signals to the on-vehicle camera of the vehicle to be tested through optical projection technology, and the radio frequency signal simulator provides virtual echo signals to the lidar of the vehicle to be tested to simulate extreme test scenarios; The closed-loop test verification module records the real-time state information of the vehicle to be tested generated by the edge computing module and the extreme test scenario data generated by the dynamic scenario generation module, and generates a quantitative test report based on the vehicle obstacle avoidance trajectory and braking response delay during the test.

7. A non-intrusive testing method for autonomous driving according to claim 6, characterized in that The matching of the extreme test scenario data suitable for the current vehicle state from the preset AI scenario library according to the real-time state information of the vehicle to be tested includes: When the speed in the real-time state information of the vehicle to be tested is greater than the first preset value, select the extreme test scenario data related to the high-speed driving scenario from the preset AI scenario library, including fast-moving obstacles, sudden braking of the vehicle in front, and pedestrians crossing the road; When the attitude change in the real-time state information of the vehicle to be tested is relatively large, select the extreme test scenario data suitable for turning from the AI scenario library, including oncoming vehicles on the opposite road.

8. The non-intrusive test method for autonomous driving according to claim 6, characterized in that, The virtual-real fusion engine calculates the virtual echo signal delay according to the real-time state information of the vehicle to be tested and the extreme test scenario data, and synchronously controls the outputs of the external VR module and the radio frequency signal simulator based on the virtual echo signal delay, specifically including: Construct a cross-modal delay benchmark template for the vehicle under different extreme test scenarios, where the cross-modal delay benchmark template is the virtual echo signal delay of the vehicle under different speeds and attitudes in different extreme test scenarios, and the virtual echo signal delay is the time difference between the image collected by the on-vehicle camera and the echo signal received by the lidar at the same moment; Based on the real-time state information of the vehicle, speed, attitude, and extreme test scenario data, obtain the corresponding virtual echo signal delay through the cross-modal delay benchmark template; Transmit the virtual image or video signal in the extreme test scenario data to the external VR module, and the external VR module projects the virtual image or video signal onto the on-vehicle camera of the vehicle to be tested based on optical projection technology; After waiting for the obtained virtual echo signal delay, transmit the virtual echo signal in the extreme test scenario data to the radio frequency signal simulator, and the radio frequency signal simulator sends the virtual echo signal to the lidar of the vehicle to be tested.

9. The autonomous driving non-intrusive test method according to claim 8, wherein The process of constructing the cross-modal delay benchmark template is: Synchronously collect multi-modal data of typical traffic participants through a real vehicle sensor array including vehicle-mounted cameras and lidar, record the perception data of each sensor at different vehicle speeds and attitudes, and obtain the timestamp t of the images collected by the vehicle-mounted cameras vision and the timestamp t of the echo signals received by the lidar r ; Based on the collected multi-modal data, mark the delay information of each sensor in the hardware trigger, signal processing, data transmission, and data generation links, and calculate the time difference between the image collected by the on-vehicle camera and the echo signal received by the lidar at the same moment as the virtual echo signal delay: Δt = t r -t vision where Δt is the time delay of the virtual echo signal; Using the time delays of the virtual echo signals collected at different speeds v and yaw angles θ, a cross-modal time delay reference template is established, and the time delay characteristics are fitted through a non-linear regression model. The model expression is: Δt = a0 + a1v + a2θ + a3vθ + ε where a0, a1, a2, and a3 are the regression coefficients of the model, used to describe the influence of vehicle speed and attitude on the time delay; ε represents the error unexplained by the model, covering factors such as sensor noise, environmental interference, and model simplification error. The fitting parameters are optimized by the least squares method to enable the model to accurately describe the time delay characteristics of multi-modal sensors. The optimization calculation formula is: where N is the number of data samples; Δt i is the time delay of the actual observed virtual echo signal of the i-th sample; v i is the vehicle speed of the i-th sample; θ i is the vehicle yaw angle of the i-th sample.

10. The non-intrusive test method for autonomous driving according to claim 8, characterized in that, The update process of the cross-modal time delay reference template includes: Obtain the real-time vehicle status information at the current moment, including the vehicle speed v t and the yaw angle θ t and the corresponding extreme test scenario data, and query the cross-modal delay benchmark template based on this information to obtain the model regression coefficients a0, a1, a2, and a3 of the current virtual echo signal delay; Based on the multi-sensor data obtained by the multi-modal perception module at the current moment and the preset AI scenario library, predict the real-time state information of the vehicle and the extreme test scenario data at the next moment; According to the predicted vehicle state information and extreme test scenario data, use the cross-modal time delay reference template to calculate the predicted time delay of the virtual echo signal at the next moment: Δt predicted = a0 + a1v t+1 + a2θ t+1 + a3v t+1 θ t+1 + ε where, Δt predicted is the time delay of the virtual echo signal at the next moment predicted based on the current model; v t+1 is the predicted vehicle speed at the next moment; θ t+1 is the predicted vehicle yaw angle at the next moment; Based on the time delay control external VR module and radio frequency signal simulator at the current moment, generate corresponding extreme test scenario data; At the next moment, the actual real-time state information of the vehicle is collected by the multi-modal perception module, and according to the actual vehicle state at the next moment, the real virtual echo signal delay Δt is calculated and obtained actual : Δt actual = t r - t vision Compare the predicted time delay Δt predicted with the actually measured time delay Δt actual to calculate the time delay error and record the error of the vehicle at each moment, generating a time series of time delay errors: ∈ = Δt actual -Δt predicted where ε represents the error unexplained by the model, that is, the time delay error; Use the recursive least squares method (RLS) to dynamically adjust the time delay model parameters to make them adapt to the time delay change. The calculation formula for updating the model parameters is as follows: Compare the predicted time delay at the next moment with the actual time delay at the next moment, calculate the time delay error, record the time delay error of the vehicle at each moment, and generate a time delay error time series; Filter and correct the time delay error time series to reduce sudden interference and errors, and obtain the corrected time delay error time series, including: Filter and correct the time delay error time series, and use a low-pass filtering method to smooth the time delay error: Δt filtered = α∈+(1 - α)Δt filtered-prev where, ∈ is the difference between the actually observed delay and the delay predicted by the model at the current moment; Δt filtered-prev is the filtered delay error at the previous moment; α is the filtering coefficient, Δt filtered is the filtered delay error at the current moment; According to the corrected time delay error time series, update the cross-modal time delay reference template. The formula is: θ k = θ k-1 + K k ∈ Among them, is the delay model parameter vector; H k = [1, v, θ, vθ] is the observation matrix; K k is the gain matrix; p k is the covariance matrix; λ is the forgetting factor, which controls the influence of historical data on model update.

Images