Radar test method, system and equipment based on instance synthesis and storage medium

Abstract

The invention discloses a radar test method, system and device based on instance synthesis and a storage medium, which are corresponding schemes, and in the scheme, instances are inserted into point cloud data to synthesize new instances so as to test the perception ability of a radar perception system to newly added instances, thereby being beneficial to constructing a more challenging test scene, and improving the test efficiency. The robustness of a radar sensing system is improved; the metamorphic test method is applied to stability and consistency evaluation of the radar sensing system, and potential defects of the radar sensing system are systematically revealed according to whether a prediction result meets expectation or not. Through a multi-target search algorithm, a guidance fuzzy test is carried out with a fitness index as guidance, so that a test scene can expose sensing weaknesses of a radar sensing system to the maximum extent, the effectiveness and coverage of the test are improved, and finally, through an iterative optimization strategy, the test effect is continuously optimized, and the test efficiency is improved. And the stability and reliability of the radar sensing system in various complex scenes are ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of autonomous driving safety and software testing, and in particular, to a radar test method, system, device and storage medium based on instance synthesis. BACKGROUND

[0002] Radar perception systems have become an important component in autonomous driving. Radar perception systems use high-precision 3D (three-dimensional) information provided by radar sensors to perceive the environment. This information is represented in the form of point clouds, which are composed of a collection of three-dimensional data points representing the position of objects relative to the sensor. Since the subsequent operations of the autonomous driving system depend on the perception results of the LiDAR-Based Perception System (LPS), the failure of the radar perception system can lead to serious safety risks. Therefore, before deploying the radar perception system to the autonomous system, it must be rigorously tested to ensure its safety and reliability.

[0003] The main goal of the test is to create extreme scenarios that can trigger perception errors of the radar perception system, thereby evaluating its robustness, which refers to the ability of the radar perception system to maintain accurate and reliable performance in diverse and challenging scenarios. Existing radar perception system testing methods usually generate test scenarios by manipulating real-world data, such as adjusting environmental conditions, to generate real test scenario problems. However, these methods do not change the semantic properties of the scene, as for the simulation of weather, mainly by adding and deleting some point clouds, without introducing new semantics. Therefore, generating new test cases by synthesizing instances becomes an effective method to simulate real-world scenarios, by inserting new instances into the original scene, including pedestrians, cars, cyclists, etc., which can introduce new semantics into the original scene, and increase the diversity of the scene.

[0004] Although the realism and diversity of LiDAR (Light Detection and Ranging) testing are crucial, one challenge is the effectiveness of test case generation, as creating and evaluating 3D point cloud data is usually time-consuming and costly. Existing research usually randomly generates test scenarios, i.e., randomly inserts instances into the scene, which can result in redundant test cases and cannot effectively target system defects. In addition, the sparsity and highly structured nature of point cloud data make the development of effective testing algorithms challenging.

[0005] In recent years, in response to the quality assurance needs of deep learning systems, researchers have proposed a variety of innovative testing methods. Among them, differential testing and metamorphic testing, as two typical technical routes, have attracted widespread attention in academia and industry.

[0006] In differential testing research, researchers effectively solve the oracle problem faced by traditional testing methods by introducing a multi-model collaborative verification mechanism. The representative work DeepXplore framework, an automated white-box testing framework designed specifically for deep learning models, innovatively proposes a neuron coverage metric, which drives the testing process by constructing a combinatorial optimization problem. The core mechanism lies in simultaneously optimizing two key objectives: maximizing the behavior difference of multiple models and improving neuron coverage. The gradient search-based optimization algorithm enables this method to efficiently explore the decision boundaries of deep neural networks, successfully detecting edge cases that traditional methods cannot find in image classification, speech recognition, and other fields. Notably, this method controls the difference threshold through parameterization, allowing dynamic balancing between testing sufficiency and false positive rate, making it suitable for different security level application scenarios.

[0007] In the field of autonomous driving system testing, metamorphic testing technology exhibits unique advantages. The LiRTest framework, a black-box testing framework designed specifically for testing the performance of radar perception systems in complex weather environments, proposes a point cloud conversion operation based on physical modeling. The system simulates the LiDAR perception degradation effects under complex weather conditions such as rain and fog, and automatically determines the robustness of the perception results based on metamorphic relationships. This method not only constructs two types of realistic interference operations, but also analyzes the response differences of models to interference through neuron coverage and other indicators. Experimental results show that LiRTest can effectively detect erroneous behavior in four mainstream three-dimensional target detection models, and through conversion data assisted retraining, significantly improves the average precision (AP) and enhances the stability of models in complex environments.

[0008] However, the current work has the following defects: Current research mainly relies on 3D simulation environments to run deep neural network (DNN) based autonomous driving systems to evaluate their extreme errors in synthetic driving scenarios. However, although existing methods can automatically generate multiple categories of test scenarios and successfully reveal system defects, they focus more on the impact of weather on perception systems, and there is still a lack of research on how to increase new instances to improve scenario diversity and thus affect LiDAR perception system performance. At the same time, these studies mainly focus on image or simple environment testing, and lack in-depth discussion on the specific impact of point cloud data on autonomous driving system perception, decision-making, and robustness.

[0009] The performance of radar perception systems in complex environments is crucial due to their high sensitivity to changes in scene content. However, current research on 3D point cloud data still faces many challenges. First, there is still a lack of effective methods for systematically analyzing driving scenes composed of point cloud data and accurately identifying perception errors of autonomous driving systems. Second, existing testing methods often randomly insert instances, affecting the effectiveness of the test. Therefore, how to construct a reasonable metamorphic relationship on 3D point cloud data so that it can not only cover a variety of driving scenes but also ensure the relevance of the test and achieve automated perception error identification has become a core problem in the field of autonomous driving testing.

[0010] Therefore, the present application is proposed. SUMMARY

[0011] The purpose of the present application is to provide a radar testing method, system, device and storage medium based on instance synthesis, which fills the gap in the field of radar perception system testing. The present application combines metamorphic testing and instance synthesis technology to design an iterative optimization framework based on guided fuzzy testing. The present application more comprehensively, effectively and efficiently improves the testing capability of radar perception systems, ensuring the robustness and stability of the system in complex environments.

[0012] The purpose of the present application is achieved by the following technical solutions: A radar testing method based on instance synthesis, comprising: Collecting original radar point cloud data under multiple driving scenes to form an original data set; Extracting different instances in the original data set and inserting selected instances into each original radar point cloud data to obtain new radar point cloud data; Introducing a metamorphic testing method, inputting the original radar point cloud data and the corresponding new radar point cloud data into the radar perception system under test, and selecting the corresponding new radar point cloud data when the prediction results output by the radar perception system under test do not meet the expected requirements; Performing guided fuzzy testing using the selected new radar point cloud data, setting fitness indicators using the performance indicators of the radar perception system under test, and using a multi-objective search algorithm to guide the fitness indicators to trigger the radar perception system under test in a higher error direction, and continuously iterating until the set stop condition is met.

[0013] A radar testing system based on instance synthesis for implementing the foregoing method, comprising: A data collection unit for collecting original radar point cloud data under multiple driving scenes to form an original data set; An instance insertion unit for extracting different instances in the original data set and inserting selected instances into each original radar point cloud data to obtain new radar point cloud data; The metamorphosis test unit is configured to input original radar point cloud data and corresponding new radar point cloud data into the radar perception system under test, and select the corresponding new radar point cloud data when the prediction result output by the radar perception system under test does not meet the expected requirement. The guided fuzzy test unit is configured to perform guided fuzzy testing using the selected new radar point cloud data, set a fitness index using the performance index of the radar perception system under test, and use a multi-objective search algorithm to guide the test to trigger higher errors in the radar perception system under test, and iteratively until the set stop condition is met.

[0014] A processing device includes one or more processors, a memory for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the foregoing method.

[0015] A readable storage medium stores a computer program that, when executed by a processor, implements the foregoing method.

[0016] The technical solution provided by the above-mentioned application can be seen that by inserting new instances in point cloud data to test the radar perception system's perception ability of new instances, it helps to construct more challenging test scenarios and improve the robustness of the radar perception system. The metamorphosis test method is applied to the stability and consistency evaluation of the radar perception system, and the potential defects of the radar perception system are systematically revealed according to whether the prediction result meets the expectation. Through the multi-objective search algorithm and guided fuzzy testing with fitness index, the test scenario can maximize the exposure of the radar perception system's perception weaknesses, improve the effectiveness and coverage of the test, and finally, through the iterative optimization strategy, continuously optimize the test effect, and ensure the stability and reliability of the radar perception system in various complex scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0018] Figure 1 A flowchart of a radar test method based on instance synthesis is provided for the embodiments of the present application. Figure 2 A schematic diagram of the overall test process is provided for the embodiments of the present application. Figure 3A point cloud data synthesis diagram based on instance insertion provided for an embodiment of the present application; Figure 4 A point cloud data occlusion processing diagram provided for an embodiment of the present application; Figure 5 A radar test system diagram based on instance synthesis provided for an embodiment of the present application; Figure 6 A processing device diagram provided for an embodiment of the present application. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0020] First, the terms possibly used in the present text are explained as follows: The term “and / or” means either of the two or both can be realized, for example, X and / or Y means three cases including “X” or “Y” or “X and Y”.

[0021] The terms “include”, “contain”, “have”, “possess” or other similar semantic descriptions should be interpreted as non-exclusive inclusion. For example, including a technical feature element (such as raw materials, components, ingredients, carriers, dosage forms, materials, sizes, parts, components, mechanisms, devices, steps, processes, methods, reaction conditions, processing conditions, parameters, algorithms, signals, data, products or articles, etc.) should be interpreted as not only including the explicitly listed technical feature element, but also including other technical feature elements not explicitly listed in the art.

[0022] The term “consisting of” means excluding any technical feature element not explicitly listed. If this term is used in a claim, the term will make the claim closed, so that it does not contain technical feature elements other than the explicitly listed technical feature elements, except for conventional impurities related thereto. If the term only appears in a certain clause of the claim, it is only limited to the elements explicitly listed in that clause, and the elements recorded in other clauses are not excluded from the overall claim.

[0023] Unless otherwise specified or limited, the terms "mounted," "connected," "connect," and "fixed" should be interpreted broadly. For example, they can refer to fixed, detachable, or integral connections; mechanical or electrical connections; direct or indirect connections through an intermediary; and internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in this document based on specific circumstances.

[0024] The following describes in detail a radar testing method, system, device, and storage medium based on example synthesis provided by the present invention. Any information not described in detail in the embodiments of the present invention is prior art known to those skilled in the art. Where specific conditions are not specified in the embodiments of the present invention, the test was performed in accordance with conventional conditions in the art or the conditions recommended by the manufacturer. Reagents or instruments used in the embodiments of the present invention, where the manufacturer is not specified, are all commercially available conventional products.

[0025] Example 1 The embodiment of the present invention provides a radar testing method based on example synthesis, such as Figure 1 As shown, it mainly includes the following steps: Step 1: Collect raw radar point cloud data under various driving scenarios to form a raw data set.

[0026] In the embodiment of the present invention, the main purpose is to collect radar point cloud data in various driving scenarios, including complex environments such as urban roads, highways, and intersections.

[0027] Step 2: Extract different instances from the original data set and insert the selected instances into each original radar point cloud data to obtain new radar point cloud data.

[0028] This step primarily involves extracting instances (such as pedestrians, vehicles, and obstacles) from the collected data and performing instance insertion. Instance insertion is used to synthesize pedestrians, vehicles, obstacles, and other objects within the point cloud data to test the radar system's ability to perceive these newly added objects. This method helps construct more challenging test scenarios and improves the robustness of the radar system.

[0029] The preferred implementation of this step is as follows: different instances are extracted from the original data set to form an instance database; for each piece of original radar point cloud data, point cloud synthesis technology is used, and under environmental constraints, the selected instance is inserted into the original radar point cloud data; at the same time, the inserted instance is posture-adjusted to ensure that the inserted instance remains consistent within the environment of the original radar point cloud data, thereby ensuring that the point cloud of the inserted instance is not self-occluded due to its own angle.

[0030] Further, after inserting the selected instance into the original radar point cloud data, the inserted post-scene is processed based on a point cloud occlusion simulation method to generate occlusion scenes of different degrees (corresponding to different new radar point cloud data).

[0031] Step 3, introduce a transmutation test method, input the original radar point cloud data and the corresponding new radar point cloud data into the radar perception system under test, and select the corresponding new radar point cloud data when the prediction result output by the radar perception system under test does not meet the expected requirement.

[0032] The preferred embodiment of this step is as follows: input the original radar point cloud data into the radar perception system under test, calculate a plurality of baseline performance indicators in combination with the prediction result; input the new radar point cloud data into the radar perception system under test, calculate a plurality of evaluation performance indicators in combination with the prediction result; calculate the difference between the baseline performance indicators and the evaluation performance indicators under the same type, and if the difference under any type exceeds the threshold value, it indicates that the prediction result does not meet the requirement, and the corresponding new radar point cloud data is selected.

[0033] In the embodiment of the application, the performance indicators include the classification accuracy of the radar perception system under test and the mean intersection over union.

[0034] As can be understood by those skilled in the art, there are a plurality of points (data points) in the scene contained in one radar point cloud data, therefore, the prediction results of all the points are used to calculate the performance indicators of the radar point cloud data, and then whether the performance indicators meet the expected requirement is determined.

[0035] Step 4, use the selected new radar point cloud data for directional fuzzy testing, set an adaptability indicator using the performance indicators of the radar perception system under test, use a multi-objective search algorithm to guide the adaptability indicator, make the test trigger the radar perception system under test in a higher error direction, and constantly iterate until a set stop condition is met.

[0036] The preferred embodiment of this step is as follows: a multi-objective search algorithm is used to optimize the selected corresponding new radar point cloud data, including: using the adaptability indicator as a guide to optimize the pose of the inserted instance in the new radar point cloud data, adjust the point cloud density and data fusion mode, and optimize the occlusion relationship between the instance and the original radar point cloud data; in addition, the new radar point cloud data is constantly optimized during the iteration process until the adaptability indicator converges or the set iteration number is reached.

[0037] In the embodiment of the application, the optimization of the pose of the inserted instance in the new radar point cloud data includes: introducing an adaptive insertion strategy, dynamically adjusting the position and direction of the inserted instance according to the spatial layout of the instance in the scene in the new radar point cloud data; and fine-tuning the inserted instance in combination with the prediction result of the radar perception system under test on the original radar point cloud data.

[0038] In order to more clearly demonstrate the technical solution and technical effects provided by the present invention, the method provided by the embodiment of the present invention is described in detail below with reference to specific embodiments.

[0039] 1. Overall overview of the plan.

[0040] Considering that existing radar perception system testing methods fail to fully integrate metamorphic testing and example synthesis techniques and lack the ability to test diverse radar point cloud data in complex driving scenarios, this paper provides a radar testing method based on example synthesis to achieve three objectives: 1. Propose a metamorphic relationship for point cloud data and apply it to a real annotated point cloud dataset (original dataset) to generate diverse test point clouds (new radar point cloud data); 2. Test and analyze the radar perception systems of multiple autonomous driving systems using point cloud data to evaluate their performance and deficiencies; and 3. Introduce a fitness metric to guide the testing process, improving test effectiveness and efficiency.

[0041] like Figure 2 The figure shows the overall testing process. By extensively collecting raw radar point cloud data from a variety of complex driving scenarios, such as urban roads, highways, and intersections, rich and diverse basic data support is provided for testing. Next, instance insertion technology is introduced to synthesize instances such as pedestrians, vehicles, and obstacles into the raw radar point cloud data to test the radar perception system's perception capabilities under these newly added instances, effectively increasing the challenge and diversity of the test. Subsequently, the present invention applies a metamorphic testing method to the stability and consistency assessment of the radar perception system. By defining the expected relationships between different input scenarios and conducting metamorphic testing, potential defects in the radar system are systematically revealed. Next, guided fuzz testing is employed to optimize the testing process. A multi-target search algorithm guides the adjustment (mutation) of the test input, enabling the test scenarios to maximize the exposure of the system's perception weaknesses, improving the effectiveness and coverage of the test. Finally, an iterative optimization strategy is implemented to continuously optimize the test results, ensuring the stability and reliability of the radar perception system in a variety of complex scenarios.

[0042] Figure 2 The selection and mutation in the algorithm are both inherent steps in the multi-target search algorithm. Specifically, radar point cloud data that violates the MR (metamorphosis relationship) is selected and subjected to guided fuzzy testing and optimization. In this process, fitness indicators are used to evaluate and select point cloud data with a greater impact on the radar perception system indicators. These data are then mutated to generate new radar point cloud data. Figure 2The multi-objective function involved in the middle part defines multiple evaluation indicators (to be described later) on which the system evaluates candidate solutions, and the multi-objective search algorithm is a method for finding the optimal solution set under the constraints of these multi-objective functions, and the effective data obtained finally is the radar point cloud data (data violating the MR rule) obtained after iteration.

[0043] II. Detailed introduction of the scheme.

[0044] 1. Collect data sets and complete instance extraction.

[0045] In the data collection phase, high-quality raw radar point cloud data is first screened and sorted to form a raw data set, to ensure that a variety of autonomous driving scenarios are covered, including urban roads, highways, intersections, and complex environments (such as rainy days, night, occlusion, etc.). The data sources include real-world LiDAR scanning data, such as the KITTI dataset, the Waymo dataset, etc., to enhance the diversity and robustness of the data. In addition, the raw radar point cloud data is preprocessed, such as denoising, coordinate normalization, and target labeling, to improve the effectiveness of subsequent applications.

[0046] In the instance extraction phase, specific instances are extracted from the collected raw data set according to the label information therein. For example, in the autonomous driving test scenario, attention is focused on vehicles, pedestrians, bicycles, and other instances.

[0047] In the embodiment of the present application, the measured radar perception system can be selected according to the requirements, for example, the existing radar perception system can be directly used, or the radar perception system can be constructed and trained according to the requirements. Taking the latter as an example, a deep learning architecture suitable for point cloud processing is selected, such as PointNet (a neural network that directly processes raw point cloud data), PointNet++ (a neural network that introduces a local region extraction mechanism based on PointNet), PV-RCNN (a three-dimensional target detection network that integrates point cloud voxel features and original point features), etc., and an appropriate network structure is selected based on experimental requirements to form the measured radar perception system. The training data can be selected from the raw data set collected in the foregoing, and a supervised learning or semi-supervised learning method is used for optimization. In order to improve the generalization ability of the measured radar perception system, a data enhancement strategy is introduced, such as random rotation, translation, scale scaling, and occlusion simulation. In addition, a reasonable loss function and optimization strategy are designed to ensure the performance of the measured radar perception system in target detection, target segmentation, or scene understanding tasks.

[0048] 2. Point cloud data synthesis based on instance insertion.

[0049] In an embodiment of the present invention, appropriate target instances (such as vehicles, pedestrians, bicycles, etc.) are selected based on the collected point cloud dataset and inserted into new scenes through point cloud synthesis technology to expand the diversity of test data. The key to instance insertion is to maintain the authenticity and rationality of the data. Therefore, it is necessary to consider the environmental constraints of the inserted objects, such as road structure, motion state, and physical consistency. For example, in urban road scenes, inserted vehicles need to comply with traffic flow rules to avoid unreasonable overlap or interlacing. In addition, methods such as point cloud interpolation, posture adjustment, and background fusion are used to ensure that the inserted instances maintain consistency within the overall point cloud environment and reduce unnatural splicing traces.

[0050] Figure 3 The figure below shows the new radar point cloud data generated after adding a car and a pedestrian. The upper part shows the original radar point cloud data with only one car in the scene, while the lower part shows the radar point cloud data after adding two instances of the car and pedestrian.

[0051] During the occlusion processing phase, the focus is on studying the occlusion of target instances in complex environments to simulate the potential line-of-sight occlusions that can occur in real-world driving scenarios. For example, a large truck on a highway may obstruct vehicles behind it, and pedestrians on urban roads may be partially obscured by parked vehicles. To this end, a point cloud-based occlusion simulation method is used to generate scenarios with varying degrees of occlusion to test the robustness of the baseline model. Specific implementation methods include adding virtual obstructing objects (such as obstacles and buildings), adjusting the viewing angle (such as simulating blind spots of cameras or LiDAR sensors), and simulating dynamic occlusion by partially deleting portions of the point cloud.

[0052] like Figure 4 The figure shows the occlusion operation on a radar point cloud data. The upper part is the data without occlusion after inserting the instance, and the lower part is the data after occlusion.

[0053] 3. The transformation relationship of instance insertion.

[0054] During instance insertion, it is necessary to ensure that the inserted instance does not adversely affect the recognition of the original scene. In other words, the transformation relationship must be maintained to ensure that the perception system can still correctly recognize the original target object when processing the scene after the instance is inserted. Specifically, instance insertion should follow the following principles: (1) Recognition stability of original targets: The recognition results of target instances in the original scene should remain stable, that is, inserting additional instances should not degrade the detection performance of existing targets. For example, if the system can accurately detect the vehicle ahead without inserting instances, then it should still be able to correctly detect the vehicle ahead after inserting pedestrians, non-motor vehicles, or other background objects.

[0055] (2) Recognizability of new instances: The inserted instances should be correctly detected by the radar perception system, and should not be misclassified or unrecognized due to data fusion problems.

[0056] In the embodiments of the present application, the classification accuracy (Accuracy, Acc) and the mean Intersection over Union (mIoU) are used as core evaluation indicators. Among them, Acc is used to measure the overall detection accuracy of the target instance by the perception system, and mIoU is used to evaluate the overlap between the prediction result and the real label in the target detection or segmentation task.

[0057] The calculation formula of the classification accuracy (Accuracy, Acc) is as follows: ; Among them, TP (True Positive) is the number of correct detection as target instance; TN (True Negative) is the number of correct detection as background; FP (False Positive) is the number of false detection as target instance; FN (False Negative) is the number of missed detection of target instance.

[0058] The calculation formula of the mean Intersection over Union (mIoU) is as follows: ; ; Among them, A represents the predicted target area; B represents the real labeled target area; N is the total number of classes (in the semantic segmentation task, the mean IoU of different classes is calculated), and IoU is the Intersection over Union.

[0059] In order to verify whether the instance insertion satisfies the metamorphosis relationship, a systematic experimental evaluation process is designed. First, in the benchmark evaluation stage, the radar perception system is run without inserting instances, and Acc and mIoU are calculated as benchmark performance indicators. Subsequently, in the instance insertion and scene transformation stage, appropriate instances (such as vehicles, pedestrians, and cyclists) are selected based on the instance database, and the insertion position and orientation are reasonably set according to the spatial layout and traffic rules of the current scene. In order to improve the data quality, point cloud fusion technology is used to make the point cloud density of the inserted instance match the surrounding environment, avoiding misjudgment of the perception system due to uneven data distribution.

[0060] During the perception system evaluation phase, the radar perception system is run, and Acc and mIoU are recalculated in the scene with the inserted instance (new radar point cloud data) and compared with the original scene (original radar point cloud data). If the perception system's Acc and mIoU drop significantly after the instance is inserted, it indicates that the inserted instance has affected the recognition of the original scene, violating the transformation relationship. To quantitatively measure the performance change, the following metrics are calculated: ; ; in, and Corresponding to the Acc and mIoU representing the original radar point cloud data, and The corresponding representations are the Acc and mIoU of the new radar point cloud data after the instance is inserted.

[0061] If the classification accuracy changes Or the change of mean intersection and union ratio If the threshold is exceeded, it is determined that the instance insertion has a significant impact on the performance of the perception system and needs further optimization.

[0062] 4. Iterative optimization based on guided fuzz testing.

[0063] In this embodiment of the present invention, a guided fuzzy test is performed using the new radar point cloud data selected in the aforementioned steps. Acc and mIoU are used as fitness indicators, and a multi-target search algorithm is adopted. Guided by the fitness indicators, the test is conducted in a direction that triggers higher errors in the tested radar perception system, and the test is iterated continuously to find scenarios that induce errors in the radar perception system.

[0064] During the iterative optimization phase, adjustments are made to scenarios where recognition performance degrades to reduce the interference of inserted instances on original target detection. First, the position and orientation of instance insertions are optimized to prevent them from affecting the detection results of key targets. Second, the point cloud density and data fusion methods are adjusted to better reflect real-world traffic conditions. Furthermore, occlusion relationships are optimized to ensure that inserted instances do not completely obscure key targets, but rather partially obscure them or remain within a reasonable visible range. To further enhance the robustness of the perception system, a data augmentation strategy is introduced, utilizing multiple insertion methods (different locations and different category combinations) for training to adapt the model to different instance insertion scenarios.

[0065] Specifically: an adaptive adjustment strategy is introduced, which dynamically determines the candidate type set and the insertable region set of the instance according to the instance space layout information in the current scene (such as the category and distribution area of the existing instance), so as to ensure that the insertion operation conforms to the distribution characteristics of the real traffic flow. Subsequently, a fine-tuning mechanism is introduced for fine selection of specific insertion instances and insertion positions in the above candidate set. The fine-tuning stage selects the instance type most suitable for the current context from the insertable instance type set, and determines the optimal insertion position and direction from the insertable region. This process combines the perception results of the benchmark model (such as the target detector) under the current insertion scheme for feedback, optimizes the insertion selection, and thus most likely generates a scenario that affects the perception system.

[0066] The detection accuracy and segmentation quality of the inserted instance are calculated at different iteration stages, and the performance change trend is observed. If Acc or mIoU decreases significantly, analyze the possible reasons, such as unreasonable position of the inserted instance, excessive occlusion, or abnormal point cloud density distribution, and optimize by adjusting the insertion strategy.

[0067] In addition, in order to test the impact of the generated scene on the perception system in depth, comparative tests are carried out in multiple experimental environments, including original point cloud scenes, initial instance insertion scenes, and insertion scenes after iterative optimization. By comparing the performance of the perception system in different scenes, the influence of instance insertion on detection and segmentation tasks can be quantified, and the effectiveness of the optimization strategy can be verified.

[0068] During the multiple rounds of optimization process, performance threshold judgment criteria are set. If the decrease of Acc and mIoU is controlled within a reasonable range (for example, ΔAcc<2% and ΔmIoU<3%), it is considered that the data meets the metamorphic relationship, otherwise the insertion strategy is continued to be adjusted or it is marked as high-impact data. In addition, in the high-impact data screening stage, the insertion data that has the greatest impact on the perception system is selected from all experimental data, i.e. those scenes that cause the greatest ΔAcc and ΔmIoU decrease, and are saved for further testing and optimization of the perception system.

[0069] Through the description of the above implementation, those skilled in the art can clearly understand that the above embodiments can be implemented by software, or by means of software and necessary general hardware platform. Based on such understanding, the technical solutions of the above embodiments can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.), and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present application.

[0070] Embodiment Two The present invention also provides a radar test system based on example synthesis, which is mainly used to implement the method provided in the above embodiment, such as Figure 5 As shown, the system mainly includes: A data collection unit is used to collect raw radar point cloud data under various driving scenarios to form a raw data set; An instance insertion unit is used to extract different instances from the original data set and insert the selected instances into each original radar point cloud data to obtain new radar point cloud data; The metamorphosis test unit is used to input the original radar point cloud data and the corresponding new radar point cloud data into the radar perception system under test respectively, and select the corresponding new radar point cloud data when the prediction result output by the radar perception system under test does not meet the expected requirements; The guided fuzzy test unit is used to perform guided fuzzy testing using the selected new radar point cloud data, set the fitness index using the performance index of the radar perception system under test, adopt a multi-target search algorithm, and use the fitness index as a guide to make the test trigger the radar perception system under test in a higher error direction, and iterate continuously until the set stop condition is met.

[0071] Considering that the main processing procedures involved in each unit have been introduced in detail in the above embodiments, they will not be repeated here.

[0072] Those skilled in the art will clearly understand that for the convenience and brevity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above.

[0073] Example 3 The present invention also provides a processing device, such as Figure 6 As shown, it mainly includes: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method provided by the aforementioned embodiment.

[0074] Furthermore, the processing device further includes at least one input device and at least one output device; in the processing device, the processor, memory, input device, and output device are connected via a bus.

[0075] In the embodiment of the present invention, the specific types of the memory, input device, and output device are not limited; for example: The input device can be a touch screen, image acquisition device, physical button or mouse; The output device may be a display terminal; The memory can be a Random Access Memory (RAM) or a non-volatile memory such as a disk memory.

[0076] Embodiment Four The application further provides a readable storage medium storing a computer program, which, when executed by a processor, implements the method provided in the foregoing embodiments.

[0077] The readable storage medium in the embodiments of the application as a computer readable storage medium can be arranged in the foregoing processing device, for example, as a memory in the processing device. In addition, the readable storage medium can also be a U disk, a mobile hard disk, a Read-Only Memory (ROM), a magnetic disk or an optical disk, and various media capable of storing program codes.

[0078] The above merely describes the preferred embodiments of the application, but the protection scope of the application is not limited to this. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed by the application, which should be covered by the protection scope of the application. Therefore, the protection scope of the application should be subject to the protection scope of the claims. The information disclosed in the background section of the specification is merely intended to deepen the understanding of the general background of the application, and should not be regarded as acknowledging or implying in any form that the information constitutes the prior art known by the person skilled in the art.

Claims

1. A radar testing method based on example synthesis, characterized in that: include: Collect raw radar point cloud data under various driving scenarios to form a raw data set; Extract different instances from the original data set and insert the selected instances into each original radar point cloud data to obtain new radar point cloud data; A metamorphic test method is introduced to input the original radar point cloud data and the corresponding new radar point cloud data into the radar perception system under test respectively. When the prediction result output by the radar perception system under test does not meet the expected requirements, the corresponding new radar point cloud data is selected; The selected new radar point cloud data is used for guided fuzzy testing. The performance indicators of the radar perception system under test are used to set the fitness index. A multi-target search algorithm is adopted. Guided by the fitness index, the test triggers the radar perception system under test in a higher error direction and iterates continuously until the set stop condition is met.

2. The radar testing method based on example synthesis according to claim 1, characterized in that: The extracting different instances from the original data set and inserting the selected instances into each original radar point cloud data to obtain new radar point cloud data includes: Extract different instances from the original data set to form an instance database; For each piece of original radar point cloud data, point cloud synthesis technology is used, and under environmental constraints, the selected instances are inserted into the original radar point cloud data; at the same time, the posture of the inserted instances is adjusted to ensure that the inserted instances remain consistent in the environment of the original radar point cloud data.

3. The radar testing method based on example synthesis according to claim 2, characterized in that: Also includes: After inserting the selected instances into the original radar point cloud data, occlusion scenes with different degrees are generated based on the point cloud occlusion simulation method.

4. The radar testing method based on example synthesis according to claim 1, characterized in that: The original radar point cloud data and the corresponding new radar point cloud data are respectively inputted into the radar perception system under test. When the prediction result outputted by the radar perception system under test does not meet the expected requirements, the corresponding new radar point cloud data is selected, including: Input the original radar point cloud data into the radar perception system under test, and combine it with the prediction results to calculate several benchmark performance indicators; Input the new radar point cloud data into the radar perception system under test, and calculate several evaluation performance indicators based on the prediction results; Calculate the difference between the baseline performance index and the evaluation performance index under the same type. If the difference under any type exceeds the threshold, it means that the prediction result does not meet its requirements, and the corresponding new radar point cloud data is selected.

5. The radar testing method based on example synthesis according to claim 1 or 4, characterized in that: The performance indicators include: classification accuracy and mean intersection of the tested radar perception system.

6. The radar testing method based on example synthesis according to claim 1, characterized in that: The multi-target search algorithm is used to guide the fitness index so that the test triggers the radar perception system under test in a direction with a higher error, including: A multi-objective search algorithm is used to optimize the selected corresponding new radar point cloud data, including: optimizing the posture of the instance inserted into the new radar point cloud data guided by the fitness index, adjusting the point cloud density and data fusion method, and optimizing the occlusion relationship between the instance and the original radar point cloud data.

7. The radar testing method based on example synthesis according to claim 6, characterized in that: Optimizing the posture of the inserted instance in the new radar point cloud data includes: introducing an adaptive insertion strategy to dynamically adjust the position and orientation of the inserted instance based on the spatial layout of the instance in the scene in the new radar point cloud data; and fine-tuning the inserted instance based on the prediction results of the tested radar perception system on the original radar point cloud data.

8. A radar test system based on example synthesis, characterized in that: The method for implementing any one of claims 1 to 7 comprises: A data collection unit is used to collect raw radar point cloud data under various driving scenarios to form a raw data set; An instance insertion unit is used to extract different instances from the original data set and insert the selected instances into each original radar point cloud data to obtain new radar point cloud data; The metamorphosis test unit is used to input the original radar point cloud data and the corresponding new radar point cloud data into the radar perception system under test respectively, and select the corresponding new radar point cloud data when the prediction result output by the radar perception system under test does not meet the expected requirements; The guided fuzzy test unit is used to perform guided fuzzy testing using the selected new radar point cloud data, set the fitness index using the performance index of the radar perception system under test, adopt a multi-target search algorithm, and use the fitness index as a guide to make the test trigger the radar perception system under test in a higher error direction, and iterate continuously until the set stop condition is met.

9. A processing device, characterized in that: include: one or more processors; a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images