Object replacement-based automatic driving target detection system testing method and system

Abstract

The application discloses an automatic driving target detection system test method and system based on object replacement, and defines a metamorphic relationship suitable for a target detection system in an automatic driving scene on the basis of applying metamorphic testing theory. Through use of an automatic driving scene data set with instance level labeling, a replaceable object is replaced. A mask matching mode is used to replace object pixels in a scene, that is, pixels of a replaceable object in an original scene are replaced by an object from another scene, so that a derived test case is generated. Compared with existing methods, the test case scene generated by the application is more real, and error revealing capability is improved. Meanwhile, a complete intersection and union ratio matrix based test result evaluation is used, so that the possibility of misjudgment of an overlapping situation of a predicted bounding box is reduced.

Description

Technical Field

[0001] This invention belongs to the field of autonomous driving testing technology, and relates to a testing method and system for a target detection system, specifically a transformative testing method and system for an autonomous driving target detection system based on object replacement. Background Technology

[0002] The object detection system in an autonomous driving system is responsible for processing image data received by cameras to identify important objects, including pedestrians, other vehicles, and traffic signs. As a perception module within the autonomous driving system, the object detection system relies on pre-trained deep neural networks and is a critical module. The reliability and robustness of the object detection system are essential for achieving safe and efficient autonomous driving; therefore, sophisticated testing techniques are needed to verify its robustness.

[0003] Existing methods widely use metamorphic testing to synthesize driving scene images to test perception modules. This typically involves mutating the input images to the visual perception module, such as adding noise to the entire image, adding extra objects to the background, or using generative adversarial networks to generate images with different appearances. The expectation is that the autonomous driving system can make acceptable decisions on these mutated images. However, current test cases generated using metamorphic operations are often of low quality. For example, global modifications (adding noise) can create invalid scenarios; inserting arbitrary objects into the background can create unrealistic scenarios.

[0004] Current methods have limitations because they do not focus on key pixels in the scene image, i.e., the objects to be identified. There are two main methods for testing autonomous driving perception modules based on images: full-scene transformation and object insertion into the background. 1. Full-scene transformation: DeepRoad uses generative adversarial networks to generate driving scenarios with different weather conditions. Test cases generated in this way are typically used to test the decision-making capabilities of autonomous driving systems, but not suitable for testing object detection systems. 2. Object insertion: Two representative methods are SemImFuzz and MetaOD. SemImFuzz directly inserts the original object location into the image without additional affine transformation, thus limiting the number of generated test cases. MetaOD inserts the object near existing objects or into the background and performs a somewhat restricted affine transformation. However, images generated in this way may contain unrealistic scene elements. Currently, existing methods for testing object detection systems suffer from the following problems: low efficiency in generating test cases and difficulty in revealing different system errors. Furthermore, existing evaluation metrics typically rely only on the intersection-over-union (IoU) area of ​​bounding boxes, which is prone to object overlap and small object recognition issues in autonomous driving scenarios, leading to numerous misjudgments.

[0005] The transformative testing of autonomous driving target detection systems involves the following technical backgrounds:

[0006] 1. Object detection algorithms based on deep neural networks: including two-stage object detection algorithms (such as Faster-RCNN and Mask-RCNN) and single-stage object detection algorithms (such as YOLO and RetinaNet).

[0007] 2. Datasets: Datasets used to train and test object detection systems, such as Cityscapes, KITTI, and Apollo.

[0008] 3. Test Case Generation Techniques: These techniques generate new test cases by processing original driving scene images. Common methods include adversarial example generation, metamorphosis operations, and generation based on generative adversarial networks.

[0009] 4. Evaluation method of test results: The classic evaluation method is to calculate the average precision (AP) based on the area of ​​intersection and union of bounding boxes. It is necessary to calculate the precision and recall that constitute the area below the precision and recall curve. Summary of the Invention

[0010] To address the shortcomings in test case generation and evaluation methods for target detection systems in autonomous driving scenarios, this invention provides a transformative testing method and system for autonomous driving target detection systems based on object replacement.

[0011] The technical solution adopted by the method of the present invention is: a testing method for an autonomous driving target detection system based on object replacement, comprising the following steps:

[0012] Step 1: Collect instance-level labeled raw autonomous driving scene data to construct a raw autonomous driving scene dataset; process the images in the raw autonomous driving scene dataset, using one part as the raw scene images and the other part as an object pool to extract source images for identifiable objects; use the raw scene images as input to the target detection system under test to obtain the target detection results on the raw scene images;

[0013] Step 2: Define the transformation relationship of the target detection system in the autonomous driving scenario, and generate derived test cases from the original scene image based on object replacement. The derived test cases include the generated new scene image and the location record information of the replaced object.

[0014] Step 3: The generated new scene image is used as input to the target detection system under test for processing to obtain the target detection results for the new scene. The cross-union matrix of the target detection results of the new scene and the target detection results of the original scene image is calculated according to the predefined metamorphic relationship to determine whether the target detection system has produced an unexpected erroneous output.

[0015] Step 4: Use the optimal matching method to filter out the incorrect predicted bounding boxes from the complete intersection-union matrix, while considering the geometric relationship, position, category, and intersection-union features between the predicted bounding boxes and the ground truth bounding boxes, to accurately locate the specific target detection errors.

[0016] Preferably, in step 1, the objects in the object pool come from different driving scenarios, including different weather, lighting and urban street scenery.

[0017] Preferably, in step 1, during the construction of the object pool, low-quality objects with an area smaller than a threshold are filtered out.

[0018] Preferably, in step 2, the metamorphic relationship is defined as follows: the detection result of a certain object in the scene is not affected by other objects, that is, the detection result of the object in the scene is context-independent; the metamorphic relationship is based on replacing objects with similar shapes to simulate object changes in real autonomous driving scenarios.

[0019] Preferably, in step 2, the method of generating derived test cases from the original scene image based on object replacement is based on instance-level annotation to obtain the semantic segmentation mask of the original object in the original scene. By iterating through the extracted objects in the object pool, the shape similarity between the original object and the extracted objects is compared, and objects with a similarity higher than a threshold are selected to replace objects in the original scene. After replacing a certain object in the original scene image, a derived test case is generated. The number of objects contained in the derived test case is the same as that in the original scene image, and the scene pixels are not destroyed. Only the pixels of the replaced object are changed to a matching object from the object pool.

[0020] As a preferred method, the object matching stage calculates the shape similarity, which involves affine transformation of the objects to be matched, including mask scaling, translation and symmetry processing; the maximum semantic mask intersection-union ratio when the bounding boxes coincide after affine transformation is selected as the shape similarity of the two objects to be replaced.

[0021] Preferably, the perfect crossover ratio in step 3 is:

[0022]

[0023]

[0024] Where IoU measures the degree of overlap between two bounding boxes by calculating the ratio of their intersection area to their union area; b represents the predicted bounding box; b gt Represents the truth bounding box; ρ 2represents the Euclidean distance between the center points of the two bounding boxes; c represents the diagonal distance of the minimum enclosing rectangle of the two bounding boxes to be matched; v is used to measure the aspect ratio similarity of the two bounding boxes, and the two bounding boxes b and b gt have their lengths and widths denoted as w, h and w respectively gt , h gt .

[0025] As a preference, in step 4, the optimal matching method is used to screen out falsely predicted bounding boxes from the complete intersection over union matrix. First, the predicted bounding boxes are evaluated in descending order of confidence, then the ground truth bounding box with the largest complete intersection over union value with a certain ground truth bounding box is preferentially selected for comparison. According to two predefined thresholds CIoULow and CIoUHigh, where 0<=CIoULow<CIoUHigh<=1, the error type of the predicted bounding box is determined; according to the error type of the predicted bounding box, the target detection error type is obtained, including classification error, localization error, recognition error or unrecognition error;

[0026] The classification error means that the predicted category of the optimally matched predicted bounding box is inconsistent with that of the ground truth bounding box, but the CIoU value exceeds the threshold CIoUHigh;

[0027] The localization error means that the predicted category of the optimally matched predicted bounding box is consistent with that of the ground truth bounding box, but the CIoU value is between CIoUHigh and CIoULow;

[0028] The recognition error refers to a redundant predicted bounding box;

[0029] The unrecognition error means that a certain ground truth bounding box has no matching predicted bounding box.

[0030] The technical solution adopted by the system of the present invention is: an autonomous driving object detection system testing system based on object replacement, comprising:

[0031] one or more processors;

[0032] a storage device, configured to store one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the aforementioned method for testing an autonomous driving object detection system based on object replacement.

[0033] This invention proposes a metamorphic testing method and system for autonomous driving target detection systems based on object replacement. This method can automatically generate more realistic autonomous driving scenarios and trigger errors caused by the boundary behavior of the target detection system. Based on metamorphic testing theory, this invention defines metamorphic relationships applicable to target detection systems in autonomous driving scenarios. By using an autonomous driving scene dataset with instance-level annotations, identifiable objects are replaced. A mask matching method is used to replace object pixels in the scene, that is, replacing the pixels of an identifiable object in the original scene with objects from other scenes, thereby generating derived test cases.

[0034] Compared with existing methods, the beneficial effects of the present invention include:

[0035] (1) This invention is based on object replacement, selecting identifiable objects in the original scene for replacement. This allows for the derivation of multiple new driving scenarios from a single driving scenario as transformative test cases. The source of the object replacement is identifiable objects from different city datasets, enriching the existing driving scenario dataset. Furthermore, this invention focuses on key pixels in the scene, i.e., pixels that identify objects. Compared to inserting objects into the background, this method is more likely to cause inconsistencies in object detection systems.

[0036] (2) The test cases generated by this invention are more likely to trigger target detection errors in the system, and the evaluation method is easier to distinguish specific error types; the generated test scenarios are more diverse; and the appearance of the test cases is close to the real scene.

[0037] (3) The test cases generated by this invention are more realistic and have an improved ability to reveal errors. At the same time, it uses a complete intersection-union matrix to evaluate the test results, which reduces the possibility of misjudging the overlap of predicted bounding boxes. Attached Figure Description

[0038] The technical solutions of the present invention will be further illustrated below using embodiments and specific implementation methods. In addition, some accompanying drawings are used in the description of the technical solutions. Those skilled in the art can obtain other drawings and the intent of the present invention from these drawings without any creative effort.

[0039] Figure 1 This is an overall flowchart of the method according to an embodiment of the present invention.

[0040] Figure 2 This is a schematic diagram of the method process according to an embodiment of the present invention.

[0041] Figure 3 A schematic diagram illustrating four types of errors revealed by the test cases used to generate the embodiments. Detailed Implementation

[0042] To facilitate understanding and implementation of the present invention by those skilled in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0043] Please see Figure 1 and Figure 2 This embodiment provides a testing method for an autonomous driving target detection system based on object replacement, including the following steps:

[0044] Step 1: Collect instance-level labeled raw autonomous driving scene data to construct a raw autonomous driving scene dataset; process the images in the raw autonomous driving scene dataset, using one part as the raw scene images and the other part as an object pool to extract source images for identifiable objects; use the raw scene images as input to the target detection system under test to obtain the target detection results on the raw scene images;

[0045] Using a dataset with instance-level annotations, pixel masks of objects to be identified can be quickly extracted from the background scene. After selecting an original driving scene image, to replace objects in the current scene, objects need to be extracted from other driving scenes to build an object pool. The size of the object pool is defined as the total number of objects in the selected other scene images. The data for a single object in the object pool includes the object's pixels, mask, and class label. After the object pool is built, iterative operations to retrieve and delete object elements are supported.

[0046] In one implementation, the objects in the object pool come from scenes with different weather, lighting, and cityscapes than the original driving scene, and the object categories in the object pool are restricted to the set of categories that the target detection system can recognize.

[0047] In one implementation, during the construction of the object pool, low-quality objects with an area smaller than a threshold are filtered out.

[0048] Step 2: Define the transformation relationship of the target detection system in the autonomous driving scenario, and generate derived test cases from the original scene image based on object replacement. The derived test cases include the generated new scene image and the location record information of the replaced object.

[0049] In one implementation, for the target detection task in an autonomous driving scenario, the metamorphic relationship suitable for the target detection system in the autonomous driving scenario is defined as follows: the detection result of a certain object in the scenario is not affected by other objects, that is, the detection result of the object in the scenario is context-independent.

[0050] In one implementation, the formal definition of the metamorphic relation is as follows:

[0051] Given an original autonomous driving scene image O and an object detection system D, the set of recognition results of the object detection system for this original autonomous driving scene image is Set. O Set O Each element in the dataset is a triple (box, class, conf), representing the recognition result of a specific object in a scene, including the bounding box (box), class label (class), and confidence score (conf). Based on the mask-matching object replacement operation, an object α in the original scene image is replaced with an object β from the object pool, generating a derived test case scene image R. The generated scene image R is input into the target detection system D under test to obtain a new set of recognition results, Set. R To eliminate the direct impact of object replacement, a collection called RepSet is defined. α and RepSet β Records the set of bounding boxes that directly overlap with the replacement object, where RepSet α It is a Set O A subset of RepSet β It is a Set R A subset of. Then, from the set of bounding boxes identified, respectively... O and Set R Subtract RepSet from the middle α and RepSet β The set. Finally, a method based on the complete intersection-union matrix is ​​used to determine whether the recognition results of the remaining objects are consistent. That is, if the equation... If this is true, then the output of the current target detection system can be considered not to violate the metamorphic relationship. The symbol represents the equivalence determination of the recognition results based on the complete intersection-union matrix.

[0052] The specific implementation of metamorphic operations involves object replacement based on mask matching. Predefined metamorphic relationships expect that replacing one object will not affect the detection results of other objects. This consistent characteristic ensures the safety of autonomous driving systems in various driving scenarios.

[0053] The specific transformation operation of object replacement based on mask matching is as follows: Given a driving scene image containing a set of N labeled and identifiable objects {O0, O1, O2, ..., O...} N-1}, randomly select an original scene object O i And use a matching object O from the object pool R Perform the replacement. The matched object O R With the original object O iThey share the same class label and have high shape similarity. Shape similarity is calculated using the intersection-union ratio of the masks after affine transformation. The purpose of affine transformation is to ensure that the minimum bounding rectangles of the masks of the two objects to be matched completely overlap. The main operations of affine transformation are translation and scaling. Original object O i The mask is mask1, and the matching object is O. R The mask is mask2. The following formula demonstrates the replacement process: the ~ symbol is a bitwise inversion of the mask, which means that all background pixels except the original object are preserved. The object is scaled and translated by applying the affine transformation function transform, or by adjusting the minimum bounding rectangle of the masks of both objects to the same size, and then replaced in place of the original object according to mask1.

[0054] (~mask1)×image1+transform(mask2×image2)

[0055] For a driving scene image dataset collected from a certain city, a driving scene image dataset from another city should be used as the object pool to extract object sources. Using the method described in step 2, a set of derived test cases can be generated from the original scene image set. Each derived test case contains the newly generated driving scene image and the location information of the replaced object. The location information of the replaced object can be saved using a JSON format file.

[0056] In one implementation, the object replacement-based method generates derived test cases from the original scene image. This method is based on instance-level annotation, which can obtain the semantic segmentation mask of the original objects in the original scene. By iterating through the extracted objects in the object pool, the shape similarity between the original object and the extracted objects is compared. Objects with a similarity higher than a threshold are selected to replace objects in the original scene. After replacing a certain object in the original scene image, a derived test case is generated. The number of objects contained in the derived test case is the same as that in the original scene image, and the scene pixels are not destroyed. Only the pixels of the replaced object are changed to a matching object from the object pool.

[0057] In one implementation, the object matching stage calculates shape similarity, which involves affine transformation of the objects to be matched, including mask scaling, translation and symmetry processing; the maximum semantic mask intersection-union ratio when the bounding boxes coincide after affine transformation is selected as the shape similarity of the two objects to be replaced.

[0058] Step 3: The generated new scene image is used as input to the target detection system under test for processing to obtain the target detection results for the new scene. The cross-union matrix of the target detection results of the new scene and the target detection results of the original scene image is calculated according to the predefined metamorphic relationship to determine whether the target detection system has produced an unexpected erroneous output.

[0059] In one implementation, the Complete Intersection over Union (CIoU) method is used to evaluate the recognition consistency of the object detection system. CIoU, in addition to considering the intersection-over-union area of ​​the predicted and ground truth bounding boxes, also takes into account geometric parameters such as the aspect ratio, center-to-center distance, and diagonal distance of the minimum bounding rectangle between the two bounding boxes, comprehensively calculating the CIoU value. Based on this CIoU calculation method, a CIoU matrix is ​​calculated for all sets of ground truth and predicted bounding boxes.

[0060] As an example, the following specific example illustrates the processing flow.

[0061] Given an original driving scene image O and a new driving scene image R generated by applying the metamorphic operation described in step 2, the core idea of ​​the metamorphic relationship is to ensure that the recognition results of other unreplaced objects in images O and R remain consistent. Given sets of recognition bounding boxes of the removed and replaced objects for two corresponding scenes, the complete intersection-union ratio (CIU) between bounding box pairs in the two sets is calculated sequentially, resulting in a CIU matrix. The CIU is calculated as follows, taking into account three important geometric factors: intersection area, center point distance, and aspect ratio similarity.

[0062]

[0063] In this context, IoU measures the degree of overlap between two bounding boxes by calculating the ratio of their intersection area to their union area; 'b' represents the predicted bounding box, specifically a predicted bounding box in the newly generated scene. gt Represents the true bounding box, which in this instance represents a predicted bounding box on the original scene; ρ 2 represents the Euclidean distance between the center points of the two bounding boxes; c represents the diagonal distance between the smallest bounding rectangles of the two bounding boxes to be matched; in the above formula... v is used to measure the similarity of the aspect ratios of two bounding boxes, b and b'. gt Let the length and width be denoted as w, h and w respectively. gt ,h gt .

[0064] Step 4: Use the optimal matching method to screen out wrongly predicted bounding boxes from the complete intersection over union matrix, and accurately locate specific object detection errors by taking into account the geometric relationship, position, category, and intersection over union features between the predicted bounding boxes and the ground-truth bounding boxes.

[0065] In one implementation, after calculating the CIoU of all predicted bounding boxes relative to the ground-truth bounding boxes, a complete intersection over union matrix can be obtained. A higher CIoU value of two bounding boxes indicates a higher matching degree, which means that the two bounding boxes are more likely to be object detection results for the same object. Based on the calculated complete intersection over union matrix, the optimal matching principle is used to sequentially judge the result of each predicted bounding box: select the predicted bounding box with the maximum CIoU value after matching with the original bounding box from each row in the matrix, and determine whether it is a correct prediction according to the CIoU value and the prediction category. Before discriminating the specific prediction error type, two CIoU thresholds CIoUHigh and CIoULow need to be defined, and the two thresholds satisfy the magnitude relationship: 0<=CIoULow<CIoUHigh<=1. Check the CIoU value of the optimally matched predicted bounding box of a current original bounding box, and judge whether the prediction result is correct according to the predefined threshold. If the CIoU value of the two bounding boxes exceeds CIoUHigh, it is considered that the two bounding boxes are predictions made for the same object; if the CIoU value of the two bounding boxes is between CIoULow and CIoUHigh, it is considered that the current predicted bounding box is offset; if the CIoU value of the two bounding boxes is less than CIoULow, it is considered that the current object is not recognized in the new scene image.

[0066] In one implementation, according to the predefined CIoU threshold, the predicted bounding boxes with inconsistent recognition are classified into four different types of errors that violate the metamorphic relation, as shown in Figure 3 the figure.

[0067] The definitions of the four types of errors are as follows.

[0068] 1) Classification error: The prediction category of the optimally matched predicted bounding box is inconsistent with that of the ground-truth bounding box, but the CIoU value exceeds the threshold CIoUHigh.

[0069] 2) Localization error: The prediction category of the optimally matched predicted bounding box is consistent with that of the ground-truth bounding box, but the CIoU value is between CIoUHigh and CIoULow.

[0070] 3) Recognition error: Redundant predicted bounding boxes.

[0071] 4) Unrecognized error: A certain ground-truth bounding box has no matching predicted bounding box.

[0072] This embodiment also provides a test system for an autonomous driving target detection system based on object replacement, including:

[0073] One or more processors;

[0074] A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the object replacement-based autonomous driving target detection system test method.

[0075] The invention will be further illustrated by specific experiments below.

[0076] This experiment was implemented using Python, with OpenCV-Python 4.8.1 and Pillow 10.1.0 as image processing libraries. The target detection system under test used YOLOv5 and Detectron2 as the specific target detection model containers. The running machine environment consisted of an Intel Core i9-10940X CPU, 64GB of RAM, and a Quadro RTX 4000 graphics card, running Ubuntu 20.04. The Cityscapes dataset was selected as the autonomous driving dataset.

[0077] For the target detection system under test (e.g., YOLO), system configuration is required before testing. Pre-set the weight file for the target detection model, for example, by selecting an appropriate YOLO version (e.g., YOLOv5s, YOLOv5m, YOLOv5l, etc.) to ensure test repeatability and result consistency. Set uniform input image normalization size (e.g., 640*640), output confidence threshold, non-maximum suppression threshold, maximum number of detection bounding boxes, and other model inference parameters. Input the collected original driving scene images and generated scene images into the target detection system under test, and obtain the system's prediction output for the two image sets.

[0078] The experimental results are shown in Table 1. To verify the error-revealing ability of the test cases generated by this invention, five cities (Frankfurt, Munster, Bochum, Hanover, and Stuttgart) were randomly selected from Cityscapes as the original driving scenario dataset in the comparative experiment. Simultaneously, the existing metamorphic testing methods MetaOD and SemImFuzz were compared with the method proposed in this invention. Under the same conditions, test cases for the YOLOv5 object detection system (specifically, the YOLOv5s model) were generated using these three methods, and the results were evaluated using the evaluation metrics described in this invention (where CIoUHigh was set to 0.8 and CIoULow to 0.5). Table 1 shows the number of test cases and error ratios generated by the three methods. The error ratio is calculated as the ratio of the total number of errors to the total number of test cases. The average error revealing ratio of the SemImFuzz method was 0.62; the average error revealing ratio of the MetaOD method was 0.74; and the average error revealing ratio of this invention was 0.87. The test cases generated by this invention are superior to the other two methods in their error-revealing ability.

[0079] Table 1: Test results of the three methods for generating test cases on five city datasets.

[0080]

[0081] This invention defines a metamorphic relationship suitable for object detection systems in autonomous driving scenarios. Test cases are generated using an instance-level labeled dataset of original autonomous driving scenarios, and objects with similar shapes are used for replacement to simulate object changes in real-world autonomous driving environments. A Complete Intersection over Union (CIoU) matrix is ​​employed to evaluate the object detection system's recognition results, addressing the common object overlap problem in autonomous driving scenarios. This invention is particularly applicable to various complex driving scenarios. The test cases generated by this method require no additional manual annotation and are evaluated using a CIoU matrix, thus achieving more automated test result evaluation and reducing misjudgments.

[0082] It should be understood that the embodiments described above are only some, not all, of the embodiments of the present invention. Furthermore, the technical features of the various embodiments or individual embodiments provided by the present invention can be arbitrarily combined to form feasible technical solutions. Such combinations are not constrained by the order of steps and / or structural composition patterns, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0083] It should be understood that the above description of the preferred embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art, under the guidance of this invention, can make substitutions or modifications without departing from the scope of protection of the claims of this invention, and all such substitutions or modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A testing method for an autonomous driving target detection system based on object replacement, characterized in that, Includes the following steps: Step 1: Collect instance-level labeled raw autonomous driving scene data to construct a raw autonomous driving scene dataset; process the images in the raw autonomous driving scene dataset, using one part as the raw scene images and the other part as an object pool to extract source images for identifiable objects; use the raw scene images as input to the target detection system under test to obtain the target detection results on the raw scene images; Step 2: Define the transformation relationship of the target detection system in the autonomous driving scenario, and generate derived test cases from the original scene image based on object replacement. The derived test cases include the generated new scene image and the location record information of the replaced object. The object replacement-based method generates derived test cases from the original scene image. It is based on instance-level annotation to obtain the semantic segmentation mask of the original object in the original scene. By iterating through the extracted objects in the object pool, it compares the shape similarity between the original object and the extracted objects, and selects the object with a similarity higher than a threshold to replace the object in the original scene. After replacing a certain object in the original scene image, a derived test case is generated. The number of objects contained in the derived test case is the same as that in the original scene image, and the scene pixels are not destroyed. Only the pixels of the replaced object are changed to a matching object from the object pool. Step 3: The generated new scene image is used as input to the target detection system under test for processing to obtain the target detection results for the new scene. The cross-union matrix of the target detection results of the new scene and the target detection results of the original scene image is calculated according to the predefined metamorphic relationship to determine whether the target detection system has produced an unexpected erroneous output. Step 4: Use the optimal matching method to filter out the incorrect predicted bounding boxes from the complete intersection-union matrix, while considering the geometric relationship, position, category, and intersection-union features between the predicted bounding boxes and the ground truth bounding boxes, to accurately locate the specific target detection errors.

2. The testing method for an autonomous driving target detection system based on object replacement according to claim 1, characterized in that: In step 1, the objects in the object pool come from different driving scenarios, including different weather, lighting and city street scenery.

3. The testing method for an autonomous driving target detection system based on object replacement according to claim 1, characterized in that: In step 1, during the construction of the object pool, low-quality objects with an area smaller than the threshold are filtered out.

4. The testing method for an autonomous driving target detection system based on object replacement according to claim 1, characterized in that: In step 2, the metamorphic relationship is defined as follows: the detection result of a certain object in the scene is not affected by other objects, that is, the detection result of the object in the scene is context-independent; the metamorphic relationship is based on replacing objects with similar shapes to simulate object changes in real autonomous driving scenarios.

5. The testing method for an autonomous driving target detection system based on object replacement according to claim 1, characterized in that: The object matching stage calculates the shape similarity, which involves affine transformation of the objects to be matched, including mask scaling, translation and symmetry processing; the maximum semantic mask intersection-union ratio when the bounding boxes coincide after affine transformation is selected as the shape similarity of the two objects to be replaced.

6. The testing method for an autonomous driving target detection system based on object replacement according to claim 1, characterized in that, The perfect crossover ratio mentioned in step 3 is: ， in, The degree of overlap between two bounding boxes is measured by calculating the ratio of the intersection area to the union area. Indicates the predicted bounding box; Represents the truth bounding box; This represents the Euclidean distance between the center points of the two bounding boxes. c This represents the diagonal distance between the smallest bounding rectangles of the two bounding boxes to be matched; Used to measure the similarity of the aspect ratios of two bounding boxes. and The length and width are respectively denoted as , and , .

7. The test method for an autonomous driving target detection system based on object replacement according to any one of claims 1-6, characterized in that: In step 4, the optimal matching method is used to screen out incorrectly predicted bounding boxes from the complete intersection over union matrix. First, the predicted bounding boxes are evaluated in descending order of confidence, then the ground-truth bounding box with the largest complete intersection over union value with a certain ground-truth bounding box is selected for comparison. According to two predefined thresholds CIoULow and CIoUHigh, where 0 ≤ CIoULow < CIoUHigh ≤ 1, the error type of the predicted bounding box is determined; according to the error type of the predicted bounding box, the target detection error type is obtained, including classification error, localization error, recognition error or unrecognized error; The classification error means that the predicted category of the optimally matched predicted bounding box is inconsistent with that of the ground-truth bounding box, but the CIoU value exceeds the threshold CIoUHigh; The localization error means that the predicted category of the optimally matched predicted bounding box is consistent with that of the ground-truth bounding box, but the CIoU value is between CIoUHigh and CIoULow; The recognition error refers to a redundant predicted bounding box; The unrecognized error means that a certain ground-truth bounding box has no matched predicted bounding box.

8. A test system for an autonomous driving target detection system based on object replacement, characterized in that, Comprising: One or more processors; A storage device, configured to store one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the object replacement-based autonomous driving object detection system testing method according to any one of claims 1 to 7.

Images