Automatic driving perception system test method, system, equipment and medium

Abstract

The invention provides an automatic driving perception system test method, system, device and medium, and the method comprises the steps: obtaining a plurality of groups of comparison images, each group of comparison images comprising a pure image and a polluted image in the same scene; for each group of comparison images, extracting a pollution centroid set corresponding to the pollution image according to the pollution image and the pure image; reversely generating a pollution shape according to the pollution centroid set to obtain a simulated pollution grey-scale map, and then adding pollution intensity to the simulated pollution grey-scale map to obtain a progressive pollution image; and constructing a sensor pollution sensing model according to the progressive pollution pixel set, and then testing the automatic driving sensing system by using the sensor pollution sensing model to obtain the maximum driving mileage of the sensing system failure. According to the invention, the problem that the performance evaluation of the automatic driving perception system is inaccurate due to the fact that the change of the perception ability of the automatic driving perception system cannot be evaluated in the gradual pollution process of the camera in the prior art is solved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving system testing technology, and in particular to a method, system, equipment and medium for testing autonomous driving perception systems. Background Technology

[0002] Cameras are fundamental to autonomous driving perception systems. In the real world, the performance of vehicle cameras slowly but steadily declines over time. For example, camera lenses gradually become blurry due to the long-term accumulation of mud, water, and dust. This progressive contamination of cameras poses a continuous challenge to autonomous driving perception systems.

[0003] However, the industry generally lacks a systematic and quantifiable method to simulate and test progressive camera contamination. Existing simulation tests mostly use ideal sensor models or only employ simple transient fault injections (such as completely blocking the camera), which cannot assess the changes in the perception capabilities of autonomous driving perception systems as the camera gradually becomes contaminated. This leads to overestimation or underestimation of the performance of autonomous driving perception systems, which does not reflect the actual performance changes of autonomous driving perception systems in real-world use. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a testing method, system, device, and medium for autonomous driving perception systems. It solves the problem in existing technologies that it is impossible to assess changes in the perception capabilities of autonomous driving perception systems as cameras become gradually contaminated, leading to inaccurate performance evaluations of autonomous driving perception systems.

[0005] According to an embodiment of the present invention, a testing method for an autonomous driving perception system includes: Acquire multiple sets of comparison images, where each set includes a clean image and a contaminated image of the same scene; For each set of comparison images, extract the pollution centroid set corresponding to the pollution image based on the pollution image and the clean image; The pollution shape is generated in reverse from the pollution centroid set to obtain a simulated pollution grayscale image. Then, the pollution intensity is added to the simulated pollution grayscale image to obtain a progressive pollution image. A sensor pollution perception model is constructed based on a progressive pollution image set. Then, the sensor pollution perception model is used to test the autonomous driving perception system to obtain the maximum driving distance when the perception system fails.

[0006] Preferably, the method for extracting the contaminated centroid set corresponding to each contaminated image based on the contaminated image and the clean image includes: Scale all contaminated and clean images to the same scale. For each set of comparison images, the contaminated image and the clean image are compared, and the contaminated pixels in the contaminated image are extracted. The k-means algorithm is used to cluster contaminated pixels to obtain multiple point clusters; Calculate the centroid of each point cluster, and use the coordinates of the pixels located at the centroid as the coordinates of the corresponding centroid. Then combine all centroids into a contaminated centroid set.

[0007] Preferably, the method for generating the pollution shape from the pollution centroid set to obtain the simulated pollution grayscale image includes: Multiple centroids are randomly selected from the set of contaminated centroids, and the spot shape of each centroid is simulated using a Gaussian function to obtain the corresponding contamination shape; By overlaying all the pollution shapes, a simulated pollution grayscale image is obtained.

[0008] Preferably, each set of comparison images also includes the mileage traveled during image acquisition; Methods for obtaining progressively polluted images by adding pollution intensity to simulated pollution grayscale images include: Methods for obtaining progressively polluted images by adding pollution intensity to simulated pollution grayscale images include: Calculate the pollution intensity in the polluted image within each set of comparison images; Based on the pollution intensity and driving mileage corresponding to all the comparison images, a pollution intensity function is constructed using the least squares method. A pollution weight matrix is ​​generated based on the pollution intensity function and the grayscale value of each pixel in the simulated pollution grayscale image; Choose a pollution color, and based on the RGB values ​​of the pollution color and the pollution weight matrix, overlay each pixel of the clean image to obtain the corresponding progressive pollution image.

[0009] Preferably, the formula for calculating the pollution intensity is as follows: Where d is the driving mileage. The base mileage represents the inflection point of the S-curve, and k is the pollution accumulation rate. This is a correction factor.

[0010] Preferably, the formula for generating the progressive contamination image is as follows: in, The pollution weight at pixel coordinates (i, j) , , Let be the RGB values ​​of the pixel at coordinates (i, j) in the clean image. , , , which are the RGB values ​​of the pixel at coordinates (i, j) in the progressively polluted image.

[0011] Preferably, the method for testing the autonomous driving perception system using a sensor pollution perception model to obtain the maximum driving distance due to perception system failure includes: Multiple driving distances are set, and multiple progressive pollution images are generated using a sensor pollution perception model; The autonomous driving perception system was tested using all progressively contaminated images, and the corresponding test results were obtained. Based on all test results, the perceived precision and recall rate are calculated. Then, based on the precision, recall rate and the corresponding driving mileage, a B-spline curve is fitted to obtain the performance change curve. The driving mileage corresponding to the preset performance threshold in the performance change curve is taken as the maximum driving mileage for the perception system failure.

[0012] On the other hand, according to embodiments of the present invention, an autonomous driving perception system testing system is also provided, which uses the above-described autonomous driving perception system testing method, including: An image processing module is used to acquire multiple sets of comparison images and extract the pollution centroid set corresponding to the pollution image based on the pollution image and the clean image in the comparison images; The inversion module is used to generate the pollution shape in reverse from the pollution centroid set to obtain a simulated pollution grayscale image, and then add pollution intensity to the simulated pollution grayscale image to obtain a progressive pollution image. A model building module is used to build a sensor pollution perception model based on a progressively polluted pixel set. The testing module is used to test the autonomous driving perception system using a sensor contamination perception model to obtain the maximum driving distance when the perception system fails.

[0013] On the other hand, according to an embodiment of the present invention, a computer is also provided, including at least one processor and a memory, the memory storing a computer program configured to be executed by the processor to implement the above-described method for testing an autonomous driving perception system.

[0014] On the other hand, according to an embodiment of the present invention, a storage medium is also provided, which is a computer-readable storage medium storing a computer program that can be executed by one or more processors to implement the above-described method for testing an autonomous driving perception system.

[0015] Compared with the prior art, the present invention has the following beneficial effects: This invention extracts the centroid set of pollution from clean and polluted images in the same scene. Then, it generates various pollution shapes based on the centroid set and adds pollution intensity to simulate the change in the camera's pollution area as the vehicle's mileage increases. This generates pollution images with different shapes and degrees of pollution, thus comprehensively simulating the state changes of the camera during the gradual pollution process. These pollution images are then used to test the autonomous driving perception system, improving the accuracy of the evaluation results and conforming to the performance changes of the autonomous driving perception system in actual use. Attached Figure Description

[0016] Figure 1 This is a diagram illustrating a test method for an autonomous driving perception system according to an embodiment of the present invention.

[0017] Figure 2 This is a pollution image obtained by a test camera in an embodiment of the present invention.

[0018] Figure 3 This is a clean image obtained by the auxiliary camera in an embodiment of the present invention.

[0019] Figure 4 This is a grayscale image of simulated pollution in an embodiment of the present invention.

[0020] Figure 5 This is a progressive contamination image according to an embodiment of the present invention.

[0021] Figure 6 This is a performance curve showing the relationship between accuracy and mileage in an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0023] like Figure 1 As shown in the figure, an embodiment of the present invention proposes a testing method for an autonomous driving perception system, including: Acquire multiple sets of comparison images, where each set includes a clean image and a contaminated image of the same scene; A test camera was mounted on the test vehicle, along with an auxiliary camera. The two cameras were positioned very close to each other to ensure their shooting range and angle of view were essentially the same. The auxiliary camera maintained a clear lens at all times using its built-in cleaning device, and its images were used as clean images. The test camera was exposed to a natural driving environment. The test vehicle underwent long-term road testing under typical driving conditions, and image sequences were simultaneously acquired from both the test camera and the auxiliary camera, such as... Figure 2 and Figure 3 As shown.

[0024] For each set of comparison images, extract the pollution centroid set corresponding to the pollution image based on the pollution image and the clean image; For each contaminated image captured by the test camera, it is compared with the image acquired by the auxiliary camera. By comparing the contaminated and clean images pixel by pixel, the contaminated areas in each image captured by the test camera are extracted. ,in Indicates the contaminated pixel. Indicates the image number.

[0025] For each extracted contaminated area Treating all its pixels as a two-dimensional point cloud, and constructing a two-dimensional coordinate system for the two-dimensional point cloud. This allows us to represent the coordinates of a pixel.

[0026] Typically, contamination on a camera spreads outward from a central point, eventually forming a circular, spot-like contamination area. Therefore, by finding this central point, the shape of the contamination area can be simulated. Thus, this invention applies the K-means clustering algorithm to divide all pixels into K clusters to identify the main contamination centroids.

[0027] The goal of the algorithm is to find K contamination centroids. That is, minimizing the objective function: in, For the image corresponding to the first The coordinates of the centroid.

[0028] For all camera surface contamination images acquired from real-vehicle tests, identifying the contamination centroids yields the contamination centroid for each image. The coordinates of the pixels located at the centroid are then used as the coordinates of the corresponding centroid. Finally, all contamination centroids are combined to form a contamination centroid set. This forms a centroid library, enabling the extraction of features from the spatial shape of the polluted area.

[0029] The pollution shape is generated in reverse from the pollution centroid set to obtain a simulated pollution grayscale image. Then, the pollution intensity is added to the simulated pollution grayscale image to obtain a progressive pollution image. Because the shape of the contamination from the camera is affected by the external environment and is therefore uncontrollable, it generally diffuses outward from a center of mass, eventually forming a circular, spot-like contamination area. Therefore, this invention randomly selects contamination from the contamination center of mass. Multiple contamination centroids are selected to generate contamination shapes in reverse. The number and objects of the contamination centroids are randomized but not repeated during selection. This ensures that the final generated contamination shape has sufficient randomness to cover various camera contamination scenarios.

[0030] For each selected pollution centroid The size (x, y) of the final generated contamination shape is determined, and the contamination shape of the centroid is simulated using a Gaussian function, as follows: in, For the first The parameters for each spot are used to control the size of the contamination shape and can be set manually.

[0031] (x, y) can be understood as the coordinates of each pixel in the polluted shape. According to the above formula, the degree of pollution of each pixel in the polluted shape can be calculated and converted into a gray value.

[0032] The pollution shape corresponding to the centroid of all pollution By overlaying the images, a complete simulated pollution grayscale image can be obtained. : Since no color is involved at this time, therefore Essentially, it's a grayscale image. Each pixel within each contaminated shape corresponds to a degree of contamination. By adding these contamination levels together, the resulting simulated contaminated grayscale image shows the contamination levels of each pixel superimposed. The higher the contamination level, the smaller its grayscale value. If a value of 0 is found at a pixel, it indicates that the pixel is uncontaminated, and the pixel's color will change to black; conversely, if a value of 0 is found at a pixel, the pixel's color will change to black. A value of 255 at a given pixel indicates a very high level of contamination, and that pixel will be white. For example... Figure 4 As shown, the diagram contains two pollution shapes: black represents unpolluted areas, and white represents polluted areas.

[0033] Simulated pollution grayscale image In the simulation, the shape and grayscale value of the pollution were determined. However, since the grayscale value was fixed, it could not reflect the trend of the pollution level becoming heavier as the vehicle's mileage increased. Therefore, it is necessary to apply a pollution intensity to the simulated pollution grayscale map to simulate the change of the pollution grayscale map with mileage.

[0034] Based on the previously obtained comparison images, the pollution intensity can be calculated for each set of comparison images by comparing the polluted image with the clean image: in, These represent the width and height of the contaminated image (the clean image), respectively. To contaminate the image at the pixel level Brightness at that location; For a clean image at the pixel level The brightness at that location. Pollution intensity.

[0035] By combining the vehicle's mileage during image acquisition with the comparison images, a set of two-dimensional data can be obtained. ,in, Let represent the mileage traveled in the i-th set of comparison images. Let represent the pollution intensity of the i-th set of comparison images.

[0036] Historical data shows that pollution intensity As the mileage increases The growth of contamination exhibits the following S-shaped trend: In the initial stage, the lens surface is clean, making it difficult for contaminants to adhere, and the contamination rate is slow; in the middle stage, as a small amount of contaminants adhere to the lens surface, the original smooth surface is damaged, making it easier for subsequent contaminants to adhere, and the contamination rate increases; in the later stage, the contaminated area on the lens surface tends to be saturated, and the contamination rate slows down. Therefore, the contamination intensity... and mileage The pollution intensity function should be a classic mathematical model describing this S-shaped growth process characterized by slow initial growth, accelerated growth in the middle stage, and saturation in the later stage. The classic Logistic function can be considered, and the final pollution intensity function is as follows: Where d is the driving mileage. The base mileage represents the inflection point of the S-curve, and k is the pollution accumulation rate. This is a correction factor.

[0037] Then, based on the two-dimensional data groups corresponding to each set of comparison objects To fit the above pollution intensity, Substitute into d, Substitute By combining historical experience with the least squares method to solve for the unknown parameters k and L, the complete pollution intensity function can be obtained.

[0038] By applying the pollution intensity function to the simulated pollution grayscale image, the grayscale value of each pixel in the simulated pollution grayscale image can be scaled to reflect the degree of pollution. Thus, different pollution shapes can be simulated based on the pollution centroid selected when generating the simulated pollution grayscale image, and different pollution levels (pollution area size) that change with driving mileage can be simulated based on the pollution intensity.

[0039] The grayscale value is amplified / reduced using a contamination intensity function, and the contamination weight at any pixel is obtained through normalization, calculated using the following equation: in, Let (i, j) be the pollution weight at the pixel with coordinates (i, j). To simulate the grayscale value of a pixel with coordinates (i, j) in a contaminated grayscale image, p(d) represents the contamination intensity. Thus, a contamination weight matrix can be generated. In the contamination weight matrix, the position of each matrix element corresponds to the coordinates of each pixel, and the value of the matrix element corresponds to the contamination weight of the pixel.

[0040] Let the RGB values ​​of the clean image at the pixel with coordinates (i, j) be respectively , , Then, a pollution color is randomly selected from the color set of common pollutants (such as dust and mud). Let the RGB values ​​of this pollution color be... , , Based on the pollution weights of all pixels obtained above, the pollution is superimposed onto the clean image, as shown in the following formula.

[0041] Taking dust pollutants as an example, the image after pollution is as follows: Figure 5 As shown.

[0042] A sensor contamination perception model is constructed based on a progressively contaminated pixel set. Then, the sensor contamination perception model is used to test the autonomous driving perception system to obtain the maximum driving distance when the perception system fails.

[0043] Progressive contamination images were used as test samples to construct a sensor contamination perception model. Different driving distances were defined when testing the autonomous driving perception system. Starting from 1,000 kilometers and increasing in 1,000-kilometer increments up to 100,000 kilometers, the sensor pollution perception model generates progressive pollution images corresponding to the driving mileage. For each driving mileage, multiple progressive pollution images with different pollution shapes are generated.

[0044] Sensor pollution perception model based on different driving mileage The progressively polluted images are sequentially input into the autonomous driving perception system for testing, and the corresponding test results are obtained. The test results are: whether the camera can identify whether the pollution completely blocks the autonomous driving perception system from perceiving the environment.

[0045] After the progressive pollution image is generated, it is judged by humans (e.g., by expert review or historical experience) whether the progressive pollution image can identify whether the pollution of the camera completely blocks the autonomous driving perception system from perceiving the environment, and the result is taken as the truth value of the corresponding progressive pollution image.

[0046] All test results were compared with the corresponding true values ​​to statistically analyze the performance of the perception system at different driving mileages, using the following indicators: in, and Precision and recall, respectively. For a true example, that is, the number of targets correctly detected by the autonomous driving perception system; A false positive is the number of other elements that the autonomous driving perception system identifies as targets. This is a false counterexample, representing the number of targets that the autonomous driving perception system failed to detect. As pollution worsens, the image becomes blurry. It shows a downward trend. and It shows an upward trend.

[0047] A series of discrete performance evaluation results and The B-spline curve method was used for fitting to generate two curves that characterize the system performance as the driving mileage increases. A continuously varying performance curve, taking accuracy as an example, is as follows: Figure 6 As shown.

[0048] Set a minimum acceptable performance standard for an autonomous driving perception system. and According to the two performance curves, as the mileage increases, when either performance curve first intersects with the corresponding minimum performance standard, the corresponding horizontal axis mileage is the maximum mileage at which the precise sensing system fails. The study clarified the maximum driving range at which an autonomous driving perception system can maintain reliable operation under continuous camera contamination, providing direct quantitative basis for the maintenance strategy (cleaning, replacement) and functional safety design of the autonomous driving perception system.

[0049] On the other hand, embodiments of the present invention also provide an autonomous driving perception system testing system, which uses the above-described autonomous driving perception system testing method, including: An image processing module is used to acquire multiple sets of comparison images and extract the pollution centroid set corresponding to the pollution image based on the pollution image and the clean image in the comparison images; The inversion module is used to generate the pollution shape in reverse from the pollution centroid set to obtain a simulated pollution grayscale image, and then add pollution intensity to the simulated pollution grayscale image to obtain a progressive pollution image. A model building module is used to build a sensor pollution perception model based on a progressively polluted pixel set. The testing module is used to test the autonomous driving perception system using a sensor contamination perception model to obtain the maximum driving distance when the perception system fails.

[0050] On the other hand, embodiments of the present invention also provide a computer, including at least one processor and a memory, the memory storing a computer program configured to be executed by the processor to implement the above-described method for testing an autonomous driving perception system.

[0051] On the other hand, embodiments of the present invention also provide a storage medium, which is a computer-readable storage medium, and stores a computer program on the storage medium. The computer program can be executed by one or more processors to implement the above-described method for testing an autonomous driving perception system.

[0052] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A testing method for an autonomous driving perception system, characterized in that: include: Acquire multiple sets of comparison images, where each set includes a clean image and a contaminated image of the same scene; For each set of comparison images, extract the pollution centroid set corresponding to the pollution image based on the pollution image and the clean image; The pollution shape is generated in reverse from the pollution centroid set to obtain a simulated pollution grayscale image. Then, the pollution intensity is added to the simulated pollution grayscale image to obtain a progressive pollution image. A sensor pollution perception model is constructed based on progressive pollution images. Then, the sensor pollution perception model is used to test the autonomous driving perception system to obtain the maximum driving distance when the perception system fails.

2. The testing method for an autonomous driving perception system as described in claim 1, characterized in that: Methods for extracting the contaminated centroid set corresponding to each contaminated image based on contaminated and clean images include: Scale all contaminated and clean images to the same scale. For each set of comparison images, the contaminated image and the clean image are compared, and the contaminated pixels in the contaminated image are extracted. The k-means algorithm is used to cluster contaminated pixels to obtain multiple point clusters; Calculate the centroid of each point cluster, and use the coordinates of the pixels located at the centroid as the coordinates of the corresponding centroid. Then combine all centroids into a contaminated centroid set.

3. The testing method for an autonomous driving perception system as described in claim 1, characterized in that: Methods for generating pollution shapes from the pollution centroid set to obtain simulated pollution grayscale images include: Multiple centroids are randomly selected from the set of contaminated centroids, and the spot shape of each centroid is simulated using a Gaussian function to obtain the corresponding contamination shape; By overlaying all the pollution shapes, a simulated pollution grayscale image is obtained.

4. The testing method for an autonomous driving perception system as described in claim 1, characterized in that: Each set of comparison images also includes the mileage traveled at the time the images were acquired; Methods for obtaining progressively polluted images by adding pollution intensity to simulated pollution grayscale images include: Calculate the pollution intensity in the polluted image within each set of comparison images; Based on the pollution intensity and driving mileage corresponding to all the comparison images, a pollution intensity function is constructed using the least squares method. A pollution weight matrix is ​​generated based on the pollution intensity function and the grayscale value of each pixel in the simulated pollution grayscale image; Choose a pollution color, and based on the RGB values ​​of the pollution color and the pollution weight matrix, overlay each pixel of the clean image to obtain the corresponding progressive pollution image.

5. The testing method for an autonomous driving perception system as described in claim 4, characterized in that: The formula for calculating the pollution intensity is as follows: Where d is the driving mileage. The base mileage represents the inflection point of the S-curve, and k is the pollution accumulation rate. This is a correction factor.

6. The testing method for an autonomous driving perception system as described in claim 4, characterized in that: The formula for generating a progressive contamination image is as follows: in, The pollution weight at pixel coordinates (i, j) , , Let be the RGB values ​​of the pixel at coordinates (i, j) in the clean image. , , , which are the RGB values ​​of the pixel at coordinates (i, j) in the progressively polluted image.

7. The testing method for an autonomous driving perception system as described in claim 1, characterized in that: Methods for testing autonomous driving perception systems using sensor-based pollution perception models to determine the maximum driving distance during perception system failure include: Multiple driving distances are set, and multiple progressive pollution images are generated using a sensor pollution perception model; The autonomous driving perception system was tested using all progressively contaminated images, and the corresponding test results were obtained. Based on all test results, the perception precision rate and recall rate are calculated. Then, based on the perception precision rate, recall rate and corresponding driving mileage, a B-spline curve is fitted to obtain the performance change curve. The driving mileage corresponding to the preset performance threshold in the performance change curve is taken as the maximum driving mileage for the perception system failure.

8. An autonomous driving perception system testing system, characterized in that: The system uses an autonomous driving perception system testing method as described in any one of claims 1-7, comprising: An image processing module is used to acquire multiple sets of comparison images and extract the pollution centroid set corresponding to the pollution image based on the pollution image and the clean image in the comparison images; The inversion module is used to generate the pollution shape in reverse from the pollution centroid set to obtain a simulated pollution grayscale image, and then add pollution intensity to the simulated pollution grayscale image to obtain a progressive pollution image. A model building module is used to build a sensor pollution perception model based on progressive pollution images; The testing module is used to test the autonomous driving perception system using a sensor contamination perception model to obtain the maximum driving distance when the perception system fails.

9. A computer, characterized in that: It includes at least one processor and a memory, the memory storing a computer program configured to be executed by the processor to implement the autonomous driving perception system test method according to any one of claims 1-7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, on which a computer program is stored. The computer program can be executed by one or more processors to implement a test method for an autonomous driving perception system as described in any one of claims 1-7.

Images