Precision detection method and device for vehicle-mounted radar sensing model, and storage medium

Abstract

The invention relates to a precision detection method and device for a vehicle-mounted radar sensing model and a storage medium. The method comprises the following steps: acquiring three-dimensional point cloud data based on a vehicle-mounted radar, and determining a real three-dimensional bounding box of a real target object in the three-dimensional point cloud data; obtaining a color image matched with the three-dimensional point cloud data, and marking the color of the real three-dimensional bounding box based on the color image; inputting the three-dimensional point cloud data into a to-be-detected target sensing model, and outputting a predicted three-dimensional bounding box for sensing a target object; and determining a difference value between the predicted three-dimensional bounding box and the real three-dimensional bounding box marked with the color based on a preset algorithm, and determining the detection precision of the to-be-detected target sensing model on real target objects with different colors based on the difference value. According to the invention, the sensing precision of the laser radar on the target can be detected more accurately and reliably from the target sensing model level.

Description

Technical Field

[0001] The present application relates to the field of target perception technology, and in particular to an accuracy detection method, device and storage medium for a vehicle-mounted radar perception model. Background Art

[0002] Autonomous driving technology is a hot topic in academia and industry. The autonomous driving algorithm architecture is mainly divided into three modules: "perception", "decision-making" and "execution". The "perception" module obtains data through sensors such as cameras and lidars, and obtains information such as the location, size and speed of traffic participants by perceiving the surrounding environment of the vehicle. Since lidar achieves perception tasks by measuring the time it takes for a laser pulse to be emitted and received, and recording the intensity of the pulse reflection, objects of different colors have different reflectivities, and the degree of absorption and reflection of the pulse is also different, which will affect the perception function of the lidar. For example, black objects have a lower reflectivity, so they may reflect fewer laser pulses, resulting in problems such as "dropping points" in the point cloud. White objects usually have a higher reflectivity, so more light can be reflected back to the lidar.

[0003] In traditional LiDAR target perception accuracy detection technology, accuracy detection is usually performed at the data level, such as detecting the target perception accuracy of LiDAR by analyzing the number of points in the point cloud. However, on the one hand, traditional technology only uses a small number of object samples for testing, and the point cloud has less effective information; on the other hand, traditional technology does not consider the possible impact of color on the target perception model. Therefore, the accuracy and reliability of LiDAR target perception accuracy detection in traditional technology are low. Summary of the invention

[0004] Based on this, it is necessary to provide an accuracy detection method, device and storage medium for a vehicle-mounted radar perception model that can improve detection accuracy and detection reliability in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for detecting the accuracy of a vehicle-mounted radar perception model. The method comprises:

[0006] Acquire three-dimensional point cloud data based on the vehicle-mounted radar, and determine a true three-dimensional bounding box of a real target object in the three-dimensional point cloud data;

[0007] Acquire a color image that matches the three-dimensional point cloud data, and annotate the color of the real three-dimensional bounding box based on the color image;

[0008] Inputting the three-dimensional point cloud data into a target perception model to be detected, and outputting a predicted three-dimensional bounding box of the perceived target object;

[0009] The difference between the predicted three-dimensional bounding box and the real three-dimensional bounding box marked with color is determined based on a preset algorithm, and the detection accuracy of the target perception model to be detected for real target objects of different colors is determined based on the difference.

[0010] In one embodiment, the step of marking the color of the real three-dimensional bounding box based on the color image comprises:

[0011] Projecting the real three-dimensional bounding box onto the color image to obtain a projected bounding box;

[0012] Determining color information of the real target object based on pixel values ​​within the projection bounding box;

[0013] The real three-dimensional bounding box is labeled with a color based on the color information.

[0014] In one of the embodiments, the target perception model to be detected is determined based on an original supervised target perception model; The The 3D point cloud data is input into the target perception model to be detected, and the predicted 3D bounding box of the perceived target object is output, including:

[0015] Annotating the three-dimensional point cloud data with color based on the color information;

[0016] Determine training data based on the color-annotated three-dimensional point cloud data and the color-annotated real three-dimensional bounding box, where the training data corresponds to color information of a single category;

[0017] Training an initial model based on the training data to obtain the to-be-detected target perception model, wherein the initial model has the same model structure as the original supervised target perception model;

[0018] The test three-dimensional point cloud data is input into the to-be-detected target perception model, and the predicted three-dimensional bounding box is obtained as an output, wherein the color information category of the test three-dimensional point cloud data is different from that of the training data.

[0019] In one embodiment, the preset algorithm includes a KITTI dataset evaluation algorithm, and determining the difference between the predicted three-dimensional bounding box and the real three-dimensional bounding box after color annotation based on the preset algorithm includes:

[0020] Inputting the test three-dimensional point cloud data into the target perception model to be detected, and also outputting the prediction confidence matching the predicted three-dimensional bounding box;

[0021] The predicted three-dimensional bounding box and the predicted confidence are input into the KITTI dataset evaluation algorithm to obtain an average precision value as the difference value.

[0022] In one embodiment, The The predicted three-dimensional bounding box and the predicted confidence are input into the KITTI data set evaluation algorithm to obtain an average precision value as the difference value, including:

[0023] Determining a target perception difficulty gradient of the training data based on the KITTI dataset evaluation algorithm;

[0024] Determine a bird's-eye view average precision value and / or a three-dimensional average precision value under the target perception difficulty gradient.

[0025] In one embodiment, the target perception model to be detected includes an original unsupervised target perception model, the preset algorithm includes an intersection-over-union detection algorithm; and determining the difference between the predicted three-dimensional bounding box and the real three-dimensional bounding box after color annotation based on the preset algorithm includes:

[0026] Based on the intersection-over-union detection algorithm, a bird's-eye view intersection-over-union ratio between the predicted three-dimensional bounding box and the real three-dimensional bounding box after the color marking is determined as the difference value.

[0027] In one of the embodiments, after determining the detection accuracy of the target perception model to be detected for real target objects of different colors based on the difference value, the method further includes:

[0028] If the difference in detection accuracy of the target perception model to be detected for real target objects of different colors is greater than a preset accuracy difference threshold, determining the target three-dimensional point cloud data contained in the real three-dimensional bounding box of each color category;

[0029] Determine a target point cloud reflectivity and a target point cloud density value of target three-dimensional point cloud data of each color category;

[0030] Determine the reflectivity difference of the target point cloud of different color categories and the density value difference of the target point cloud of different color categories.

[0031] In a second aspect, the present application also provides an accuracy detection device for a vehicle-mounted radar perception model.

[0032] The device comprises:

[0033] A data acquisition module, used to acquire three-dimensional point cloud data based on the vehicle-mounted radar, and determine a real three-dimensional bounding box of a real target object in the three-dimensional point cloud data;

[0034] A color labeling module, used to obtain a color image matching the three-dimensional point cloud data, and label the color of the real three-dimensional bounding box based on the color image;

[0035] A prediction module, used for inputting the three-dimensional point cloud data into a perception model of a target to be detected, and outputting a predicted three-dimensional bounding box of the perceived target object;

[0036] The detection accuracy determination module is used to determine the difference between the predicted three-dimensional bounding box and the real three-dimensional bounding box after color marking based on a preset algorithm, and determine the detection accuracy of the target perception model to be detected for real target objects of different colors based on the difference.

[0037] In a third aspect, the present application further provides a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements any step of the accuracy detection method for the vehicle-mounted radar perception model in the first aspect.

[0038] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any step of the accuracy detection method for a vehicle-mounted radar perception model in the first aspect.

[0039] The above-mentioned accuracy detection method, device and storage medium for the vehicle-mounted radar perception model annotates the color of the real three-dimensional bounding box of the real target object through the color image matching the three-dimensional point cloud data, then inputs the three-dimensional point cloud data into the target perception model to be detected, and outputs the predicted three-dimensional bounding box. Then, the difference value between the predicted bounding box and the real bounding box after the color is annotated is calculated through a preset algorithm, which not only considers the information of the three-dimensional point cloud data itself, but also incorporates the color information into the reference dimension. Based on the difference value, the detection accuracy of the model for real target objects of different colors can be accurately judged, and the perception accuracy of the laser radar for the target can be more accurately and reliably detected from the target perception model level. Furthermore, by determining the detection accuracy of the target perception model to be detected for real target objects of different colors, it is also possible to detect the impact of color on the perception ability of the laser radar, providing a solid detection foundation for improving the reliability of autonomous driving technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0041] Figure 1 is an application environment diagram of an accuracy detection method for a vehicle-mounted radar perception model in one embodiment;

[0042] Figure 2 is a flow chart of a method for detecting the accuracy of a vehicle-mounted radar perception model in one embodiment;

[0043] Figure 3 It is a flowchart of a method for detecting the accuracy of a vehicle-mounted radar perception model in a specific embodiment;

[0044] Figure 4 is a structural block diagram of an accuracy detection device for a vehicle-mounted radar perception model in one embodiment;

[0045] Figure 5 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0047] The terms "module", "unit", etc. used below are a combination of software and / or hardware that can implement a predetermined function. Although the devices described in the following embodiments are preferably implemented in hardware, the implementation of software, or a combination of software and hardware is also possible and conceivable.

[0048] The accuracy method for the vehicle-mounted radar perception model provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The terminal 102 obtains the three-dimensional point cloud data based on the vehicle-mounted radar and sends it to the server 104. The server 104 determines the real three-dimensional bounding box of the real target object in the three-dimensional point cloud data; obtains a color image matching the three-dimensional point cloud data, and annotates the color of the real three-dimensional bounding box based on the color image; inputs the three-dimensional point cloud data into the target perception model to be detected, and outputs the predicted three-dimensional bounding box of the perceived target object; determines the difference between the predicted three-dimensional bounding box and the real three-dimensional bounding box after the color is annotated based on a preset algorithm, and determines the detection accuracy of the target perception model to be detected for real target objects of different colors based on the difference value. The above steps can also be performed by the server 104 or the terminal 102 on a single end, and this application does not limit this. Among them, the server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0049] In some embodiments, Figure 2 As shown in FIG. 1 , a method for detecting the accuracy of a vehicle-mounted radar perception model is provided. The method is applied to Figure 1The application scenario in is taken as an example to illustrate, including the following steps:

[0050] S201: Acquire three-dimensional point cloud data based on the vehicle-mounted radar, and determine a real three-dimensional bounding box of a real target object in the three-dimensional point cloud data.

[0051] In the embodiment of the present application, the vehicle-mounted radar includes a sensor device for detecting target objects in the vehicle's surrounding environment, and senses the speed, position, or relative distance of the target object from the vehicle by emitting electromagnetic waves and receiving reflected waves. The vehicle-mounted radar may be a laser radar, etc. The data acquired by the vehicle-mounted radar usually exists in the form of three-dimensional point cloud data, which includes a data set composed of multiple three-dimensional points, and each three-dimensional point may include X-axis, Y-axis, and Z-axis coordinate information, which is used to represent the position of the target object in space.

[0052] The real target object described in the embodiment of the present application includes objects existing in the actual scene sensed by the vehicle-mounted radar, such as vehicles, pedestrians, obstacles, etc. The real target object is represented as a set of spatial points in the three-dimensional point cloud data. The real three-dimensional bounding box of the real target object in the three-dimensional point cloud data includes a three-dimensional rectangular box used to represent the spatial position and boundary of the object in the three-dimensional point cloud data, which can be understood as the circumscribed rectangular box corresponding to the three-dimensional point of the real target object.

[0053] In some embodiments, the real 3D bounding box can be determined based on a semi-supervised perception model or an unsupervised perception model in traditional technology, or by manual annotation, or by manual correction based on a perception model in traditional technology. This application does not specifically limit the method for determining the real 3D bounding box.

[0054] S203: Acquire a color image that matches the three-dimensional point cloud data, and annotate the real three-dimensional bounding box with color based on the color image.

[0055] In the embodiment of the present application, a color image can be obtained by a camera device installed on the vehicle. The camera device simultaneously obtains the color image when the vehicle-mounted radar obtains the three-dimensional point cloud data. The matched three-dimensional point cloud data and the color image correspond to the same actual scene. For the real target object in the three-dimensional point cloud data, it can correspond to the matched color image and mark the color of the real three-dimensional bounding box.

[0056] In some embodiments, the step of marking the color of the real three-dimensional bounding box based on the color image comprises:

[0057] S301: Project the real three-dimensional bounding box onto the color image to obtain a projected bounding box.

[0058] S303: Determine color information of the real target object based on pixel values ​​within the projected bounding box.

[0059] S305: Label the real three-dimensional bounding box with color based on the color information.

[0060] In the embodiment of the present application, the real three-dimensional bounding box is projected onto the color image to obtain a projected bounding box. Specifically, the projected bounding box of the real target object in the color image can be obtained by projecting the length, width, height and position information of the three-dimensional bounding box onto the matching color image. The projected bounding box can be an external rectangular frame of the real target object.

[0061] Based on the pixel values ​​in the projected bounding box, the color information of the real target object can be determined. Specifically, the color information of the real target object can be determined based on the average value of the pixel RGB values ​​in the projected bounding box. The color information can also be manually corrected. After the color information is determined, the color of the real three-dimensional bounding box is annotated. Specifically, the standard color information may include black, white, gray, blue, red, etc. The real three-dimensional bounding boxes corresponding to the real target objects of the same type but different colors (such as vehicles) are annotated with color information, and the real three-dimensional bounding boxes can be classified according to the color information.

[0062] S205: Input the three-dimensional point cloud data into a perception model of the target to be detected, and output a predicted three-dimensional bounding box of the perceived target object.

[0063] In the embodiment of the present application, the three-dimensional point cloud data is used as input data, and the target perception model to be detected is input, and the predicted three-dimensional bounding box of the perceived target object is output. The target perception model to be detected is used to detect objects in the three-dimensional point cloud data, that is, after the three-dimensional point cloud data is collected by the vehicle-mounted radar, it is input into the target perception model to be detected, and the detected object is output to realize the object perception in the vehicle driving scene.

[0064] In some embodiments, the target perception model to be detected may include an original supervised target perception model or an original unsupervised target perception model. Specifically, the target perception model to be detected may analyze the input three-dimensional point cloud data and output a predicted three-dimensional bounding box of the perceived target object, which is used to represent the position and size of the perceived target object in three-dimensional space. The target perception model to be detected may refer to the target detection model in traditional technology.

[0065] S207: Determine the difference between the predicted three-dimensional bounding box and the real three-dimensional bounding box with the color annotated based on a preset algorithm, and determine the detection accuracy of the target perception model to be detected for real target objects of different colors based on the difference.

[0066] In an embodiment of the present application, the preset algorithm may include a preset geometric error evaluation algorithm, which usually quantifies the difference by analyzing the degree of overlap or displacement error. Specifically, the preset algorithm may be an intersection over union (IoU) algorithm, a KITTI data set evaluation algorithm, and the like. Based on the preset algorithm, the difference between the predicted three-dimensional bounding box and the real three-dimensional bounding box after the color is annotated can be determined. If the difference is small for the real target object of the first color and large for the real target object of the second color, it means that the detection accuracy of the target perception model to be detected for the real target object of the first color is high, and the detection accuracy of the real target object of the second color is low. In some specific embodiments, the difference value can be used as the detection accuracy of the target perception model to be detected for the real target object of the target color.

[0067] In some embodiments, if the target perception model to be detected is the original supervised target perception model, then after the three-dimensional point cloud data is input, the target perception model to be detected outputs a predicted three-dimensional bounding box and a predicted confidence. In some specific embodiments, the difference between the predicted three-dimensional bounding box and the real three-dimensional bounding box after the color is annotated can be determined according to the intersection-over-union algorithm as the detection accuracy of the corresponding color. In other embodiments, the predicted three-dimensional bounding box and the predicted confidence can also be input into the KITTI dataset evaluation algorithm to obtain the average accuracy value as the detection accuracy of the corresponding color.

[0068] In other embodiments, the target perception model to be detected includes the original unsupervised target perception model, and the preset algorithm includes an intersection-over-union detection algorithm; determining the difference value between the predicted three-dimensional bounding box and the real three-dimensional bounding box after color annotation based on the preset algorithm includes, based on the intersection-over-union detection algorithm, determining the bird's-eye view intersection-over-union ratio between the predicted three-dimensional bounding box and the real three-dimensional bounding box after color annotation as the difference value.

[0069] In the embodiment of the present application, an unsupervised 3D point cloud target detection model can be selected as the target perception model to be detected, and unsupervised model reasoning can be performed after inputting the 3D point cloud data, and the predicted 3D bounding box can be output. Then, the intersection over union (IoU-BEV) of the predicted 3D bounding box and the real 3D bounding box with the color annotated is determined by the intersection over union algorithm, and the detection accuracy of the target perception model to be detected for real target objects of different colors is determined as the difference value.

[0070] The accuracy detection method for the vehicle-mounted radar perception model provided in the embodiment of the present application annotates the color of the real three-dimensional bounding box of the real target object through a color image that matches the three-dimensional point cloud data, then inputs the three-dimensional point cloud data into the target perception model to be detected, and outputs the predicted three-dimensional bounding box. Then, the difference value between the predicted bounding box and the real bounding box after the color annotation is calculated through a preset algorithm, which not only considers the information of the three-dimensional point cloud data itself, but also incorporates the color information into the reference dimension. Based on the difference value, the detection accuracy of the model for real target objects of different colors can be accurately judged, and the perception accuracy of the laser radar for the target can be more accurately and reliably detected from the target perception model level. Furthermore, by determining the detection accuracy of the target perception model to be detected for real target objects of different colors, the influence of color on the perception ability of the laser radar can also be detected, providing a solid detection foundation for improving the reliability of autonomous driving technology.

[0071] To further test the accuracy of the lidar's perception of the target, the accuracy of the pre-trained model can also be tested while retaining the model structure of the original supervised target perception model.

[0072] In some embodiments, the to-be-detected target perception model is determined based on an original supervised target perception model; The The 3D point cloud data is input into the target perception model to be detected, and the predicted 3D bounding box of the perceived target object is output, including:

[0073] S401: Label the three-dimensional point cloud data with color based on the color information.

[0074] S403: Determine training data based on the color-labeled three-dimensional point cloud data and the color-labeled real three-dimensional bounding box, where the training data corresponds to color information of a single category.

[0075] S405: Train an initial model based on the training data to obtain the to-be-detected target perception model, wherein the initial model has the same model structure as the original supervised target perception model.

[0076] S407: Input the test three-dimensional point cloud data into the to-be-detected target perception model, and output the predicted three-dimensional bounding box, wherein the color information category of the test three-dimensional point cloud data is different from that of the training data.

[0077] In the embodiment of the present application, after the color information of the real target object is determined, the 3D point cloud data is annotated with color. Specifically, the 3D point cloud data may include multiple real target objects, and the 3D point cloud data is annotated with color according to the corresponding color information.

[0078] After data preprocessing based on the three-dimensional point cloud data after color annotation and the real three-dimensional bounding box after color annotation, the training data is determined, and each group of training data corresponds to the color information of a single category. In some embodiments, the data preprocessing may include dividing the three-dimensional point cloud data after color annotation and the real three-dimensional bounding box after color annotation into multiple groups of training data according to different color categories, and removing the three-dimensional point cloud data of other color target objects and the data enhancement module (ObjectSample module) in each group of training data. The data enhancement module is used to sample objects from other point cloud files and add them to the current point cloud. The data enhancement module can only retain the three-dimensional point cloud data of the current color category by removing the data enhancement module, and maintain the singleness of the color category corresponding to each group of training data. The data preprocessing also includes dividing the training data of each group of color categories into training set data and verification set data according to a preset ratio, and the preset ratio can be 1:1. Due to the difference in the number of samples of target objects of different color categories, the training data corresponding to the color category whose sample number is less than the preset threshold can also be removed. For example, if there are 100 pieces of training data in the yellow category and the preset threshold is 200, the 100 pieces of training data in the yellow category are removed so that the training data of each color category meets the requirements.

[0079] In an embodiment of the present application, m groups of training data can be determined based on the three-dimensional point cloud data after color annotation and the real three-dimensional bounding box after the color annotation, each group of training data corresponds to a single category of color information and each group of training data is divided into training set data and validation set data, where m≥1. The initial model is trained based on the m groups of training data to obtain m perception models of the target to be detected, each perception model of the target to be detected corresponds to a single category of color information. The initial model has the same model structure as the original supervised target perception model.

[0080] In an embodiment of the present application, the initial model can be trained based on a machine learning method, including a deep learning method, a reinforcement learning method, and the like. After the initial model is constructed, the training set data is input into the initial model, and the corresponding predicted three-dimensional bounding box is output. During the training process of the initial model, the training parameters in the initial model are continuously iteratively adjusted until the preset requirements are met. Meeting the preset requirements can be comparing the predicted three-dimensional bounding box with the validation set data, and when the difference is less than a preset parameter adjustment threshold, it is considered that the preset requirements are met; it can also be the number of iterations of the preset training process, and when the number of iterations meets the preset value, it is considered that the preset requirements are met.

[0081] In order to test the accuracy of each target perception model for other color category 3D point cloud data, the corresponding test 3D point cloud data is input into each target perception model, and m predicted 3D bounding boxes are output respectively. Among them, the color information category of the test 3D point cloud data is different from that of the training data, that is, if the target perception model to be detected is trained with the training data of color category A, the color category corresponding to the test 3D point data is B, C, and so on.

[0082] In the embodiment of the present application, by retaining the original supervised target perception model structure, the initial model is retrained based on the training data of different color categories to obtain the target perception model to be detected, which can further improve the perception detection accuracy of the laser radar to the target. Furthermore, by predicting the three-dimensional bounding boxes of target objects of different colors through the target perception model to be detected, the influence of different colors on the detection accuracy can be further analyzed.

[0083] In some embodiments, the preset algorithm includes a KITTI dataset evaluation algorithm, and determining the difference between the predicted three-dimensional bounding box and the real three-dimensional bounding box after color annotation based on the preset algorithm includes:

[0084] S501: Input the test three-dimensional point cloud data into the perception model of the target to be detected, and output the predicted three-dimensional bounding box and the prediction confidence matching the predicted three-dimensional bounding box.

[0085] S503: Input the predicted three-dimensional bounding box and the predicted confidence into the KITTI dataset evaluation algorithm to obtain an average precision value as the difference value.

[0086] In the embodiment of the present application, since the target perception model to be detected has the same model structure as the original supervised target perception model, and the original supervised target perception model can output a predicted three-dimensional bounding box and a matching prediction confidence after inputting three-dimensional point cloud data, the test three-dimensional point cloud data is input into the target perception model to be detected, and the prediction confidence matching the predicted three-dimensional bounding box can also be output. The predicted three-dimensional bounding box and the prediction confidence are input into the KITTI data set evaluation algorithm to obtain the average accuracy value as the difference value.

[0087] In some embodiments, inputting the predicted three-dimensional bounding box and the predicted confidence into the KITTI dataset evaluation algorithm to obtain an average precision value as the difference value includes:

[0088] S601: Determine the target perception difficulty gradient of the training data based on the KITTI dataset evaluation algorithm.

[0089] S603: Determine an average bird's-eye view accuracy value and / or a three-dimensional average accuracy value under the target perception difficulty gradient.

[0090] In an embodiment of the present application, based on the KITTI data set evaluation algorithm, the target perception difficulty gradient of the training data can be determined. The target perception difficulty gradient is determined based on the validity of the three-dimensional point cloud data. Specifically, if the occlusion rate of the target object is higher, the target perception difficulty gradient is higher. For example, the target perception difficulty gradient can be normal difficulty, medium difficulty or difficult difficulty. Determine the bird's-eye view average accuracy value (AP40-BEV) and / or the three-dimensional average accuracy value (AP40-3D) under the target perception difficulty gradient. Specifically, one of the bird's-eye view average accuracy value or the three-dimensional average accuracy value can be determined as the difference value, or both the bird's-eye view average accuracy value and the three-dimensional average accuracy value can be used as the difference value.

[0091] To further determine the influence of different colors on the accuracy of the vehicle-mounted radar perception model, in some embodiments, after determining the detection accuracy of the to-be-detected target perception model for real target objects of different colors based on the difference value, the method further includes:

[0092] S601: If the difference in detection accuracy of the target perception model to be detected for real target objects of different colors is greater than a preset accuracy difference threshold, the target three-dimensional point cloud data contained in the real three-dimensional bounding box of each color category is determined.

[0093] S603: Determine a target point cloud reflectivity and a target point cloud density value of the target three-dimensional point cloud data of each color category.

[0094] S605: Determine the reflectivity difference of the target point cloud of different color categories and the density value difference of the target point cloud of different color categories.

[0095] In the embodiment of the present application, when the difference in detection accuracy of the target perception model to be detected for real target objects of different colors is greater than a preset accuracy difference threshold, it is considered that different colors have a greater impact on the perception accuracy of the target perception model to be detected. Determine the target three-dimensional point cloud data contained in the real three-dimensional bounding box of each color category, and determine the target point cloud reflectivity and target point cloud density value of the target three-dimensional point cloud data of each color category.

[0096] Specifically, the method determines the average value or median of the reflectivity of all target three-dimensional point cloud data of each color category as the target point cloud reflectivity, and determines the average value or median of the point cloud density values ​​of all target three-dimensional point cloud data of each color category as the target point cloud density value.

[0097] Furthermore, the reflectivity difference of target point clouds of different color categories and the density value difference of target point clouds of different color categories are determined to provide a data basis for further analysis of the impact of different colors on the accuracy of the vehicle-mounted radar perception model.

[0098] In some specific embodiments, the accuracy detection method for the vehicle-mounted radar perception model provided in the embodiments of the present application is as follows: Figure 3 As shown in the figure, the real scene perception dataset of autonomous driving is obtained based on the vehicle-mounted radar, and the matching RGB images, 3D point cloud data and the real 3D bounding box annotation files after color annotation are obtained in the dataset to construct a color annotation dataset. The 3D point cloud data of target objects of different color categories are analyzed, and the point cloud reflectivity and point cloud density indicators of target objects of different color categories are counted. For the supervised target detection model, the color annotation dataset is divided into a training set and a validation set. The model is trained while keeping the model structure unchanged, and the model is inferred on the validation set to obtain the predicted bounding box and the predicted confidence. The detection accuracy index AP40 is used to evaluate the perception performance difference of the trained supervised target detection model for target objects of different color categories. For the unsupervised target detection model, the model is inferred based on the color annotation dataset to obtain the predicted bounding box, and the detection accuracy index IoU-BEV is used to evaluate the perception performance difference of the unsupervised target detection model for target objects of different color categories.

[0099] This application performs accuracy detection of the vehicle-mounted radar perception model based on an open source large-scale autonomous driving real-scene perception dataset, ensuring the universality of the detection; in addition, the accuracy detection method of the vehicle-mounted radar perception model of this application is widely applicable to annotable objects.

[0100] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0101] Based on the same inventive concept, the embodiment of the present application also provides a device 900 for detecting the accuracy of a vehicle-mounted radar perception model for implementing the above-mentioned method for detecting the accuracy of a vehicle-mounted radar perception model. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in the embodiments of one or more devices for detecting the accuracy of a vehicle-mounted radar perception model provided below can refer to the limitations of the method for detecting the accuracy of a vehicle-mounted radar perception model in the above text, and will not be repeated here.

[0102] In one embodiment, Figure 4 As shown, a precision detection device for a vehicle-mounted radar perception model is provided, comprising:

[0103] The data acquisition module 901 is used to acquire three-dimensional point cloud data based on the vehicle-mounted radar, and determine a real three-dimensional bounding box of a real target object in the three-dimensional point cloud data;

[0104] A color labeling module 902 is used to obtain a color image matching the three-dimensional point cloud data, and label the color of the real three-dimensional bounding box based on the color image;

[0105] Prediction module 903, used for inputting the three-dimensional point cloud data into the target perception model to be detected, and outputting a predicted three-dimensional bounding box of the perceived target object;

[0106] The detection accuracy determination module 904 is used to determine the difference between the predicted three-dimensional bounding box and the real three-dimensional bounding box after color annotation based on a preset algorithm, and determine the detection accuracy of the target perception model to be detected for real target objects of different colors based on the difference.

[0107] In some embodiments, the color labeling module 902 is also used to project the real three-dimensional bounding box onto the color image to obtain a projected bounding box; determine the color information of the real target object based on the pixel values ​​in the projected bounding box; and label the color of the real three-dimensional bounding box based on the color information.

[0108] In some embodiments, the target perception model to be detected is determined based on the original supervised target perception model; the prediction module 903 is also used to label the color of the three-dimensional point cloud data based on the color information; based on the three-dimensional point cloud data after color annotation and the real three-dimensional bounding box after color annotation, the training data is determined, and the training data corresponds to a single category of color information; based on the training data, an initial model is trained to obtain the target perception model to be detected, and the initial model has the same model structure as the original supervised target perception model; the test three-dimensional point cloud data is input into the target perception model to be detected, and the predicted three-dimensional bounding box is output, and the color information category of the test three-dimensional point cloud data is different from that of the training data.

[0109] In some embodiments, the preset algorithm includes a KITTI dataset evaluation algorithm, and the detection accuracy determination module 904 is also used to input the test three-dimensional point cloud data into the perception model of the target to be detected, and output a prediction confidence that matches the predicted three-dimensional bounding box; the predicted three-dimensional bounding box and the prediction confidence are input into the KITTI dataset evaluation algorithm to obtain an average accuracy value as the difference value.

[0110] In some embodiments, the detection accuracy determination module 904 is further used to determine the target perception difficulty gradient of the training data based on the KITTI dataset evaluation algorithm; and determine the bird's-eye view average accuracy value and / or three-dimensional average accuracy value under the target perception difficulty gradient.

[0111] In some embodiments, the target perception model to be detected includes the original unsupervised target perception model, and the detection accuracy determination module 904 is also used to determine the bird's-eye view intersection-and-union ratio of the predicted three-dimensional bounding box and the real three-dimensional bounding box after the color annotation based on the intersection-and-union ratio detection algorithm, as the difference value.

[0112] In some embodiments, the accuracy detection device 900 of the vehicle-mounted radar perception model is also used to determine the target three-dimensional point cloud data contained in the real three-dimensional bounding box of each color category if the difference in detection accuracy of the target perception model to be detected for real target objects of different colors is greater than a preset accuracy difference threshold; determine the target point cloud reflectivity and target point cloud density value of the target three-dimensional point cloud data of each color category; determine the difference in target point cloud reflectivity of different color categories, and the difference in target point cloud density values ​​of different color categories.

[0113] Each module in the above-mentioned accuracy detection device 900 for vehicle-mounted radar perception model can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0114] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for detecting the accuracy of a vehicle-mounted radar perception model is implemented.

[0115] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0116] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the accuracy detection method for the vehicle-mounted radar perception model described in any of the above embodiments are implemented.

[0117] In one embodiment, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps of the accuracy detection method for a vehicle-mounted radar perception model described in any of the above embodiments.

[0118] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0119] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0120] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0121] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0122] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for detecting the accuracy of a vehicle-mounted radar perception model, characterized in that: The method comprises: Acquire three-dimensional point cloud data based on the vehicle-mounted radar, and determine a true three-dimensional bounding box of a real target object in the three-dimensional point cloud data; Acquire a color image that matches the three-dimensional point cloud data, and annotate the color of the real three-dimensional bounding box based on the color image; Inputting the three-dimensional point cloud data into a target perception model to be detected, and outputting a predicted three-dimensional bounding box of the perceived target object; The difference between the predicted three-dimensional bounding box and the real three-dimensional bounding box marked with color is determined based on a preset algorithm, and the detection accuracy of the target perception model to be detected for real target objects of different colors is determined based on the difference.

2. The method according to claim 1, characterized in that The step of marking the color of the real three-dimensional bounding box based on the color image comprises: Projecting the real three-dimensional bounding box onto the color image to obtain a projected bounding box; Determining color information of the real target object based on pixel values ​​within the projected bounding box; The real three-dimensional bounding box is labeled with a color based on the color information.

3. The method according to claim 2, characterized in that The target perception model to be detected is determined based on the original supervised target perception model; The The 3D point cloud data is input into the target perception model to be detected, and the predicted 3D bounding box of the perceived target object is output, including: Annotating the three-dimensional point cloud data with color based on the color information; Determine training data based on the color-annotated three-dimensional point cloud data and the color-annotated real three-dimensional bounding box, where the training data corresponds to color information of a single category; Training an initial model based on the training data to obtain the to-be-detected target perception model, wherein the initial model has the same model structure as the original supervised target perception model; The test three-dimensional point cloud data is input into the target perception model to be detected, and the predicted three-dimensional bounding box is obtained as an output. The color information category of the test three-dimensional point cloud data is different from that of the training data.

4. The method according to claim 3, characterized in that The preset algorithm includes a KITTI data set evaluation algorithm, and the difference value between the predicted three-dimensional bounding box and the real three-dimensional bounding box after the color is marked based on the preset algorithm includes: Inputting the test three-dimensional point cloud data into the target perception model to be detected, and outputting the prediction confidence matching the predicted three-dimensional bounding box; The predicted three-dimensional bounding box and the predicted confidence are input into the KITTI dataset evaluation algorithm to obtain an average precision value as the difference value.

5. The method according to claim 4, characterized in that The The predicted three-dimensional bounding box and the predicted confidence are input into the KITTI data set evaluation algorithm to obtain an average precision value as the difference value, including: Determining a target perception difficulty gradient of the training data based on the KITTI dataset evaluation algorithm; Determine an average bird's-eye view accuracy value and / or a three-dimensional average accuracy value under the target perception difficulty gradient.

6. The method according to any one of claims 1 to 5, characterized in that: The target perception model to be detected includes an original unsupervised target perception model, the preset algorithm includes an intersection-over-union detection algorithm; and the method of determining the difference between the predicted three-dimensional bounding box and the real three-dimensional bounding box after color marking based on the preset algorithm includes: Based on the intersection-over-union detection algorithm, a bird's-eye view intersection-over-union ratio between the predicted three-dimensional bounding box and the real three-dimensional bounding box after the color marking is determined as the difference value.

7. The method according to any one of claims 1 to 5, characterized in that: Based on the After the difference value determines the detection accuracy of the target perception model to be detected for real target objects of different colors, the method further includes: If the difference in detection accuracy of the target perception model to be detected for real target objects of different colors is greater than a preset accuracy difference threshold, determining the target three-dimensional point cloud data contained in the real three-dimensional bounding box of each color category; Determine a target point cloud reflectivity and a target point cloud density value of target three-dimensional point cloud data of each color category; Determine the reflectivity difference of the target point cloud of different color categories and the density value difference of the target point cloud of different color categories.

8. An accuracy detection device for a vehicle-mounted radar perception model, characterized in that: The device comprises: A data acquisition module, used to acquire three-dimensional point cloud data based on the vehicle-mounted radar, and determine a real three-dimensional bounding box of a real target object in the three-dimensional point cloud data; A color labeling module, used to obtain a color image matching the three-dimensional point cloud data, and label the color of the real three-dimensional bounding box based on the color image; A prediction module, used for inputting the three-dimensional point cloud data into a perception model of a target to be detected, and outputting a predicted three-dimensional bounding box of the perceived target object; The detection accuracy determination module is used to determine the difference between the predicted three-dimensional bounding box and the real three-dimensional bounding box after color marking based on a preset algorithm, and determine the detection accuracy of the target perception model to be detected for real target objects of different colors based on the difference value.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images