Target detection method and device, electronic equipment and storage medium

Abstract

This disclosure provides a target detection method, apparatus, electronic device, and storage medium. The method includes: acquiring each predicted bounding box corresponding to the data to be detected and the classification confidence of each predicted bounding box; determining multiple original grids in the data to be detected; determining the heat information of the original grids based on the predicted bounding boxes whose center points are located within the original grids; determining at least one target output grid; and for each target output grid, outputting the predicted bounding box with the highest classification confidence among the predicted bounding boxes whose center points are located within the target output grid, thereby obtaining the target detection result of the data to be detected. This method does not require calculating the intersection-union ratio (IU), solving the problem of long IU calculation time in the prior art using NMS, greatly improving the efficiency of target detection. Furthermore, this method filters the original grids through heat information, achieving effective suppression of negative samples and solving the problem of low recall in the prior art.

Description

Technical Field

[0001] This disclosure relates to the field of target detection technology, and in particular to a target detection method, apparatus, electronic device and storage medium. Background Technology

[0002] Non-Maximum Suppression (NMS) is a widely used technique in computer vision for detecting and removing redundant object bounding boxes or keypoints from images or point clouds. NMS is commonly used in object detection tasks to remove duplicate detection results, thereby improving the accuracy and efficiency of the algorithm.

[0003] The general steps of object detection methods are as follows: First, the image is fed into a deep learning network to predict N bounding boxes. Then, M bounding boxes with a classification confidence threshold set manually are selected from these N bounding boxes. Finally, NMS is used to leave a final bounding box for each positive sample.

[0004] However, traditional NMS has the following problems: 1. NMS calculation requires multiple calculations of the crossover ratio, which is very time-consuming; 2. If the manually set classification confidence threshold is too high, positive samples will be lost, and if it is too low, negative samples will be included in the final detection results, resulting in low recall. In addition, a low threshold will further exacerbate the time consumption problem of NMS. Summary of the Invention

[0005] To address, or at least partially address, the aforementioned technical problems, this disclosure provides a target detection method, apparatus, electronic device, and storage medium to solve the problems of low recall and long processing time in existing detection methods.

[0006] In a first aspect, embodiments of this disclosure provide a target detection method, the method comprising:

[0007] Obtain the predicted bounding boxes corresponding to the data to be detected and the classification confidence of each predicted bounding box;

[0008] Multiple original grids are determined in the data to be detected, and the heat information of the original grids is determined according to each prediction box corresponding to the original grids, wherein each prediction box corresponding to the original grids is a prediction box whose center point is located within the original grid.

[0009] Based on the heat information of each original raster, at least one target output raster is determined. For each target output raster, the prediction box with the highest classification confidence in each prediction box is output to obtain the target detection result of the data to be detected.

[0010] Secondly, embodiments of this disclosure also provide a target detection device, the device comprising:

[0011] The acquisition module is used to acquire the predicted bounding boxes corresponding to the data to be detected and the classification confidence of each predicted bounding box;

[0012] A heat determination module is used to determine multiple original grids in the data to be detected, and to determine the heat information of the original grids based on each prediction box corresponding to the original grids, wherein each prediction box corresponding to the original grids is a prediction box whose center point is located within the original grid.

[0013] The output module is used to determine at least one target output grid based on the heat information of each of the original grids, and for each target output grid, output the prediction box with the highest classification confidence in each prediction box to obtain the target detection result of the data to be detected.

[0014] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising: one or more processors; a storage device for storing one or more programs; and when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the target detection method as described above.

[0015] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the target detection method described above.

[0016] This disclosure provides a target detection method that obtains the predicted bounding boxes and their classification confidence scores for the data to be detected. Multiple original grids are then identified in the data. Based on the predicted bounding boxes whose center points are within the original grids, the heat information of each original grid is determined. At least one target output grid is then identified based on the heat information of each original grid. For each target output grid, the predicted bounding box with the highest classification confidence score among the predicted bounding boxes whose center points are within the target output grid is output, thus obtaining the target detection result for the data to be detected. This method achieves target detection of the data to be detected. It eliminates the need to calculate the intersection-union ratio (IU), solving the problem of long IU calculation time in existing technologies using NMS, greatly improving the efficiency of target detection. Furthermore, this method filters the original grids using heat information, suppressing negative samples in each predicted bounding box, solving the problem of low recall in existing technologies that directly use threshold suppression for negative samples. Attached Figure Description

[0017] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0018] Figure 1 This is a schematic diagram of a prior art target detection process provided in an embodiment of the present disclosure;

[0019] Figure 2 A typical process diagram of an NMS provided in this disclosure embodiment;

[0020] Figure 3 This is a flowchart of a target detection method according to an embodiment of the present disclosure;

[0021] Figure 4 A schematic diagram of a preset recall curve provided in an embodiment of this disclosure;

[0022] Figure 5 A schematic diagram illustrating the selection of the final output prediction box within a grid, as provided in an embodiment of this disclosure;

[0023] Figure 6 This is a schematic diagram of the structure of a target detection device according to an embodiment of the present disclosure;

[0024] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. Detailed Implementation

[0025] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0026] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0027] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0028] Before providing a detailed description of the target detection method provided in the embodiments of this disclosure, the technical problem solved by this method will be explained by way of example.

[0029] Object detection aims to identify foreground objects of interest (such as people, cars, bicycles, and other traffic participants) in image or point cloud data, which are called positive samples. Background objects that are not of interest to people (such as the sky, road surface, etc.) are called negative samples.

[0030] In the prior art, the general steps of a target detection method are as follows: First, the original image or point cloud enters a deep learning network to predict N proposal bboxes (prediction boxes). Through a manually set classification confidence threshold, M prediction boxes (M < N) greater than this threshold are selected from them. However, these prediction boxes may overlap or be close to each other, resulting in duplicate detection or false detection. Therefore, the NMS (Non-Maximum Suppression) method is used to finally leave a final proposal box for each positive sample. As Figure 1 shown Figure 1 is a schematic diagram of a target detection process of the prior art provided by an embodiment of the present disclosure.

[0031] Generally, NMS solves this problem by comparing the similarity between different prediction boxes and retaining the prediction box with the highest confidence. Specifically, NMS usually includes the following steps:

[0032] 1) For each prediction box of each category, sort them in descending order according to the confidence.

[0033] 2) Select the prediction box with the highest confidence and compare it with all the other prediction boxes.

[0034] 3) If the similarity (e.g., intersection over union) between the prediction box with the highest confidence and other prediction boxes is greater than a preset threshold, then delete the other prediction boxes from the candidate set.

[0035] Repeat steps 2) and 3) until all the prediction boxes of all categories are processed.

[0036] NMS is an essential part of target detection. It can improve the accuracy and efficiency of the algorithm and reduce the cases of false detection and duplicate detection. Refer to Figure 2 , Figure 2 is a schematic diagram of a typical process of NMS provided by an embodiment of the present disclosure. As shown in (a) of Figure 2 , there may be multiple prediction boxes for one target. After NMS, the final prediction box shown in (b) of Figure 2 can be obtained. That is, the non-maximum suppression process is the process from (a) of Figure 2 to (b) of Figure 2 .

[0037] However, traditional NMS has several problems: First, NMS calculations require calculating the IOU (Intersection over Union) with the remaining boxes for each candidate box, which is very time-consuming and inconvenient for parallel processing. Furthermore, removing negative samples based on a classification confidence threshold is problematic; if the threshold is set too high, positive samples will be lost; if it is set too low, more negative samples will be generated, making it difficult to balance precision and recall.

[0038] To address the slow speed of Non-Maximum Detection System (NMS), Zhaohui Zheng et al. proposed Cluster-NMS, which uses mathematical methods to improve the computation process of NMS and accelerate it. However, it still requires calculating Interchange of Units (IOU), which remains a bottleneck in NMS computation. To improve the recall rate of object detection, the academic community proposed Soft-NMS, which introduces a classification confidence penalty mechanism to retain positive samples suppressed by the NMS process. However, this method still requires a large classification confidence threshold to filter negative samples. When the distribution of new scenes differs significantly from the training set, positive samples are easily filtered out by the classification confidence, resulting in low recall. Lowering the classification confidence threshold, however, does not provide an effective means for suppressing independent negative samples in existing NMS methods, and further reducing the classification confidence threshold exacerbates the time-consuming problem of NMS.

[0039] Therefore, in order to solve the above problems, this disclosure provides a target detection method. By dividing the data to be detected into individual original grids to collect the heat information of each original grid, the cross-union ratio (CUI) is avoided, thus solving the problem of long time consumption in the NMS calculation of the CUI in the prior art. This greatly improves the efficiency of target detection. Furthermore, by determining the final output grid through the heat information of each original grid, negative samples are suppressed, thus solving the problem of low recall rate in the prior art where threshold suppression of negative samples is directly used.

[0040] Figure 3 This is a flowchart illustrating a target detection method according to an embodiment of this disclosure. The method can be executed by a target detection device, which can be implemented in software and / or hardware, and can be configured in an electronic device. Figure 3 As shown, the method may specifically include the following steps:

[0041] S110. Obtain the predicted bounding boxes corresponding to the data to be detected and the classification confidence of each predicted bounding box.

[0042] The data to be detected can be data for target detection, such as images or point clouds. Optionally, the data to be detected can be image data or point cloud data collected by the perception system of an autonomous vehicle.

[0043] That is, the target detection method provided in this embodiment can be used to perform target detection on image data collected by the perception system of an autonomous vehicle, so as to locate the prediction box corresponding to each positive sample from the image data; or, it can be used to perform target detection on point cloud data collected by the perception system of an autonomous vehicle, so as to locate the prediction box corresponding to each positive sample from the point cloud data. Positive samples can be traffic participants such as pedestrians and vehicles.

[0044] Through the above implementation methods, target detection can be performed on image data or point cloud data collected by the perception system to achieve collision detection, trajectory planning, and other functions for autonomous vehicles, thereby further improving the driving safety of autonomous vehicles.

[0045] It should be noted that, in addition to image data or point cloud data collected by the perception system of autonomous vehicles, the data to be detected can also be data from other computer vision fields, such as each frame of video data collected by traffic monitoring cameras. This disclosure does not limit this.

[0046] Specifically, deep learning networks can be used to predict the bounding boxes corresponding to the data to be detected, as well as the classification confidence of each bounding box. Each bounding box can be a localized bounding box for each target in the data to be detected; one target can correspond to one or more bounding boxes. The classification confidence of a bounding box describes the likelihood that it belongs to the corresponding category, i.e., the probability of belonging to the corresponding target.

[0047] In one example, obtaining the predicted bounding boxes and their classification confidence scores for the data to be detected includes: inputting the data to be detected into a pre-trained object detection model; determining the feature map corresponding to the data to be detected using the object detection model; and determining the predicted bounding boxes and their classification confidence scores based on the feature map.

[0048] The object detection model can be a neural network model such as YOLO, RetinaNet, or CenterNet. This model can be used to locate at least one predicted bounding box for each object in the data to be detected and output the classification confidence score for each predicted bounding box.

[0049] Specifically, the data to be detected can be input into a pre-trained target detection model. The target detection model can then extract features from the data to be detected, obtaining a downsampled feature map corresponding to the data. For each grid in the feature map, the target detection model provides multiple (e.g., 3) anchor boxes as priors. Each prior is then classified and regressed to finally obtain multiple predicted boxes and the classification confidence of each predicted box.

[0050] In the example above, the feature map corresponding to the data to be detected is obtained through the object detection model, and then each prediction box and its corresponding classification confidence are determined based on the feature map, thus achieving the initial acquisition of each prediction box. The accuracy of each prediction box is ensured as much as possible through model prediction.

[0051] For example, a predicted bounding box can be represented by [cls,x,y,w,h,score], where cls represents the category corresponding to the predicted bounding box, i.e. the corresponding target; x and y represent the horizontal and vertical coordinates of the predicted bounding box (such as the coordinates of the center point of the predicted bounding box); w and h represent the width and height of the predicted bounding box; and score represents the classification confidence of the predicted bounding box.

[0052] S120. In the data to be detected, determine multiple original grids, and determine the heat information of the original grids based on the prediction boxes corresponding to the original grids. The prediction boxes corresponding to the original grids are the prediction boxes whose center points are located within the original grids.

[0053] The original grid can be multiple grids divided in the data to be detected. In this embodiment, the number of original grids can be preset, and multiple original grids can be determined in the data to be detected based on the preset number of grids; or, the size of the original grids can be preset, and multiple original grids can be determined in the data to be detected based on the preset grid size; or, the number of grids corresponding to different sizes of data can be preset, and multiple original grids can be determined in the data to be detected based on the data size (such as image size); or, the number of grids corresponding to different sizes of targets can be preset, and multiple original grids can be determined in the data to be detected based on the size of the targets in the data to be detected.

[0054] For example, a calibration table can be predetermined, which includes data of various sizes and the grid size corresponding to targets of various sizes. The corresponding grid size can be queried in the calibration table according to the size of the data to be detected and the size of the targets therein. Then, multiple original grids can be divided in the data to be detected according to the queried grid size.

[0055] Alternatively, the grid can be divided into targets such that all predicted boxes in an original grid correspond to the same target, and the prediction boxes for the same target do not fall into different original grids. The number or size of the original grids can be determined, and then each original grid can be determined in the data to be detected based on the number or size of the original grids.

[0056] After determining the original grid cells, the next step is to determine the predicted bounding boxes corresponding to each original grid cell. Specifically, the predicted bounding boxes whose center points are located within the original grid cells can be defined as the predicted bounding boxes corresponding to the original grid cells. It should be noted that for each original grid cell, the predicted bounding boxes whose center points are located within the original grid cells can be understood as predicted bounding boxes for the same target.

[0057] Furthermore, for each original grid cell, the heat information of the original grid cell can be determined based on the prediction boxes corresponding to the original grid cell. The heat information of the original grid cell can be understood as describing the probability that the target corresponding to the original grid cell belongs to the positive sample.

[0058] In one specific implementation, the heat information of the original grid includes at least one of heat confidence and heat voting value, wherein the heat confidence is the sum of the classification confidence of each predicted box corresponding to the original grid, and the heat voting value is the number of each predicted box corresponding to the original grid.

[0059] That is, the popularity information can be popularity confidence, which can be obtained by summing the classification confidence of each predicted box corresponding to the original grid; or, the popularity information can be popularity voting value, which can be obtained by the number of each predicted box corresponding to the original grid; or, the popularity information can include both popularity confidence and popularity voting value.

[0060] In the above embodiments, the purpose of determining the popularity information, including at least one of popularity confidence and popularity voting value, is that for deep learning tasks, when the distribution of a new scene differs significantly from the training set, the original classification confidence threshold adjusted based on the test set will not be able to fit the new scene well, resulting in low recall. To solve this problem, in this embodiment, combined with the characteristics of deep learning tasks, the trend of its classification and regression performance is utilized. That is, in object detection tasks, positive sample regions are more sensitive than negative sample regions, and the classification confidence of proposals after regression is greater and more clustered, resulting in more predicted boxes corresponding to positive samples than to negative samples, and the classification confidence of predicted boxes corresponding to positive samples is higher than that of predicted boxes corresponding to negative samples.

[0061] Therefore, in order to better suppress negative samples, the heat information of each original grid is determined by combining the characteristics of more clustered distribution of positive sample prediction boxes and / or higher classification confidence. This is to statistically analyze the heat information of each target, so as to suppress the original grid containing the prediction box corresponding to the negative sample, thereby achieving accurate suppression of negative samples. This is beneficial to discover more positive samples while suppressing negative samples, and solves the problem of low recall rate in the existing technology that directly uses the classification confidence threshold to suppress negative samples.

[0062] Optionally, before determining the heat information of the original raster based on each predicted bounding box corresponding to the original raster, the method further includes: obtaining a preset confidence threshold, comparing the preset confidence threshold with the classification confidence of each predicted bounding box, and removing predicted bounding boxes from the data to be detected whose classification confidence is lower than the preset confidence threshold.

[0063] The preset confidence threshold can be a value much lower than the classification confidence threshold used in existing NMS, such as 0.00001. For example, the preset confidence threshold can be less than the classification confidence threshold used in the prior art, and the preset confidence threshold can maintain a set multiple relationship with the classification confidence threshold used in the prior art; or, the preset confidence threshold can be a value greater than zero and close to zero.

[0064] Specifically, before calculating the heat information of each original grid, the predicted boxes corresponding to the data to be detected can be filtered according to the obtained preset confidence threshold. Predicted boxes with classification confidence lower than the preset confidence threshold are removed, thus filtering out predicted boxes that obviously belong to negative samples.

[0065] Through the above implementation method, a very low preset confidence threshold can be used to first remove all prediction boxes that are obviously negative samples, while retaining almost all positive samples. Then, the heat statistics of each original grid can be performed to suppress all negative samples. This method can improve the efficiency and accuracy of subsequent heat statistics.

[0066] S130. Based on the heat information of each original grid, determine at least one target output grid. For each target output grid, output the prediction box with the highest classification confidence in each prediction box to obtain the target detection result of the data to be detected.

[0067] Specifically, after determining the heat information of each original raster, at least one target output raster can be determined from all the original rasteres based on the heat information. The target output raster can be the raster that needs to output the final prediction box, while other rasters do not need to output the final prediction box.

[0068] For example, if the heat information of the original raster includes heat confidence, the original raster with a heat confidence greater than a preset first threshold can be identified as the target output raster. This method, combined with the characteristic that predicted boxes of positive samples have higher classification confidence, can filter out target output raster containing predicted boxes of positive samples, thereby suppressing negative samples. Alternatively, if the heat information of the original raster includes heat voting values, the original raster with heat voting values ​​greater than a preset second threshold can be identified as the target output raster. This method, combined with the characteristic that predicted boxes of positive samples are more clustered, can filter out target output raster containing predicted boxes of positive samples, thereby suppressing negative samples.

[0069] In one example, determining at least one target output raster based on the heat information of each original raster includes: when the heat information includes heat confidence and heat voting value, determining the original raster with a heat confidence greater than a preset first threshold and a heat voting value greater than a preset second threshold as the target output raster.

[0070] That is, if the popularity information includes popularity confidence and popularity voting value, then the original raster with a popularity confidence greater than a preset first threshold and a popularity voting value greater than a preset second threshold can be used as the target output raster. In this way, the characteristics of more clustered predicted bounding boxes and higher classification confidence of positive samples can be combined to suppress negative samples, further ensuring the accuracy of negative sample suppression.

[0071] In this embodiment, considering that some positive samples may have extreme cases, such as some targets having a high confidence level in the original grid but a low vote value, or some targets having a high vote value in the original grid but a low confidence level, in order to screen out positive samples in extreme cases and further improve the recall rate, a soft curve can be introduced to determine the target output grid.

[0072] In another example, determining at least one target output grid based on the popularity information of each original grid includes: obtaining a preset recall curve when the popularity information includes popularity confidence and popularity voting value, wherein the preset recall curve is used to describe each critical popularity voting value and the corresponding critical popularity confidence; for each original grid, determining a reference coordinate point of the original grid in the coordinate system of the preset recall curve according to the popularity voting value and popularity confidence of the original grid; and determining the original grid corresponding to the reference coordinate point located above the preset recall curve as the target output grid according to the position of each reference coordinate point and the preset recall curve in the coordinate system.

[0073] The preset recall curve can be a soft curve, which describes each critical popularity vote value and its corresponding critical popularity confidence level. For example, the preset recall curve can be expressed by the following formula:

[0074]

[0075] Where s represents the critical confidence level of popularity, v represents the critical vote value of popularity, and k is an empirical parameter that is manually adjusted. Specifically, a preset recall curve is determined using the above formula and the manually adjusted empirical parameter. The horizontal axis of the coordinate system containing the preset recall curve represents the vote value of popularity, and the vertical axis represents the confidence level of popularity.

[0076] Furthermore, for each original grid cell, a reference coordinate point is obtained based on the popularity vote value and popularity confidence value of the original grid cell. Then, it is determined whether the reference coordinate point is located above the preset recall curve. If so, the original grid cell is determined as the target output grid cell.

[0077] For example, Figure 4 This is a schematic diagram of a preset recall curve provided in an embodiment of the present disclosure. See also: Figure 4 The preset recall curve can be used to filter reference coordinate points located above it to obtain the target output raster.

[0078] This approach combines the characteristics of more clustered predicted bounding boxes and higher classification confidence of positive samples to suppress negative samples. Furthermore, it can take into account extreme cases where some positive samples have high popularity confidence but low popularity voting value, or some positive samples have high popularity voting value but low popularity confidence, thereby further improving the recall rate.

[0079] It should be noted that, in another implementation, the target output grid can be determined by combining a preset first threshold, a preset second threshold, and a preset recall curve. Specifically, when the popularity information includes popularity confidence and popularity voting value, the original grids with popularity confidence greater than the preset first threshold and popularity voting value greater than the preset second threshold can be identified as target output grids. For the remaining original grids other than the target output grids, reference coordinate points of the remaining original grids in the coordinate system of the preset recall curve can be determined based on the popularity voting value and popularity confidence. Then, the remaining original grids corresponding to the reference coordinate points located above the preset recall curve can be identified as target output grids.

[0080] After determining each target output grid, the next step is to identify the prediction box with the highest classification confidence among the prediction boxes corresponding to the target output grid. This final prediction box is then output, meaning that only the prediction box with the highest classification confidence is retained under the same target output grid, thereby obtaining the target detection result of the data to be detected.

[0081] For example, Figure 5 This is a schematic diagram illustrating the selection of the final output prediction box within a grid, as provided in an embodiment of this disclosure. Figure 5 As shown, assuming the number of original grids corresponding to the number of samples to be detected is 9, after filtering by heat information, the number of target output grids is 4, namely the first row and third column, the second row and first column, the second row and second column, and the third row and third column are the target output grids. Light-colored dots represent suppressed prediction boxes, and dark-colored dots represent retained prediction boxes, which are the final output prediction boxes.

[0082] In this embodiment, after filtering by a pre-set confidence threshold, only the predicted bounding box with the highest classification confidence is retained in the output raster for the same target. This can achieve a non-maximum suppression effect similar to that of traditional NMS under the same classification confidence threshold. However, the method provided in this embodiment does not require calculating the IOU between any two proposal bounding boxes. Through the raster method, it is only necessary to count the center points of each predicted bounding box, which greatly reduces the amount of computation and the statistical process can be easily parallelized using a GPU (graphics processing unit).

[0083] The target detection method provided in this embodiment obtains the predicted bounding boxes and their classification confidence scores for the data to be detected. Then, it determines multiple original grids in the data. Based on the predicted bounding boxes whose center points are within the original grids, it determines the heat information of the original grids. Based on the heat information of each original grid, it determines at least one target output grid. For each target output grid, it outputs the predicted bounding box with the highest classification confidence score among the predicted bounding boxes whose center points are within the target output grid, thus obtaining the target detection result for the data to be detected. This method achieves target detection of the data to be detected. It eliminates the need to calculate the intersection-union ratio (IU), solving the problem of long IU calculation time in existing technologies using NMS, greatly improving the efficiency of target detection. Furthermore, this method filters the original grids using heat information, suppressing negative samples in each predicted bounding box, solving the problem of low recall in existing technologies that directly use thresholds to suppress negative samples, and improving the robustness of target detection.

[0084] Figure 6 This is a schematic diagram of the structure of a target detection device according to an embodiment of this disclosure. Figure 6As shown: The device includes: an acquisition module 610, a heat determination module 620, and an output module 630.

[0085] The acquisition module 610 is used to acquire each prediction box corresponding to the data to be detected and the classification confidence of each prediction box;

[0086] The heat determination module 620 is used to determine multiple original grids in the data to be detected, and to determine the heat information of the original grids according to each prediction box corresponding to the original grids, wherein each prediction box corresponding to the original grids is a prediction box whose center point is located within the original grid.

[0087] The output module 630 is used to determine at least one target output grid based on the heat information of each of the original grids, and for each target output grid, output the prediction box with the highest classification confidence in each prediction box to obtain the target detection result of the data to be detected.

[0088] Optionally, the heat information of the original grid includes at least one of heat confidence and heat voting value, wherein the heat confidence is the sum of the classification confidence of each predicted box corresponding to the original grid, and the heat voting value is the number of each predicted box corresponding to the original grid.

[0089] Optionally, the output module 630 is further configured to, when the popularity information includes popularity confidence and popularity voting value, determine the original grid with the popularity confidence greater than a preset first threshold and the popularity voting value greater than a preset second threshold as the target output grid.

[0090] Optionally, the output module 630 is further configured to: obtain a preset recall curve when the popularity information includes popularity confidence and popularity voting value, wherein the preset recall curve is used to describe each critical popularity voting value and the corresponding critical popularity confidence; for each original grid, determine a reference coordinate point of the original grid in the coordinate system where the preset recall curve is located based on the popularity voting value and popularity confidence of the original grid; and determine the original grid corresponding to the reference coordinate point located above the preset recall curve as the target output grid based on each reference coordinate point and the position of the preset recall curve in the coordinate system.

[0091] Optionally, the device further includes a filtering module, which is used to obtain a preset confidence threshold, compare the preset confidence threshold with the classification confidence of each prediction box, and remove prediction boxes with classification confidence lower than the preset confidence threshold from the data to be detected.

[0092] Optionally, the acquisition module 610 is specifically used for:

[0093] The data to be detected is input into a pre-trained target detection model; the target detection model determines the feature map corresponding to the data to be detected, and the prediction boxes and classification confidence of each prediction box are determined based on the feature map.

[0094] Optionally, the data to be detected is image data or point cloud data collected by the perception system of the autonomous vehicle.

[0095] The target detection device provided in this disclosure embodiment can execute the steps in the target detection method provided in this disclosure method embodiment, and has the execution steps and beneficial effects, which will not be repeated here.

[0096] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of this disclosure. See below for details. Figure 7 It shows a schematic diagram of a structure suitable for implementing the electronic device 500 in the embodiments of this disclosure. Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0097] like Figure 7 As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 501, which can perform various appropriate actions and processes to implement the methods of the embodiments described herein, based on a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing device 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0098] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts, thereby implementing the target detection method as described above. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by the processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0099] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0100] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to:

[0101] Obtain the predicted bounding boxes corresponding to the data to be detected and the classification confidence of each predicted bounding box;

[0102] Multiple original grids are determined in the data to be detected, and the heat information of the original grids is determined according to each prediction box corresponding to the original grids, wherein each prediction box corresponding to the original grids is a prediction box whose center point is located within the original grid.

[0103] Based on the heat information of each original raster, at least one target output raster is determined. For each target output raster, the prediction box with the highest classification confidence in each prediction box is output to obtain the target detection result of the data to be detected.

[0104] Optionally, when one or more of the above-described procedures are executed by the electronic device, the electronic device may also perform other steps described in the above embodiments.

[0105] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0106] Option 1: A target detection method, the method comprising:

[0107] Obtain the predicted bounding boxes corresponding to the data to be detected and the classification confidence of each predicted bounding box;

[0108] Multiple original grids are determined in the data to be detected, and the heat information of the original grids is determined according to each prediction box corresponding to the original grids, wherein each prediction box corresponding to the original grids is a prediction box whose center point is located within the original grid.

[0109] Based on the heat information of each original raster, at least one target output raster is determined. For each target output raster, the prediction box with the highest classification confidence in each prediction box is output to obtain the target detection result of the data to be detected.

[0110] Option 2: According to the method described in Option 1, the heat information of the original grid includes at least one of heat confidence and heat voting value, wherein the heat confidence is the sum of the classification confidence of each predicted box corresponding to the original grid, and the heat voting value is the number of each predicted box corresponding to the original grid.

[0111] Solution 3: According to the method described in Solution 2, determining at least one target output raster based on the thermal information of each of the original rasters includes:

[0112] When the popularity information includes popularity confidence and popularity voting value, the original raster with popularity confidence greater than a preset first threshold and popularity voting value greater than a preset second threshold is determined as the target output raster.

[0113] Option 4: According to the method described in Option 2, determining at least one target output raster based on the thermal information of each of the original rasters includes:

[0114] When the popularity information includes popularity confidence and popularity voting value, a preset recall curve is obtained, wherein the preset recall curve is used to describe each critical popularity voting value and the corresponding critical popularity confidence.

[0115] For each of the original grid cells, a reference coordinate point of the original grid cell in the coordinate system of the preset recall curve is determined based on the popularity vote value and popularity confidence value of the original grid cell.

[0116] Based on the positions of each reference coordinate point and the preset recall curve in the coordinate system, the original grid corresponding to the reference coordinate point located above the preset recall curve is determined as the target output grid.

[0117] Option 5: According to the method described in Option 1, before determining the heat information of the original raster based on each prediction box corresponding to the original raster, the method further includes:

[0118] Obtain a preset confidence threshold, and compare the preset confidence threshold with the classification confidence of each prediction box;

[0119] Predicted bounding boxes with a classification confidence level lower than the preset confidence threshold are removed from the data to be detected.

[0120] Solution 6: According to the method described in Solution 1, obtaining each predicted bounding box corresponding to the data to be detected and the classification confidence of each predicted bounding box includes:

[0121] The data to be detected is input into a pre-trained target detection model;

[0122] The target detection model determines the feature map corresponding to the data to be detected, and the prediction boxes and classification confidence of each prediction box are determined based on the feature map.

[0123] Option 7: According to the method described in Option 1, the data to be detected is image data or point cloud data collected by the perception system of the autonomous vehicle.

[0124] Option 8: A target detection device, comprising:

[0125] The acquisition module is used to acquire the predicted bounding boxes corresponding to the data to be detected and the classification confidence of each predicted bounding box;

[0126] A heat determination module is used to determine multiple original grids in the data to be detected, and to determine the heat information of the original grids based on each prediction box corresponding to the original grids, wherein each prediction box corresponding to the original grids is a prediction box whose center point is located within the original grid.

[0127] The output module is used to determine at least one target output grid based on the heat information of each of the original grids, and for each target output grid, output the prediction box with the highest classification confidence in each prediction box to obtain the target detection result of the data to be detected.

[0128] Option 9: An electronic device, the electronic device comprising:

[0129] One or more processors;

[0130] Storage device for storing one or more programs;

[0131] When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of schemes 1-7.

[0132] Option 10: A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of Options 1-7.

[0133] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

Claims

1. A target detection method characterized by, The method includes: Obtain the predicted bounding boxes corresponding to the data to be detected and the classification confidence of each predicted bounding box; Multiple original grids are determined in the data to be detected, and the heat information of the original grids is determined according to each prediction box corresponding to the original grids, wherein each prediction box corresponding to the original grids is a prediction box whose center point is located within the original grid. Based on the heat information of each original grid, at least one target output grid is determined. For each target output grid, the prediction box with the highest classification confidence in each prediction box is output to obtain the target detection result of the data to be detected. The heat information of the original grid includes at least one of heat confidence and heat voting value, wherein the heat confidence is the sum of the classification confidence of each predicted box corresponding to the original grid, and the heat voting value is the number of each predicted box corresponding to the original grid; the data to be detected is image data or point cloud data collected by the perception system of an autonomous vehicle.

2. The method of claim 1, wherein, Determining at least one target output raster based on the thermal information of each of the original rasters includes: When the popularity information includes popularity confidence and popularity voting value, the original raster with popularity confidence greater than a preset first threshold and popularity voting value greater than a preset second threshold is determined as the target output raster.

3. The method of claim 1, wherein, Determining at least one target output raster based on the thermal information of each of the original rasters includes: When the popularity information includes popularity confidence and popularity voting value, a preset recall curve is obtained, wherein the preset recall curve is used to describe each critical popularity voting value and the corresponding critical popularity confidence. For each of the original grid cells, a reference coordinate point of the original grid cell in the coordinate system of the preset recall curve is determined based on the popularity vote value and popularity confidence value of the original grid cell. Based on the positions of each reference coordinate point and the preset recall curve in the coordinate system, the original grid corresponding to the reference coordinate point located above the preset recall curve is determined as the target output grid.

4. The method of claim 1, wherein, Before determining the heat information of the original raster based on each prediction box corresponding to the original raster, the method further includes: Obtain a preset confidence threshold, and compare the preset confidence threshold with the classification confidence of each prediction box; Remove prediction boxes from the data to be detected that have a classification confidence level lower than the preset confidence threshold.

5. The method of claim 1, wherein, The acquisition of each predicted bounding box corresponding to the data to be detected and the classification confidence of each predicted bounding box includes: The data to be detected is input into a pre-trained target detection model; The target detection model determines the feature map corresponding to the data to be detected, and the prediction boxes and classification confidence of each prediction box are determined based on the feature map.

6. A target detection apparatus characterized by comprising: include: The acquisition module is used to acquire the predicted bounding boxes corresponding to the data to be detected and the classification confidence of each predicted bounding box; A heat determination module is used to determine multiple original grids in the data to be detected, and to determine the heat information of the original grids based on each prediction box corresponding to the original grids, wherein each prediction box corresponding to the original grids is a prediction box whose center point is located within the original grid. An output module is used to determine at least one target output grid based on the heat information of each of the original grids, and for each target output grid, output the prediction box with the highest classification confidence among the corresponding prediction boxes to obtain the target detection result of the data to be detected; the heat information of the original grids includes at least one of heat confidence and heat voting value, wherein the heat confidence is the sum of the classification confidence of each prediction box corresponding to the original grid, and the heat voting value is the number of each prediction box corresponding to the original grid; the data to be detected is image data or point cloud data collected by the perception system of an autonomous vehicle.

7. An electronic device, comprising: The electronic device includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-5.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-5.

Images