Target detection post-processing method and device, equipment and storage medium

Abstract

The invention provides a target detection post-processing method and device, equipment and a storage medium, and belongs to the technical field of automatic driving. The method generates, using a first processor, a plurality of candidate boxes based on environmental data, and marks the plurality of candidate boxes based on a confidence threshold, thereby determining an index location of a first candidate box. And then reading the first candidate boxes based on the index positions by using a second processor, judging the categories of the plurality of first candidate boxes, calculating the intersection-to-union ratio of the two first candidate boxes of the same category, and obtaining an output result list based on the intersection-to-union ratio and an intersection-to-union ratio threshold value. According to the method, the respective advantages of the first processor and the second processor are fully utilized, so that the processing speed and efficiency are greatly improved while the accuracy of the whole target detection post-processing flow is ensured. Therefore, the speed and the accuracy of the whole process of the target detection algorithm in the automatic driving system are improved, and the adaptability and the stability of the vehicle automatic driving system under complex traffic conditions are enhanced.

Description

Technical Field

[0001] The present disclosure relates to the field of autonomous driving technology, and in particular to a target detection post-processing method, device, equipment, and storage medium. Background Art

[0002] With the development of artificial intelligence (AI), particularly breakthroughs in computer vision enabled by deep learning, autonomous driving technology has made rapid progress. The core of autonomous driving systems lies in their ability to accurately perceive their surroundings, which relies on efficient and accurate object detection algorithms. In autonomous driving systems, object detection algorithms must process large amounts of data in real time to identify and locate objects around the vehicle, such as other vehicles, pedestrians, and traffic signs. These algorithms typically involve steps such as image acquisition, preprocessing, object detection model inference, and post-processing.

[0003] However, the algorithm computing efficiency in related technologies is low, which makes it difficult to meet the urgent needs of autonomous driving systems for fast and accurate target recognition. Summary of the Invention

[0004] The present disclosure provides a target detection post-processing method, device, equipment and storage medium, which can solve the technical problems existing in the related art. The technical solution is as follows.

[0005] In a first aspect, the present disclosure provides a target detection post-processing method, the method comprising:

[0006] The first processor generates a plurality of candidate frames based on environmental data, wherein the environmental data includes image data of the environment and point cloud data of the environment;

[0007] The first processor marks the plurality of candidate boxes based on a confidence threshold, and if the confidence of the candidate box is higher than the confidence threshold, marks the candidate box as a first candidate box; if the confidence of the candidate box is lower than the confidence threshold, marks the candidate box as a second candidate box;

[0008] The first processor determines an index position of the first candidate box;

[0009] A second processor reads the first candidate box based on the index position;

[0010] The second processor determines categories of the plurality of first candidate frames, calculates an IoU of two first candidate frames of the same category, and obtains an output result list based on the IoU and an IoU threshold;

[0011] The first processor is a GPU (Graphics Processing Unit) or an embedded neural network processor NPU (Neural-network Processing Unit), and the second processor is a CPU (Central Processing Unit).

[0012] In a possible implementation, marking the plurality of candidate boxes based on a confidence threshold includes:

[0013] The first processor establishes a first array, the length of the first array is the same as the number of the candidate boxes, and the first processor marks the position corresponding to the first candidate box in the first array as 1, and marks the position corresponding to the second candidate box in the first array as 0.

[0014] In a possible implementation, the confidence threshold is 0.7-0.9.

[0015] In a possible implementation, determining the index position of the first candidate box includes:

[0016] The first processor calculates a prefix sum of the first array and forms a second array.

[0017] In a possible implementation, the second processor reads the candidate box based on the index position, including:

[0018] The second processor sorts the first candidate boxes based on the first array and the second array to form an initial list, wherein the initial list does not include the second candidate box.

[0019] In a possible implementation, obtaining an output result list based on the IoU and the IoU threshold includes:

[0020] The second processor classifies the first candidate boxes in the initial list. If the intersection-of-units ratio of two first candidate boxes in the same category is lower than the intersection-of-units ratio threshold, the two candidate boxes are retained in the output result list. If the intersection-of-units ratio of two first candidate boxes is higher than the intersection-of-units ratio threshold, one of the first candidate boxes is removed from the output result list.

[0021] In a possible implementation, the intersection-over-union ratio threshold is 0.7-0.9.

[0022] In a second aspect, the present disclosure provides a target detection post-processing device, the device comprising:

[0023] a first processor, configured to generate a plurality of candidate boxes based on the environmental data, mark the plurality of candidate boxes based on a confidence threshold, and determine an index position of the first candidate box;

[0024] The second processor is used to read the first candidate box based on the index position, determine the category of multiple first candidate boxes, calculate the intersection of union and ratio of two first candidate boxes of the same category, and obtain an output result list based on the intersection of union and ratio and the intersection of union threshold.

[0025] In a possible implementation, the first processor includes:

[0026] The acquisition module is used to establish the first array, where the length of the first array is the same as the number of the candidate boxes.

[0027] The marking module is configured to mark the position corresponding to the first candidate box in the first array as 1, and mark the position corresponding to the second candidate box in the first array as 0.

[0028] A determination module is configured to calculate a prefix sum of the first array and form a second array.

[0029] In one possible implementation, the second processor includes:

[0030] A reading module is configured to sort the first candidate boxes based on the first array and the second array to form an initial list, wherein the initial list does not include the second candidate box.

[0031] An output module is used to classify the first candidate boxes in the initial list. If the intersection-of-union (IoU) of two first candidate boxes of the same category is lower than the IoU threshold, the two candidate boxes are retained in the output result list; if the IoU of two first candidate boxes is higher than the IoU threshold, one of the first candidate boxes is removed from the output result list.

[0032] In a third aspect, the present disclosure provides a computer device comprising a processor and a memory, wherein the memory is used to store at least one computer program, and the at least one computer program is loaded by the processor and executed by the automatic parking method as claimed in any one of the claims of the first aspect.

[0033] In a fourth aspect, the present disclosure provides a computer-readable storage medium, wherein the storage medium stores at least one program code, and the at least one program code is loaded and executed by a processor to implement the target detection post-processing method as described in any one of the first aspects.

[0034] The technical solution provided by the present disclosure includes at least the following beneficial effects:

[0035] The present disclosure provides a target detection post-processing method, in which a first processor and a second processor are used to perform operations. The first processor (GPU or NPU) has good parallel processing capabilities and is suitable for performing fast processing tasks of large-scale data, while the second processor (CPU) can handle complex logical judgments. The present disclosure utilizes the respective advantages of the first processor and the second processor, so that the entire target detection post-processing process can greatly improve the processing speed and efficiency while ensuring accuracy. This improves the speed and accuracy of the entire process of the target detection algorithm in the autonomous driving system, and enhances the adaptability and stability of the vehicle autonomous driving system in complex traffic conditions.

[0036] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings, which are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure. In the drawings:

[0038] Figure 1 is a flow chart of a target detection post-processing method shown in an embodiment of the present disclosure;

[0039] Figure 2 is a schematic diagram of a first array and a second array shown in an embodiment of the present disclosure;

[0040] Figure 3 Schematic diagram of a target detection post-processing device shown in an embodiment of the present disclosure.

[0041] The above drawings illustrate specific embodiments of the present disclosure, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the present disclosure in any way, but rather to illustrate the concepts of the present disclosure to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0042] In order to make the objectives, technical solutions and advantages of the present disclosure more clear, the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.

[0043] The terms used in the embodiments of the present disclosure are intended only to explain the embodiments of the present disclosure and are not intended to limit the present disclosure. Unless otherwise defined, technical or scientific terms used herein should have the same ordinary meaning as those of ordinary skill in the art to which the present disclosure pertains. The terms "first," "second," "third," and similar terms used in the patent specification and claims of the present disclosure do not denote any order, quantity, or importance, but are merely used to distinguish between different components. Similarly, terms such as "a" or "an" do not denote a limitation on quantity, but rather denote the presence of at least one. Terms such as "include" or "comprising" mean that the elements or objects preceding the term "include" or "comprising" include the elements or objects listed after the term and their equivalents, and do not exclude other elements or objects. Terms such as "connected" or "connected" are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0044] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, storage, display, etc.), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the sample data involved in this application were obtained with full authorization.

[0045] With the development of artificial intelligence (AI), particularly breakthroughs in computer vision enabled by deep learning, autonomous driving technology has made rapid progress. The core of autonomous driving systems lies in their ability to accurately perceive their surroundings, which relies on efficient and accurate object detection algorithms. In autonomous driving systems, object detection algorithms must process large amounts of data in real time to identify and locate objects around the vehicle, such as other vehicles, pedestrians, and traffic signs. These algorithms typically involve steps such as image acquisition, preprocessing, object detection model inference, and post-processing.

[0046] However, the algorithm computing efficiency in related technologies is low, which makes it difficult to meet the urgent needs of autonomous driving systems for fast and accurate target recognition.

[0047] In view of the above technical problems, an embodiment of the present disclosure provides a target detection post-processing method, which can be applied to the automatic driving system of a vehicle.

[0048] Figure 1This is a flowchart of a target detection post-processing method provided by an embodiment of the present disclosure, see Figure 1 , the method comprises the following steps:

[0049] 101. A first processor generates multiple candidate boxes based on environmental data.

[0050] Among them, the environmental data includes two-dimensional image data of the environment and three-dimensional point cloud data of the environment. The images captured by the vehicle's camera and the point cloud data scanned by the lidar are transmitted to the first processor. The first processor inputs these data into the target detection model for inference, thereby obtaining a series of candidate boxes and the confidence score corresponding to each candidate box. Among them, the target detection model can be an advanced algorithm based on deep learning, such as YOLO (You Only Look Once, real-time target detection algorithm) or Centerpoint (Center-based 3D Object Detection and Tracking, center-based three-dimensional target detection and tracking).

[0051] The first processor can be an AI (Artificial Intelligence) chip such as a GPU (Graphics Processing Unit) or an embedded neural network processor NPU (Neural-network Processing Unit). The first processor (GPU or NPU) has good parallel processing capabilities and is suitable for performing fast processing tasks for large-scale data. The first processor generates candidate boxes at a high speed, which has a decisive impact on the operation speed of subsequent steps.

[0052] 102. The first processor marks multiple candidate boxes based on a confidence threshold. If the confidence of the candidate box is higher than the confidence threshold, the candidate box is marked as a first candidate box. If the confidence of the candidate box is lower than the confidence threshold, the candidate box is marked as a second candidate box.

[0053] A high confidence level for a candidate box indicates that a real object, such as a vehicle, pedestrian, or building, may be present in the box, and that the boundary of the candidate box is closer to the boundary of the real object. A low confidence level indicates that a real object may not be present in the box, and that the light outline may be misidentified as an obstacle due to reflections.

[0054] The first processor generates a large number of candidate boxes based on the environmental data. If each candidate box is processed during subsequent processing, the processing speed will be affected. Therefore, multiple candidate boxes are marked based on the confidence threshold. During subsequent processing, the candidate boxes with lower confidence (second candidate boxes) can be omitted, which speeds up the processing speed and reduces the need to access global memory, further improving the overall processing efficiency.

[0055] In some examples, the first processor creates a first array whose length is the same as the number of candidate boxes. The first processor marks the position corresponding to the first candidate box in the first array as 1, and marks the position corresponding to the second candidate box in the first array as 0. The second candidate box does not appear in the final output result list.

[0056] like Figure 2 As shown in the figure, assume there is a series of candidate boxes, labeled A, B, C, D, E, F, G, and H, which are the output results of the object detection algorithm. Each candidate box corresponds to a confidence score. If the confidence score of the candidate box exceeds the preset threshold, the corresponding position in the temporary array is marked as 1, otherwise it is marked as 0. For example, after this step, a first array with the content [1 0 1 1 0 0 1 0] can be obtained, indicating that the four candidate boxes A, C, D, and G are the first candidate boxes, and the four candidate boxes B, E, F, and H are the second candidate boxes.

[0057] In some examples, the confidence threshold is 0.7-0.9.

[0058] If the confidence threshold is too high, some candidate boxes with real objects will be missed. In other words, the second array will lack candidate boxes with real objects, which will increase the risk of vehicle collision.

[0059] If the confidence threshold is too low, the first processor might misidentify leaves, shadows, or texture interference as vehicles or pedestrians, leading to frequent braking or steering, increasing the risk of autonomous driving. Furthermore, this can cause the second array to retain too many candidate boxes, slowing the second processor's data read speed and subsequently processing these candidate boxes.

[0060] 103. The first processor determines an index position of the first candidate box.

[0061] Since the subsequent steps require the second processor to perform operations, the second processor needs to determine the index position of each first candidate box.

[0062] The first processor calculates the prefix sum of the first array and forms a second array. For example, the prefix sum of [1 0 1 1 0 0 1 0] is [0 1 1 2 3 3 3 4], that is, the second array is [0 1 1 2 3 3 3 4]. The second array indicates the index position of each first candidate box in the second array. Specifically, A is the first candidate box with a confidence score greater than the confidence threshold, and its index position is 0. C is the first candidate box with a second confidence score greater than the confidence threshold, and its index position is 1. D is the first candidate box with a third confidence score greater than the confidence threshold, and its index position is 2. G is the first candidate box with a fourth confidence score greater than the confidence threshold, and its index position is 3. B, E, F, and H, whose confidence scores are lower than the confidence threshold, will not be included in the second array.

[0063] In this way, the second processor will not read the second candidate box whose confidence score is lower than the confidence threshold when reading data, thereby speeding up the processing speed of the second processor.

[0064] 104. The second processor reads the first candidate box based on the index position.

[0065] The second processor reads the second array. Thus, the second processor only needs to read the first candidate frame, without having to read the second candidate frame. This speeds up the data read by the second processor. Furthermore, in subsequent calculations, the second processor only needs to perform calculations on the second candidate frame, thereby increasing the computing speed of the second processor.

[0066] The second processor is a CPU (Central Processing Unit), which can process complex logical judgments. Therefore, using the second processor to process subsequent steps can speed up the processing speed.

[0067] The present disclosure utilizes the respective advantages of the first processor and the second processor, so that the entire target detection post-processing process can greatly improve the processing speed and efficiency while ensuring accuracy.

[0068] In some examples, the second processor sorts the first candidate boxes based on the first array and the second array to form an initial list, where the initial list does not include the second candidate box. In subsequent calculations, the second processor only needs to perform operations on the first candidate box in the initial list, thereby improving the operation speed of the second processor.

[0069] 105. The second processor determines the categories of the multiple first candidate frames, calculates the intersection-over-union (IoU) of two first candidate frames of the same category, and obtains an output result list based on the IoU and IoU threshold.

[0070] The second processor can classify the first candidate frames into different categories based on the image data or point cloud data in the first candidate frames, such as dividing the first candidate frames into pedestrian candidate frames, vehicle candidate frames, building candidate frames, etc. Then, the intersection-over-union ratio is calculated for two candidate frames of each type.

[0071] For example, the intersection-over-union (IoU) ratio between two pedestrian candidate frames is calculated. If the IoU ratio is small, it indicates that the overlapping area between the two pedestrian candidate frames is small. Therefore, the pedestrians in the two pedestrian candidate frames are likely far apart, so both pedestrian candidate frames are retained. If the IoU ratio is large, it indicates that the overlapping area between the two pedestrian candidate frames is large. Therefore, the pedestrians in the two pedestrian candidate frames are likely close together or the same person, so one of the pedestrian candidate frames is removed. This can further reduce the number of candidate frames while ensuring their accuracy, thereby further improving subsequent processing speed.

[0072] The second processor does not calculate the intersection-and-union (IoU) of first candidate boxes between different categories, which can prevent the second processor from mistakenly deleting first candidate boxes. For example, if the IoU calculated by the second processor between the pedestrian candidate box and the vehicle candidate box is high, the pedestrian candidate box may be mistakenly deleted, which may cause the vehicle to make an incorrect judgment during autonomous driving, increasing the risk of autonomous driving.

[0073] In some examples, the second processor classifies the first candidate boxes in the initial list and then performs an NMS (Non-Maximum Suppress) algorithm on the first candidate boxes, that is, calculating the intersection-over-union (IoU) of two first candidate boxes of the same category and then comparing the IoU to a preset IoU threshold.

[0074] If the IoU of two first candidate boxes of the same category is lower than the IoU threshold, it means that the objects in the two first candidate boxes are far apart. In this case, the two candidate boxes are retained in the output result list to ensure that the vehicle can avoid these two objects during autonomous driving.

[0075] If the IoU of two first candidate boxes is higher than the IoU threshold, it means that the objects in the two first candidate boxes are close to each other or are the same object. In this case, one of the first candidate boxes is removed from the output result list to avoid affecting the prediction of obstacle trajectory when the vehicle is planning the path.

[0076] The final output list includes the location, category, and confidence scores of the candidate boxes, providing critical perception data for autonomous driving systems. This optimized process design not only improves the efficiency of the object detection algorithm but also ensures rapid and accurate object recognition within the autonomous driving system, providing strong technical support for the development of autonomous driving technology.

[0077] In some examples, the IoU threshold is 0.7-0.9. If the IoU threshold is too low, one of the two first candidate frames of the same class may be rejected even if the real objects are far apart. This can cause some real objects to be missed, increasing the risk of collision between the vehicle and the real objects and reducing the reliability of autonomous driving.

[0078] If the IoU threshold is too high, a large number of first candidate boxes of the same category will be retained, potentially misidentifying the same object as two, which can interfere with the vehicle's prediction of obstacle trajectories during path planning. This can also result in a large amount of data in the output result list, slowing the autonomous driving system's subsequent processing of the output result list.

[0079] like Figure 3 As shown, the embodiment of the present disclosure further provides a target detection post-processing device, the device comprising:

[0080] The first processor 310 is configured to generate multiple candidate boxes based on the environmental data, mark the multiple candidate boxes based on a confidence threshold, and determine an index position of a first candidate box.

[0081] The second processor 320 is used to read the first candidate box based on the index position, determine the category of multiple first candidate boxes, calculate the intersection of union (IoU) of two first candidate boxes of the same category, and obtain an output result list based on the IoU and IoU threshold.

[0082] like Figure 3 As shown, in some examples, the first processor 310 includes:

[0083] The acquisition module 311 is configured to establish a first array, wherein the length of the first array is the same as the number of candidate boxes.

[0084] The marking module 312 is configured to mark the position corresponding to the first candidate box in the first array as 1, and mark the position corresponding to the second candidate box in the first array as 0.

[0085] The determination module 313 is configured to calculate the prefix sum of the first array and form a second array.

[0086] like Figure 3 As shown, in some examples, the second processor 320 includes:

[0087] The reading module 321 is configured to sort the first candidate boxes based on the first array and the second array to form an initial list, wherein the initial list does not include the second candidate box.

[0088] The output module 322 is used to classify the first candidate boxes in the initial list. If the intersection-of-union (IoU) of two first candidate boxes of the same category is lower than the IoU threshold, the two candidate boxes are retained in the output result list. If the IoU of the two first candidate boxes is higher than the IoU threshold, one of the first candidate boxes is removed from the output result list.

[0089] An embodiment of the present disclosure further provides a computer device, which includes a processor and a memory, wherein the memory is used to store at least one computer program, and the at least one computer program is loaded by the processor and executes the above-mentioned target detection post-processing method.

[0090] The present disclosure also provides a computer-readable storage medium having at least one program code stored therein, which is loaded and executed by a processor to implement the above-described target detection post-processing method. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, or an optical data storage device.

[0091] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0092] The above description is only for the purpose of facilitating those skilled in the art to understand the technical solution of this application and is not intended to limit this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application shall be included in the scope of protection of this application.

Claims

1. A target detection post-processing method, characterized in that: The method comprises: The first processor generates a plurality of candidate frames based on environmental data, wherein the environmental data includes image data of the environment and point cloud data of the environment; The first processor marks the plurality of candidate boxes based on a confidence threshold, and if the confidence of the candidate box is higher than the confidence threshold, marks the candidate box as a first candidate box; if the confidence of the candidate box is lower than the confidence threshold, marks the candidate box as a second candidate box; The first processor determines an index position of the first candidate box; A second processor reads the first candidate box based on the index position; The second processor determines categories of the plurality of first candidate frames, calculates an IoU of two first candidate frames of the same category, and obtains an output result list based on the IoU and an IoU threshold; Among them, the first processor is a graphics processing unit GPU or an embedded neural network processor NPU, and the second processor is a central processing unit CPU.

2. The method according to claim 1, characterized in that The marking of the plurality of candidate boxes based on the confidence threshold comprises: The first processor establishes a first array, the length of the first array is the same as the number of the candidate boxes, and the first processor marks the position corresponding to the first candidate box in the first array as 1, and marks the position corresponding to the second candidate box in the first array as 0.

3. The method according to claim 2, characterized in that The confidence threshold is 0.7-0.

9.

4. The method according to claim 2, characterized in that The determining the index position of the first candidate box includes: The first processor calculates a prefix sum of the first array and forms a second array.

5. The method according to claim 4, characterized in that The second processor reads the candidate frame based on the index position, including: The second processor sorts the first candidate boxes based on the first array and the second array to form an initial list, wherein the initial list does not include the second candidate box.

6. The method according to claim 5, characterized in that The obtaining of an output result list based on the IoU and IoU threshold comprises: The second processor classifies the first candidate boxes in the initial list. If the intersection-of-units ratio of two first candidate boxes in the same category is lower than the intersection-of-units ratio threshold, the two candidate boxes are retained in the output result list. If the intersection-of-units ratio of two first candidate boxes is higher than the intersection-of-units ratio threshold, one of the first candidate boxes is removed from the output result list.

7. The method according to claim 1, characterized in that The intersection-over-union ratio threshold is 0.7-0.

9.

8. A target detection post-processing device, characterized in that: The device comprises: a first processor, configured to generate a plurality of candidate boxes based on the environmental data, mark the plurality of candidate boxes based on a confidence threshold, and determine an index position of the first candidate box; The second processor is used to read the first candidate box based on the index position, determine the category of multiple first candidate boxes, calculate the intersection of union and ratio of two first candidate boxes of the same category, and obtain an output result list based on the intersection of union and ratio and the intersection of union threshold.

9. A computer device, characterized in that: The computer device includes a processor and a memory, the memory is used to store at least one computer program, and the at least one computer program is loaded by the processor and executes the target detection post-processing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The storage medium stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the target detection post-processing method according to any one of claims 1 to 7.

Images