Target detection post-processing method, apparatus and device, and storage medium
By performing target detection on traffic scene images captured by vehicle-mounted cameras, determining the occlusion ratio of the detection box, and performing two rounds of filtering, the problem of false detection affecting normal driving in traffic scenes is solved, and the accuracy and safety of target detection are improved.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- ZHEJIANG GEELY HLDG GRP CO LTD
- Filing Date
- 2025-12-01
- Publication Date
- 2026-07-30
AI Technical Summary
In traffic scenarios, while existing target detection methods reduce the false negative rate, they are prone to falsely detecting targets that do not affect normal driving, causing vehicles to brake suddenly and affecting normal driving.
By performing target detection on traffic scene images captured by vehicle-mounted cameras, the occlusion ratio of the detection boxes is determined, and two filtering processes are performed: first, targets occluded by other detection boxes are filtered out based on the occlusion ratio, and second, targets that are occasionally falsely detected are filtered out based on the historical frame detection box set, forming the final target detection box set.
It effectively avoids false detections of targets intruding into the vehicle's trajectory, prevents emergency braking, and improves the accuracy and safety of target detection in traffic scenarios.
Smart Images
Figure CN2025139073_30072026_PF_FP_ABST
Abstract
Description
Post-processing methods, apparatus, equipment and storage media for target detection
[0001] Related applications
[0002] This application claims priority to Chinese patent application No. 202510091984.7, filed on January 21, 2025, the entire contents of which are incorporated herein by reference. Technical Field
[0003] This application relates to the field of target detection, and in particular to a target detection post-processing method, apparatus, equipment and storage medium. Background Technology
[0004] In conventional target detection, the goal is usually to detect as many targets as possible in the scene to reduce the false negative rate. However, in traffic scenarios, if the focus remains on reducing the false negative rate, some targets that would not normally affect driving may be detected. If these targets are mistakenly identified as collision-risk targets, they could actually impair the vehicle's ability to drive normally. Summary of the Invention
[0005] This application provides a target detection post-processing method, the method comprising the following steps:
[0006] Target detection is performed on the input image to obtain an original set of detection boxes. The input image is a traffic scene image captured by a vehicle-mounted camera.
[0007] Determine the occlusion ratio of a single detection box in the original detection box set, wherein the occlusion ratio represents the degree to which the target object contained in the single detection box is occluded by other objects;
[0008] The original set of detection boxes is filtered once according to the occlusion ratio to obtain the first set of target detection boxes;
[0009] The original detection box set is further filtered based on the historical frame detection box set corresponding to the original detection box set to obtain the second target detection box set.
[0010] In one embodiment, the step of determining the occlusion ratio of a single detection box in the original detection box set includes:
[0011] Select high-confidence detection boxes from the original set of detection boxes;
[0012] Determine the coordinates of the first point corresponding to the high-confidence detection box, the coordinates of the second point corresponding to a single detection box in the original detection box set, and the coordinates of the third point corresponding to the vehicle camera;
[0013] The first distance between the high-confidence detection frame and the vehicle-mounted camera is calculated based on the first point coordinates and the third point coordinates; the second distance between the single detection frame and the vehicle-mounted camera is calculated based on the second point coordinates and the third point coordinates.
[0014] The occlusion ratio of the individual detection frame is determined based on the first distance and the second distance.
[0015] In one embodiment, the step of determining the occlusion ratio of the individual detection box based on the first distance and the second distance includes:
[0016] If the sum of the first distance and the preset distance threshold is less than the second distance, the occupancy ratio of the single detection frame is determined according to the occupancy angles corresponding to the high confidence detection frame and the single detection frame, respectively. The occupancy angle represents the angular range occupied by the detection frame in the field of view of the vehicle camera.
[0017] If the sum of the first distance and the preset distance threshold is greater than or equal to the second distance, then the occlusion ratio of the single detection box is determined to be the preset minimum ratio.
[0018] In one embodiment, the step of determining the occlusion ratio of a single detection frame based on the occupancy angles corresponding to the high-confidence detection frame and the single detection frame respectively includes:
[0019] Calculate the first occupancy angle corresponding to the high-confidence detection box and the second occupancy angle corresponding to the single detection box;
[0020] The angular overlap range between the high-confidence detection frame and the individual detection frame is determined based on the first occupation angle and the second occupation angle.
[0021] The occlusion ratio of a single detection box is calculated based on the angular overlap interval.
[0022] In one embodiment, the step of filtering the original set of detection boxes according to the occlusion ratio to obtain a first set of target detection boxes includes:
[0023] The initial confidence of each detection box is updated based on the occlusion ratio to obtain the updated confidence.
[0024] Based on the updated confidence level, the original set of detection boxes is filtered once to obtain the first set of target detection boxes.
[0025] In one embodiment, the step of performing a secondary filtering on the original detection box set based on the historical frame detection box set corresponding to the original detection box set to obtain a second target detection box set includes:
[0026] Select an occluded detection box set from the original detection box set, wherein the occluded detection box set is a set of detection boxes whose occlusion ratio is greater than a preset ratio threshold;
[0027] Calculate the set of maximum intersection-union ratios between any detection box in the historical frame detection box set corresponding to the original detection box set and any detection box in the occluded detection box set;
[0028] The original set of detection boxes is further filtered based on the set of maximum intersection-union ratios to obtain a second set of target detection boxes.
[0029] Furthermore, to achieve the above objectives, this application also proposes a target detection post-processing device, which includes:
[0030] The requirements analysis module is used to select the target driving area from the map based on the user's driving needs;
[0031] The layer acquisition module is used to determine the first layer and the second layer corresponding to the target driving area. The first layer is used to reflect the user takeover status of the vehicle in the target driving area when performing driving assistance functions, and the second layer is used to reflect the availability of the vehicle's memory driving function in the target driving area.
[0032] The route recommendation module is used to generate a recommended driving route for the user in the target driving area based on the first layer and the second layer.
[0033] In addition, to achieve the above objectives, this application also proposes a target detection post-processing device, the device comprising: a memory, a processor, and a target detection post-processing program stored in the memory and executable on the processor, the target detection post-processing program being configured to implement the steps of the target detection post-processing method as described above.
[0034] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, storing a target detection post-processing program, which, when executed by a processor, implements the steps of the target detection post-processing method as described above.
[0035] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a target detection post-processing program. When the target detection post-processing program is executed by a processor, it implements the steps of the target detection post-processing method as described above.
[0036] This application performs object detection on an input image to obtain an original set of detection boxes. The input image is a traffic scene image captured by a vehicle-mounted camera. The occlusion ratio of each detection box in the original set is determined, where the occlusion ratio represents the degree to which a target object contained in a single detection box is occluded by other objects. The original set of detection boxes is then filtered once based on the occlusion ratio to obtain a first set of target detection boxes. Finally, the original set of detection boxes is filtered a second time based on the historical frame detection box set corresponding to the original set of detection boxes to obtain a second set of target detection boxes. Compared to traditional object detection methods, this method performs two post-processing operations on the original set of detection boxes detected in the traffic scene. The first filtering removes detection boxes that are partially occluded by other detection boxes based on the occlusion ratio of each individual detection box in the original set. The second filtering removes occasionally falsely detected detection boxes based on the historical frame detection box set. This avoids situations where falsely detected targets that do not affect normal vehicle operation intrude into the vehicle's trajectory, causing emergency braking. Attached Figure Description
[0037] Figure 1 is a schematic diagram of the target detection post-processing device in the hardware operating environment involved in the embodiment of this application;
[0038] Figure 2 is a flowchart of the first embodiment of the target detection post-processing method of this application;
[0039] Figure 3 is a flowchart of the second embodiment of the target detection post-processing method of this application;
[0040] Figure 4 is a schematic diagram of the occlusion in the target detection post-processing method of this application;
[0041] Figure 5 is a schematic diagram of the overlapping angle merging of the target detection post-processing method of this application;
[0042] Figure 6 is a schematic diagram of the occlusion ratio calculation of the target detection post-processing method of this application;
[0043] Figure 7 is a flowchart of the third embodiment of the target detection post-processing method of this application;
[0044] Figure 8 is a structural block diagram of the first embodiment of the target detection post-processing device of this application.
[0045] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Embodiments of the present invention
[0046] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of this application.
[0047] Referring to Figure 1, which is a schematic diagram of the target detection post-processing device structure in the hardware operating environment of the embodiment of this application.
[0048] As shown in Figure 1, the target detection post-processing device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; the user interface 1003 may also include standard wired and wireless interfaces. The network interface 1004 may include standard wired and wireless interfaces (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0049] Those skilled in the art will understand that the structure shown in Figure 1 does not constitute a limitation on the target detection post-processing device, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0050] As shown in Figure 1, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a target detection post-processing program.
[0051] In the target detection post-processing device shown in Figure 1, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the target detection post-processing device of this application can be set in the target detection post-processing device, and the target detection post-processing device calls the target detection post-processing program stored in the memory 1005 through the processor 1001 and executes the target detection post-processing method provided in the embodiment of this application.
[0052] The target detection post-processing method provided in this application can be deployed on a computer device for execution. The computer device can be an in-vehicle device (such as an in-vehicle controller, in-vehicle processor or other in-vehicle unit, etc.), or it can be a terminal outside the vehicle, an independent server, a cloud, a server cluster, a distributed system, an Internet of Things device or a vehicle network device, etc. This embodiment does not limit this, and those skilled in the art can set up the computer device for implementing the target detection post-processing method according to actual needs.
[0053] In conventional target detection, the goal is usually to detect as many targets as possible in the scene to reduce the false negative rate. However, in traffic scenarios, if the focus remains on reducing the false negative rate, some targets that would not normally affect driving may be detected. If these targets are mistakenly identified as collision-risk targets, they could actually impair the vehicle's ability to drive normally.
[0054] For example, if vehicle A is driving normally at a certain moment, and vehicle B is obstructed by non-motorized vehicles, pillars, etc., then vehicle B will not cross the obstruction and affect vehicle A's normal driving at that moment. However, if vehicle B is still detected as the target, vehicle A may mistakenly detect vehicle B as a target about to intrude into its driving trajectory, causing vehicle A to brake suddenly and thus affecting its normal driving. Therefore, there is an urgent need in the industry for a target detection post-processing method that can avoid the impact of falsely detected targets on normal vehicle driving.
[0055] This application provides a target detection post-processing method. Referring to FIG2, FIG2 is a flowchart of the first embodiment of the target detection post-processing method of this application.
[0056] In this embodiment, the target detection post-processing method includes the following steps:
[0057] Step S1: Perform target detection on the input image to obtain an original set of detection boxes. The input image is a traffic scene image captured by a vehicle-mounted camera.
[0058] Understandably, the input image above can contain any element that may exist in a traffic scene, such as motor vehicles, non-motor vehicles, pedestrians, pillars, fences, etc. The original detection box set above is a set of several detection boxes that contain the above arbitrary elements. For example, the original detection box set above can be represented as Input={{x,y,z,w,h,l,sita,conf},{},{}…}, where each detection box contains a center point (x,y,z), a detection size (w,h,l), a detection box heading angle sata, and a detection box confidence score conf.
[0059] It should be understood that the above object detection can be BEV (Bird's Eye View) object detection, and its specific process can be described as follows: the network model extracts features from the above input image to obtain a 1*C*W*H feature map, where 1 represents the batch size, C represents the number of channels, W represents the width, and H represents the height; the depth of each pixel value obtained by depth estimation is then processed by the LSS algorithm (a bottom-up 3D object detection algorithm that constructs bird's eye view features) to obtain the bird's eye view, and then further feature extraction is performed on the object detection head (output layer) and preset anchors, and finally, a single detection box in the above original detection box set is decoded.
[0060] Step S2: Determine the occlusion ratio of a single detection box in the original detection box set, whereby the occlusion ratio represents the degree to which the target object contained in the single detection box is occluded by other objects.
[0061] In a specific implementation, the specific parameters of the portion of any detection box that is occluded by other detection boxes (such as the angular range or area occupied by the detection box in the field of view of the vehicle camera) can be divided by the specific parameters when the detection box is not occluded. The resulting quotient can be used to represent the degree to which the target object contained in a single detection box is occluded by other objects, i.e., the aforementioned occlusion ratio.
[0062] Step S3: Filter the original detection box set according to the occlusion ratio to obtain the first target detection box set.
[0063] It's understandable that some missed detections of occluded targets are permissible in traffic scene target detection because they are not the closest targets and pose no collision risk. However, false detections should be avoided as much as possible, because false detections could cause occluded targets to reach extremely high expected speeds, thus intruding into the trajectory of normally traveling vehicles and forcing them to take emergency braking measures. More importantly, because the full appearance of an occluded target cannot be clearly seen, it is more prone to false detection. Here, "occluded target" refers to a target that is partially obscured.
[0064] Based on this, this embodiment proposes to filter the original detection box set according to the occlusion ratio of each detection box in the original detection box set, thereby filtering out the detection boxes in the original detection box set that are occluded to a certain extent (which can be defined according to the actual scene, such as 70%), and obtaining the above-mentioned first target detection box set.
[0065] Step S4: Perform a second filtering on the original detection box set based on the historical frame detection box set corresponding to the original detection box set to obtain the second target detection box set.
[0066] It should be noted that the above-mentioned set of historical frame detection boxes refers to the set of detection boxes corresponding to the original set of detection boxes in a historical frame (e.g., the previous frame).
[0067] In practical implementation, object detection suffers from occasional missed detections, such as significant differences between detection boxes in the previous and next frames. These differences may be due to these occasional missed detections. Furthermore, occluded targets in traffic scenes can also be missed. Therefore, a multi-frame continuous verification strategy can be adopted to eliminate the negative impact of occasional missed detections. Specifically, based on this strategy, all occluded detection boxes can be selected from the original detection box set. Then, these occluded detection boxes are further filtered using the historical frame detection box set, thereby removing occluded detection boxes from the original set that show significant differences between the current and historical frames, resulting in the second set of object detection boxes.
[0068] It is understood that all the detection boxes contained in the first and second target detection box sets described above are the detection boxes that the target detection post-processing method described in this embodiment finally needs to obtain. Compared with the detection boxes in the original detection box set, the detection boxes in the first and second target detection box sets increase the number of missed detections of occluded targets, but greatly reduce false detections, which can prevent targets from generating large predicted speeds that intrude into the vehicle's driving trajectory, and therefore are more suitable for traffic scenarios.
[0069] This embodiment performs object detection on an input image to obtain an original set of detection boxes. The input image is a traffic scene image captured by a vehicle-mounted camera. The occlusion ratio of each detection box in the original set is determined, where the occlusion ratio represents the degree to which the target object contained in the individual detection box is occluded by other objects. The original set of detection boxes is then filtered once based on the occlusion ratio to obtain a first set of target detection boxes. Finally, the original set of detection boxes is filtered a second time based on the historical frame detection box set corresponding to the original set of detection boxes to obtain a second set of target detection boxes. Compared to traditional object detection methods, this embodiment performs two post-processing operations on the original set of detection boxes detected in the traffic scene. The first filtering removes detection boxes that are partially occluded by other detection boxes based on the occlusion ratio of each individual detection box in the original set. The second filtering removes occasionally falsely detected detection boxes based on the historical frame detection box set. This avoids situations where falsely detected targets that do not affect normal vehicle operation intrude into the vehicle's trajectory and cause emergency braking.
[0070] Referring to Figure 3, Figure 3 is a flowchart of the second embodiment of the target detection post-processing method of this application.
[0071] In one feasible implementation, step S2 may include:
[0072] Step S21: Select high-confidence detection boxes from the original set of detection boxes.
[0073] It should be noted that the above-mentioned high-confidence detection boxes can refer to detection boxes with a confidence level greater than a preset confidence threshold (e.g., 0.7).
[0074] Step S22: Determine the coordinates of the first point corresponding to the high-confidence detection box, the coordinates of the second point corresponding to a single detection box in the original detection box set, and the coordinates of the third point corresponding to the vehicle camera.
[0075] It should be noted that the coordinates of the first, second, and third points mentioned above are all in the same reference coordinate system, such as the vehicle coordinate system with the center point of the vehicle where the camera is located as the origin.
[0076] Step S23: Calculate the first distance between the high-confidence detection frame and the vehicle-mounted camera based on the first point coordinates and the third point coordinates, and calculate the second distance between the single detection frame and the vehicle-mounted camera based on the second point coordinates and the third point coordinates.
[0077] Understandably, since the detection box is rectangular, it generally has four points (i.e., the four vertices of the rectangle). Based on this, the first point coordinate mentioned above can refer to the coordinates of the point closest to the vehicle camera among the four points of the high-confidence detection box in the vehicle coordinate system, and the second point coordinate mentioned above can refer to the coordinates of the point closest to the vehicle camera among the four points of a single detection box in the original detection box set in the vehicle coordinate system.
[0078] For example, assuming the coordinates of the first or second point are (X1, Y1) and the coordinates of the third point are (XC, YC), the first or second distance can be calculated based on the following formula: dis=sqrt((X1-XC)*(X1-XC)+(Y1-YC)*(Y1-YC)), where dis represents the first or second distance.
[0079] Step S24: Determine the occlusion ratio of the individual detection frame based on the first distance and the second distance.
[0080] In a specific implementation, a first specific parameter (such as the angle range or area occupied by the detection box in the field of view of the vehicle camera) corresponding to the part of a single detection box that is occluded by other detection boxes can be determined based on the first distance and the second distance. Then, the first specific parameter is compared with the second specific parameter when the single detection box is not occluded, so as to obtain the above-mentioned occlusion ratio.
[0081] In one feasible implementation, step S24 can be achieved through the following steps S241 and S242.
[0082] Step S241: If the sum of the first distance and the preset distance threshold is less than the second distance, then the occupancy ratio of the single detection frame is determined according to the occupancy angles corresponding to the high confidence detection frame and the single detection frame, where the occupancy angle represents the angular range occupied by the detection frame in the field of view of the vehicle camera.
[0083] It should be noted that the above preset distance threshold can be set according to the size of the target object in the actual scene, for example, 0.3m.
[0084] Step S242: If the sum of the first distance and the preset distance threshold is greater than or equal to the second distance, then the occlusion ratio of the single detection box is determined to be the preset minimum ratio.
[0085] In a specific implementation, if the sum of the first distance and the preset distance threshold is less than the second distance, it indicates that the single detection box is occluded by the high-confidence detection box. Therefore, the occlusion ratio of the single detection box can be determined based on the angle range occupied by the high-confidence detection box and the single detection box in the field of view of the vehicle camera. If the sum of the first distance and the preset distance threshold is greater than or equal to the second distance, it indicates that the single detection box is not occluded by the high-confidence detection box (or even if the single detection box is occluded by the high-confidence detection box, the detection effect is the same as when there is no occlusion). Therefore, it can be determined that the occlusion ratio of the single detection box is the preset minimum ratio (e.g., 1%).
[0086] In one feasible implementation, step S241 may include:
[0087] Step S2411: Calculate the first occupancy angle corresponding to the high confidence detection box and the second occupancy angle corresponding to the single detection box.
[0088] It should be noted that the first occupancy angle refers to the angular range occupied by the high-confidence detection box in the field of view of the vehicle camera, and the second occupancy angle refers to the angular range occupied by the single detection box in the field of view of the vehicle camera. Specifically, the occupancy angle (including the first and second occupancy angles) can be expressed in the form of [a, a+b], where a represents the starting angle at which the edge of the detection box begins to be captured by the vehicle camera, and a+b represents the ending angle when the detection box completely leaves the field of view of the vehicle camera.
[0089] In practical implementation, taking the calculation of the first occupancy angle corresponding to the high-confidence detection box as an example, assuming that the first distance obtained based on the above formula is dis1, and the horizontal coordinates of the four points of the high-confidence detection box in the vehicle coordinate system are X1, X2, X3, and X4 respectively, then the above first occupancy angle can be calculated based on the following formula:
[0090] angle1=arccos((X1-XC) / dist1);
[0091] angle2=arccos((X2-XC) / dist1);
[0092] angle3=arccos((X3-XC) / dist1);
[0093] angle4=arccos((X4-XC) / dist1);
[0094] a=min(angle1,angle2,angle3,angle4);
[0095] b=max(angle1,angle2,angle3,angle4).
[0096] Where angle1, angle2, angle3, and angle4 represent the angles between the straight lines between the corresponding points X1, X2, X3, and X4 and the corresponding points of the vehicle-mounted camera, and the reference direction (e.g., due north), respectively. 'a' represents the starting angle at which the edge of the high-confidence detection box begins to be captured by the vehicle-mounted camera, and 'b' represents the angle from the starting angle until the high-confidence detection box completely leaves the field of view of the vehicle-mounted camera. For example, refer to Figure 4, which is a schematic diagram of occlusion in the target detection post-processing method of this application. As can be seen from Figure 4, B is the detection box occluded by non-motorized vehicles and pillars. The occupancy angle of the vehicle-mounted camera (i.e., a normally driving vehicle) corresponding to B can be expressed as [a, a+b].
[0097] Step S2412: Determine the angular overlap range between the high-confidence detection frame and the single detection frame based on the first occupancy angle and the second occupancy angle.
[0098] Step S2413: Calculate the occlusion ratio of the single detection box based on the angle overlap interval.
[0099] It should be understood that, since the single detection box is occluded by the high-confidence detection boxes, there is an angular overlap between the single detection box and the high-confidence detection boxes. Referring to Figure 5, Figure 5 is a schematic diagram of the overlapping angle merging in the target detection post-processing method of this application. Assume that a single detection box is occluded by three high-confidence detection boxes, and the corresponding occluded target set is Occbox1={[a1,b1],[a2,b2],[a3,b3]}, where [a1,b1], [a2,b2], and [a3,b3] represent the occupancy angles (i.e., the first occupancy angles mentioned above) corresponding to the three high-confidence detection boxes. Since [a1,b1] and [a2,b2] have an overlap, they can be merged into [a1,b2]; while [a3,b3] does not overlap with other intervals and can be directly retained. Therefore, the first occupancy angles corresponding to these three high-confidence detection boxes can be merged and output as Occbox1={[a1,b2],[a3,b3]}.
[0100] Furthermore, the above-mentioned occlusion ratio can be calculated through the following steps: first, calculate the angular overlap interval between the first occupancy angle and the second occupancy angle; then, calculate the total occlusion angle based on the angular overlap interval; finally, divide the total occlusion angle by the second occupancy angle to obtain the above-mentioned occlusion ratio.
[0101] To more clearly illustrate the calculation principle of the occlusion ratio, please refer to Figure 6, which is a schematic diagram of the occlusion ratio calculation in the target detection post-processing method of this application. In Figure 6, Occbox1 (i.e., [a1,b2], [a3,b3]) represents the first occupancy angle corresponding to the high-confidence detection box, and box1 (i.e., [a,b]) represents the second occupancy angle corresponding to a single detection box. It can be seen that the occlusion ratio in Figure 6 can be calculated based on the following formula:
[0102] ratio=(b2-a) / (ba)+(b-a3) / (ba).
[0103] This embodiment selects a high-confidence detection box from the original detection box set; determines the first point coordinates corresponding to the high-confidence detection box, the second point coordinates corresponding to a single detection box in the original detection box set, and the third point coordinates corresponding to the vehicle camera; calculates a first distance between the high-confidence detection box and the vehicle camera based on the first and third point coordinates, and calculates a second distance between the single detection box and the vehicle camera based on the second and third point coordinates; if the sum of the first distance and a preset distance threshold is less than the second distance, calculates a first occupancy angle corresponding to the high-confidence detection box and a second occupancy angle corresponding to the single detection box; determines the angular overlap interval between the high-confidence detection box and the single detection box based on the first and second occupancy angles; calculates the occlusion ratio of the single detection box based on the angular overlap interval; if the sum of the first distance and the preset distance threshold is greater than or equal to the second distance, determines that the occlusion ratio of the single detection box is a preset minimum ratio. In this embodiment, when a single detection box is occluded by a high-confidence detection box, the method determines the angular overlap range between the high-confidence detection box and the single detection box based on the first occupancy angle corresponding to the high-confidence detection box and the second occupancy angle corresponding to the single detection box. Then, the occupancy ratio of the single detection box is calculated based on the angular overlap range, thereby improving the calculation accuracy of the occupancy ratio and providing a data foundation for the subsequent step of filtering the original detection box set based on the occupancy ratio.
[0104] Referring to Figure 7, which is a flowchart of the third embodiment of the target detection post-processing method of this application.
[0105] In one feasible implementation, step S3 can be achieved through the following steps S31 and S32:
[0106] Step S31: Update the initial confidence of the individual detection box according to the occlusion ratio to obtain the updated confidence.
[0107] In the specific implementation, the initial confidence of the above single detection box can be updated according to the following formula:
[0108] conf=(conf'–conf_th)*ratio+conf_th.
[0109] Where conf represents the updated confidence level, conf' represents the initial confidence level, conf_th represents the hyperparameter threshold (generally set to around 0.7), and ratio represents the occlusion ratio.
[0110] Step S32: Based on the updated confidence level, the original set of detection boxes is filtered once to obtain the first set of target detection boxes.
[0111] In the specific implementation, the original set of detection boxes can be filtered based on the updated confidence level. When the updated confidence level is less than a certain threshold (e.g., 80%), the detection box corresponding to that updated confidence level is directly deleted. Furthermore, after deleting detection boxes with updated confidence levels less than a certain threshold, overlapping detection boxes in the original set can be filtered out using NMS (Non-Maximum Suppression).
[0112] In one feasible implementation, step S4 can be achieved through the following steps S41, S42 and S43.
[0113] Step S41: Select an occluded detection box set from the original detection box set. The occluded detection box set is a set of detection boxes whose occlusion ratio is greater than a preset ratio threshold.
[0114] It should be noted that the above-mentioned preset ratio threshold can be flexibly set according to the actual scenario, for example, it can be set to 70%.
[0115] Step S42: Calculate the maximum cross-union ratio (CUI) between any detection box in the historical frame detection box set corresponding to the original detection box set and any detection box in the occluded detection box set.
[0116] In practical implementation, any detection box in the aforementioned set of occluded detection boxes can be transformed from the vehicle coordinate system to the world coordinate system based on the positioning signal. The positioning signal is a 3x3 rotation matrix R and a 1x3 translation matrix T. Based on the rotation matrix R, the yaw angle around the Z-axis can be obtained. Assuming the detection box in the vehicle coordinate system is represented as box1(x,y,z,l,w,h,sita), the transformed global coordinate system detection box can be represented as box2(x1,y1,z1,l,w,h,sita1), and the transformation formula is as follows:
[0117]
[0118] .
[0119] Following the same steps, any detection box in the historical frame detection box set can be transformed as described above, and the maximum intersection-union ratio (CIU) set between any detection box in the historical frame detection box set and any detection box in the occluded detection box set can be calculated based on the transformed detection boxes. For example, assuming OccBox (i.e., the transformed occluded detection box set) contains {detection box 1, detection box 2}, and BoxH (i.e., the transformed historical frame detection box set) contains {detection box A, detection box B, detection box C}, first, the CIU of detection box 1 is compared with those of detection boxes A, B, and C, and the maximum CIU corresponding to detection box 1 is selected; the process for obtaining the maximum CIU of detection box 2 is similar to that of detection box 1, and will not be elaborated here. Therefore, in this example, the final set of maximum CIUs contains two maximum CIUs.
[0120] Step S43: Perform a second filtering on the original detection box set based on the maximum intersection-union set to obtain a second target detection box set.
[0121] In a specific implementation, the maximum cross-union ratio (CUI) in the maximum CUI set can be compared with a preset CUI threshold: if the maximum CUI is greater than the preset CUI threshold, the detection box corresponding to the maximum CUI is retained; if the maximum CUI is less than or equal to the preset CUI threshold, the detection box corresponding to the maximum CUI is deleted. Finally, the retained detection boxes can be inversely transformed according to the transformation formula in step S42 above to transform the world coordinate system corresponding to the retained detection boxes into the vehicle coordinate system. The final set of detection boxes is the second target detection box set mentioned above.
[0122] In this embodiment, the initial confidence of a single detection box is updated based on the occlusion ratio to obtain an updated confidence. The original detection box set is then filtered once based on the updated confidence to obtain a first target detection box set. An occluded detection box set is selected from the original detection box set; this occluded detection box set consists of detection boxes whose occlusion ratio is greater than a preset threshold. The maximum intersection-union ratio (CIRR) between any detection box in the historical frame detection box set corresponding to the original detection box set and any detection box in the occluded detection box set is calculated. Based on the CIRR set, the original detection box set is filtered a second time to obtain a second target detection box set. Compared to traditional target detection post-processing methods, the method described in this embodiment updates the initial confidence of a single detection box based on the occlusion ratio, performs a first-stage screening of the original detection box set based on the updated confidence, and performs a second-stage screening of the original detection box set based on the maximum intersection-union ratio (IUU) between any detection box in the historical frame detection box set and any detection box in the occluded detection box set. This reduces the confidence of the occluded detection boxes, thereby reducing false detections in the target detection process and the missed detection of other vehicles caused by false detections.
[0123] Furthermore, this application also proposes a storage medium storing a target detection post-processing program, which, when executed by a processor, implements the steps of the target detection post-processing method described above.
[0124] Referring to Figure 8, which is a structural block diagram of the first embodiment of the target detection post-processing device of this application.
[0125] As shown in Figure 8, the target detection post-processing device proposed in this embodiment includes:
[0126] The target detection module 801 is used to perform target detection on the input image to obtain an original set of detection boxes. The input image is a traffic scene image captured by a vehicle-mounted camera.
[0127] The occlusion calculation module 802 is used to determine the occlusion ratio of a single detection box in the original detection box set, wherein the occlusion ratio represents the degree to which the target object contained in the single detection box is occluded by other objects;
[0128] A primary filtering module 803 is used to perform a primary filtering on the original set of detection boxes according to the occlusion ratio to obtain a first set of target detection boxes;
[0129] The secondary filtering module 804 is used to perform secondary filtering on the original detection box set based on the historical frame detection box set corresponding to the original detection box set, to obtain a second target detection box set.
[0130] This embodiment performs target detection on an input image to obtain an original set of detection boxes. The input image is a traffic scene image captured by a vehicle-mounted camera. The occlusion ratio of each detection box in the original set is determined, where the occlusion ratio represents the degree to which the target object contained in the individual detection box is occluded by other objects. The original set of detection boxes is then filtered once based on the occlusion ratio to obtain a first set of target detection boxes. Finally, the original set of detection boxes is filtered a second time based on the historical frame detection box set corresponding to the original set of detection boxes to obtain a second set of target detection boxes. Compared to traditional target detection methods, this embodiment performs two post-processing operations on the original set of detection boxes detected in the traffic scene. The first filtering removes detection boxes that are partially occluded by other detection boxes based on the occlusion ratio of each individual detection box in the original set. The second filtering removes occasionally falsely detected detection boxes based on the historical frame detection box set. This avoids situations where falsely detected targets that do not affect normal vehicle operation intrude into the vehicle's trajectory and cause emergency braking.
[0131] Based on the first embodiment of the target detection post-processing device described in this application, a second embodiment of the target detection post-processing device of this application is proposed.
[0132] In this embodiment, the occlusion calculation module 802 is further configured to select high-confidence detection boxes from the original detection box set; determine the first point coordinates corresponding to the high-confidence detection box, the second point coordinates corresponding to a single detection box in the original detection box set, and the third point coordinates corresponding to the vehicle camera; calculate a first distance between the high-confidence detection box and the vehicle camera based on the first point coordinates and the third point coordinates; calculate a second distance between the single detection box and the vehicle camera based on the second point coordinates and the third point coordinates; and determine the occlusion ratio of the single detection box based on the first distance and the second distance.
[0133] Furthermore, the occlusion calculation module 802 is also configured to determine the occlusion ratio of a single detection frame based on the occupancy angles corresponding to the high-confidence detection frame and the single detection frame respectively if the sum of the first distance and the preset distance threshold is less than the second distance, wherein the occupancy angle represents the angular range occupied by the detection frame in the field of view of the vehicle camera; and if the sum of the first distance and the preset distance threshold is greater than or equal to the second distance, then the occlusion ratio of the single detection frame is determined to be a preset minimum ratio.
[0134] Furthermore, the occlusion calculation module 802 is also used to calculate the first occupancy angle corresponding to the high confidence detection box and the second occupancy angle corresponding to the single detection box; determine the angle overlap interval between the high confidence detection box and the single detection box based on the first occupancy angle and the second occupancy angle; and calculate the occlusion ratio of the single detection box based on the angle overlap interval.
[0135] Furthermore, the primary filtering module 803 is also used to update the initial confidence of the individual detection box according to the occlusion ratio to obtain an updated confidence; and to perform a primary filtering on the original detection box set based on the updated confidence to obtain a first target detection box set.
[0136] Furthermore, the secondary filtering module 804 is also used to select an occluded detection box set from the original detection box set, wherein the occluded detection box set is a set of detection boxes whose occlusion ratio is greater than a preset ratio threshold; calculate the maximum intersection-union ratio (IUR) set between any detection box in the historical frame detection box set corresponding to the original detection box set and any detection box in the occluded detection box set; and perform secondary filtering on the original detection box set based on the maximum IUR set to obtain a second target detection box set.
[0137] Other embodiments or specific implementations of the target detection post-processing device of this application can be found in the above-described method embodiments, and will not be repeated here.
[0138] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.
[0139] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0140] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0141] The above are merely embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. A target detection post-processing method, wherein, The method includes the following steps: Target detection is performed on the input image to obtain an original set of detection boxes. The input image is a traffic scene image captured by a vehicle-mounted camera. Determine the occlusion ratio of a single detection box in the original detection box set, wherein the occlusion ratio represents the degree to which the target object contained in the single detection box is occluded by other objects; The original set of detection boxes is filtered once according to the occlusion ratio to obtain the first set of target detection boxes; The original detection box set is further filtered based on the historical frame detection box set corresponding to the original detection box set to obtain the second target detection box set.
2. The target detection post-processing method as described in claim 1, wherein, The step of determining the occlusion ratio of a single detection box in the original detection box set includes: Select high-confidence detection boxes from the original set of detection boxes; Determine the coordinates of the first point corresponding to the high-confidence detection box, the coordinates of the second point corresponding to a single detection box in the original detection box set, and the coordinates of the third point corresponding to the vehicle camera; The first distance between the high-confidence detection frame and the vehicle-mounted camera is calculated based on the first point coordinates and the third point coordinates; the second distance between the single detection frame and the vehicle-mounted camera is calculated based on the second point coordinates and the third point coordinates. The occlusion ratio of the individual detection frame is determined based on the first distance and the second distance.
3. The target detection post-processing method as described in claim 2, wherein, The step of determining the occlusion ratio of a single detection box based on the first distance and the second distance includes: If the sum of the first distance and the preset distance threshold is less than the second distance, the occupancy ratio of the single detection frame is determined according to the occupancy angles corresponding to the high confidence detection frame and the single detection frame, respectively. The occupancy angle represents the angular range occupied by the detection frame in the field of view of the vehicle camera. If the sum of the first distance and the preset distance threshold is greater than or equal to the second distance, then the occlusion ratio of the single detection box is determined to be the preset minimum ratio.
4. The target detection post-processing method as described in claim 3, wherein, The step of determining the occlusion ratio of a single detection frame based on the occupancy angles corresponding to the high-confidence detection frame and the single detection frame includes: Calculate the first occupancy angle corresponding to the high-confidence detection box and the second occupancy angle corresponding to the single detection box; The angular overlap range between the high-confidence detection frame and the individual detection frame is determined based on the first occupation angle and the second occupation angle. The occlusion ratio of a single detection box is calculated based on the angular overlap interval.
5. The target detection post-processing method as described in claim 1, wherein, The step of filtering the original detection box set according to the occlusion ratio to obtain the first target detection box set includes: The initial confidence of each detection box is updated based on the occlusion ratio to obtain the updated confidence. Based on the updated confidence level, the original set of detection boxes is filtered once to obtain the first set of target detection boxes.
6. The target detection post-processing method as described in claim 1, wherein, The step of performing a secondary filtering on the original detection box set based on the historical frame detection box set corresponding to the original detection box set to obtain the second target detection box set includes: Select an occluded detection box set from the original detection box set, wherein the occluded detection box set is a set of detection boxes whose occlusion ratio is greater than a preset ratio threshold; Calculate the set of maximum intersection-union ratios between any detection box in the historical frame detection box set corresponding to the original detection box set and any detection box in the occluded detection box set; The original set of detection boxes is further filtered based on the set of maximum intersection-union ratios to obtain a second set of target detection boxes.
7. A target detection post-processing device, wherein, The target detection post-processing device includes: The target detection module is configured to perform target detection on the input image to obtain an original set of detection boxes. The input image is a traffic scene image captured by a vehicle-mounted camera. The occlusion calculation module is configured to determine the occlusion ratio of a single detection box in the original detection box set, wherein the occlusion ratio represents the degree to which the target object contained in the single detection box is occluded by other objects; A primary filtering module is configured to perform a primary filtering on the original set of detection boxes based on the occlusion ratio to obtain a first set of target detection boxes. The secondary filtering module is configured to perform secondary filtering on the original detection box set based on the historical frame detection box set corresponding to the original detection box set, to obtain a second target detection box set.
8. A target detection post-processing device, wherein, The device includes: a memory, a processor, and a target detection post-processing program stored in the memory and executable on the processor, the target detection post-processing program being configured to implement the steps of the target detection post-processing method as described in any one of claims 1 to 6.
9. A storage medium, wherein, The storage medium is a computer-readable storage medium, and the storage medium stores a target detection post-processing program, which, when executed by a processor, implements the steps of the target detection post-processing method as described in any one of claims 1 to 6.
10. A computer program product, wherein, The computer program product includes a target detection post-processing program, which, when executed by a processor, implements the steps of the target detection post-processing method as described in any one of claims 1 to 6.