Redundant bounding box processing method for dense crowd and target detection method
By combining the density and nearest neighbor target evaluation redundancy scores in dense crowd scenes, and dynamically adjusting the redundancy box suppression strategy, the problem of misjudging redundant boxes in traditional NMS in dense scenes is solved, and higher detection accuracy is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHENZHEN MAXVISION TECH
- Filing Date
- 2023-09-26
- Publication Date
- 2026-06-09
AI Technical Summary
In dense crowd scenarios, traditional NMS methods are prone to mistaking neighboring non-identical detection boxes as redundant boxes, leading to missed detections. Existing technologies struggle to improve the processing accuracy of redundant detection boxes in dense object scenarios.
In dense crowd scenes, the redundancy score of each initial detection box is evaluated by combining the density and the nearest target, the accuracy of redundant detection boxes is recalculated, the probability of redundant boxes is measured by a Gaussian function, and the suppression strategy for redundant boxes is dynamically adjusted.
It improves the processing accuracy of redundant detection boxes in dense crowd scenes, reduces false positives and false negatives, and enhances the accuracy of target detection.
Smart Images
Figure CN117315580B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of digital image processing technology, and more specifically, to a method for processing redundant detection boxes in dense groups and a method for target detection. Background Technology
[0002] NMS stands for Non-maximum suppression. It's used in object detection as a post-processing stage to eliminate redundant, overlapping detection boxes of the same object. However, NMS distinguishes different objects solely based on the Intersection over Union (IoU) threshold. If the IoU between two overlapping detection boxes exceeds a pre-set threshold, they are considered to be detection boxes of the same object, and one is identified as a duplicate and deleted. This approach works well in general object detection tasks. However, it faces challenges in densely populated scenarios, where multiple different objects overlap, where the density varies greatly. When the object density between adjacent objects is high, the IoU between neighboring non-identical object detection boxes and the current object's detection may exceed the set threshold. NMS may mistakenly identify these neighboring non-identical object detection boxes as redundant detection boxes of the current object, thus suppressing them and causing missed detections. Summary of the Invention
[0003] The technical problem addressed by this application is to provide a method for processing redundant detection boxes in dense crowds. This method is more suitable for dense crowd scenarios than the traditional NMS method and can improve the processing accuracy of redundant detection boxes in dense crowd scenarios.
[0004] To address the aforementioned technical problems, in a first aspect, this application provides a method for processing redundant detection frames in dense crowds, comprising:
[0005] Select the current box from the multiple preliminary detection boxes;
[0006] Detect the density of the current bounding box and surrounding objects in a dense crowd scene; detect the nearest neighbor object to the current bounding box in a dense crowd scene; and,
[0007] The redundancy score of each initial detection box is evaluated by combining the density and the nearest target to the current box.
[0008] In the method for handling redundant detection boxes in dense crowds, compared to the traditional NMS method which crudely processes redundant boxes based on the intersection-union ratio between boxes, this method, when processing the redundancy of each preliminary detection box around the current box in a dense crowd scene, considers the density of surrounding targets and the nearest neighbor of the current box to re-evaluate the redundancy of each preliminary detection box around the current box. This results in a more accurate detection of redundant boxes in dense scenes, which is beneficial for improving the accuracy of target detection.
[0009] In one possible implementation, the redundancy score of each initial detection box is evaluated by combining the density and the nearest target to the current box, including:
[0010] When the overlap between a preliminary detection box and the current box is greater than or equal to the first threshold T1, the redundancy score S = S1*f(M,b) of each preliminary detection box is recalculated using the parameters of the target corresponding to the nearest target box and the parameters of the target corresponding to the current box. i ,d m The parameters for each target include its center position, length, and width.
[0011]
[0012]
[0013]
[0014] Where T2 represents the second threshold; σ represents the variance of the Gaussian function; M represents the current bounding box; b i c represents the initial detection box after recalculating the redundancy score; d represents the detection box of the nearest neighboring target to the current box; m This represents the density of the current bounding box relative to surrounding targets. and These represent the initial detection boxes b. i x: x-coordinate of the center position, y-coordinate of the center position, width, and length; c y c w c and h c b. The x-coordinate, y-coordinate, width, and length of the center position of the detection bounding box of the nearest target to the current bounding box; i|M Representative b i The coordinate distribution of μ relative to M M S1 represents the coordinate distribution of c relative to M; S1 is the score calculated using the NMS method.
[0015] In one possible implementation, when the dense group scene is a dense human scene, the second threshold T2 is set to 0.3.
[0016] In one possible implementation, the redundancy score of each initial detection box is evaluated by combining the density and the nearest target to the current box, including:
[0017] When the overlap between a preliminary detection box and the current box is less than the first threshold T1, the redundancy score of the preliminary detection box is S = S1, where S1 is calculated using the NMS method.
[0018] In one possible implementation, the first threshold T1 is set by setting a threshold T3 and a density d. m The larger of the two.
[0019] In one possible implementation, the threshold T3 is set to a value of 0.3.
[0020] In one possible implementation, the current box is selected from multiple preliminary detection boxes by selecting the current box in descending order of the category confidence obtained when detecting multiple preliminary detection boxes.
[0021] Secondly, this application also provides a target detection method, which includes:
[0022] A deep learning-based first-object detection network is used to perform object detection on dense crowd scene images, obtaining multiple preliminary detection boxes; and,
[0023] The method for handling redundant detection boxes in dense crowds processes multiple preliminary detection boxes to obtain the final target detection result of the dense crowd scene image.
[0024] In one possible implementation, the detection obtains the density and the detection obtains the nearest neighbor target, specifically including:
[0025] Construct a second object detection network: The second object detection network includes a feature extraction layer F1 that connects to the pooling layer of the first object detection network, a fully connected layer FC1 that connects to the feature extraction layer, and a detection head Head1 that is used to predict the nearest neighbor object of the current box and the density between the current box and surrounding objects;
[0026] Collect the second object detection network dataset: for each object labeled in the dense crowd scene image, the corresponding nearest neighbor object labeled for each object, and multiple preliminary detection boxes obtained by the first object detection network;
[0027] Multiple preliminary detection boxes obtained using labeled data and the first object detection network are used as training data for the second object detection network. The labeled data includes the data labels of the detected objects and the data labels of the nearest neighbor objects of each detected object.
[0028] Smooth-L1 is used as the loss function during training. Attached Figure Description
[0029] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0030] Figure 1 A flowchart of a method for handling redundant detection boxes in dense groups according to an embodiment of this application; a schematic diagram of the structure of the detection network E used in the target detection method.
[0031] Figure 2 This is a flowchart illustrating the method for handling redundant detection frames in dense groups according to an embodiment of this application. Detailed Implementation
[0032] To make the technical problems, technical solutions, and beneficial effects to be solved by this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this application.
[0033] It should be noted that when a component is referred to as being "fixed to" or "set on" another component, it can be directly on or indirectly on that other component. When a component is referred to as being "connected to" another component, it can be directly connected to or indirectly connected to that other component.
[0034] It should be understood that the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.
[0035] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.
[0036] The method for handling redundant detection frames for dense groups and the target detection method of this application are now described in conjunction with the accompanying drawings.
[0037] The object detection method includes: using a first object detection network based on deep learning to detect objects in a dense crowd scene image, obtaining multiple preliminary detection boxes; and using a redundant detection box processing method for dense crowds to process the multiple preliminary detection boxes into redundant boxes, obtaining the final object detection result for the dense crowd scene image. The object detection method provided in this application uses this object detection method to process the detected object detection boxes into redundant boxes, and is applicable to object detection in dense crowd scene images. The dense crowd may include, but is not limited to, dense crowds, dense vehicles, and other dense object objects.
[0038] Please refer to Figure 1 , Figure 1 This is a schematic diagram of the detection network E used in the target detection method. The detection network E includes a first target detection network and a second target detection network.
[0039] Specifically, the first object detection network performs preliminary object detection on dense crowd scene images, obtaining multiple preliminary detection boxes. This first object detection network includes a backbone network for extracting features to obtain feature maps, an RPN network structure for generating sensing areas, pooling layers, and a detection head that outputs multiple preliminary detection boxes. The backbone network can be, but is not limited to, ResNet-50 or Darknet53. The pooling layers can use RoI Align pooling. The detection head includes two parallel full-connection layer branches. One full-connection layer branch outputs each preliminary detection box, including the x-coordinate of the detection box's center position, the y-coordinate of the detection box's center position, the length of the detection box, and the height of the detection box. The other full-connection layer branch connects to an activation layer and outputs a class confidence score.
[0040] The second object detection network is used to detect the nearest neighbor of the selected current bounding box from multiple preliminary detection boxes, as well as the density of the current bounding box with surrounding objects. It's worth noting that in dense scenes, redundant detection boxes are unavoidable in the multiple preliminary detection boxes. To suppress redundant boxes, a current bounding box is selected from the preliminary detection boxes for consideration. Using the current bounding box as the object, the nearest neighbor of the current bounding box and the density of the current bounding box with surrounding objects are detected. This allows subsequent re-evaluation of whether other preliminary detection boxes are redundant based on the nearest neighbor and density. Taking a dense crowd scene image as an example, human targets in this scene image exhibit a relatively high density, i.e., overlap. The current bounding box selects a human target A, but the nearest neighbor of the current bounding box is not the same human target A, and the nearest neighbor is not the target selected by the redundant bounding box during the detection of human target A. The dataset used for the second object detection network includes each target labeled in the dense crowd scene image, the corresponding nearest neighbor of each labeled target, and multiple preliminary detection boxes obtained by the first object detection network. The density of the current bounding box and surrounding targets is determined by the intersection-union ratio (IUU) between the current bounding box and the detection boxes of the closest surrounding targets. That is, the density of the current bounding box and surrounding targets is the maximum IUU between the current bounding box and the detection boxes of surrounding targets. The detection box mentioned in this application refers to the rectangular box used to select targets during target detection.
[0041] Specifically, the second object detection network includes a feature extraction layer F1 connected to the pooling layer of the first object detection network, a fully connected layer FC1 connected to the feature extraction layer, and a detection head Head1 for predicting the nearest neighbor object of the current box and the density of the current box and surrounding objects. The feature extraction layer F1 includes two conv+relu modules for further extracting more detailed features. Each conv+relu module includes a convolutional layer and an activation layer connected in sequence; the convolutional layer can be multiple convolutional layers. The fully connected layer FC1 includes a 1024-dimensional full connection layer, a relu layer, a full connection layer, and a relu layer distributed sequentially. The detection head Head1 connected to the fully connected layer FC1 includes two parallel full connection layer branches. One full connection layer branch outputs information about the nearest neighbor object for each selected current box, including the x-coordinate of the center position of the nearest detection box, the y-coordinate of the center position of the detection box, the width of the detection box, and the height of the detection box. The other full connection layer branch connects to the activation layer and outputs the density of each selected current box.
[0042] Furthermore, after using the redundancy detection box processing mechanism of dense groups to suppress redundancy detection boxes of multiple preliminary detection boxes, the final target detection result is output.
[0043] Please refer to Figure 2 The method for processing redundant detection frames in dense populations provided in this application includes the following processing steps:
[0044] Step S100: Select the current box from multiple preliminary detection boxes; these multiple preliminary detection boxes are obtained by performing preliminary target detection in a dense crowd scene image, that is, these multiple preliminary detection boxes have not yet undergone redundant box processing.
[0045] Step S200: Detect the density of the current bounding box and surrounding targets in a dense crowd scene, and detect the nearest target to the current bounding box in a dense crowd scene.
[0046] Step S300: Evaluate the redundancy score of each preliminary detection box by combining the density and the nearest target to the current box.
[0047] For step S100: the method for selecting the current box from multiple preliminary detection boxes is as follows: the current box is selected in descending order of the class confidence scores obtained when detecting multiple preliminary detection boxes. Since the detection box with the highest class confidence score is the most likely target object to be detected, the detection box with the highest class confidence score is processed first as the current box, that is, redundant detection box suppression processing is performed first on the detection boxes with high class confidence scores.
[0048] For step S200, the density of the current bounding box with surrounding targets and the nearest target of the current bounding box can be obtained using the detection network E used in the above target detection method, that is, specifically through the second target detection network.
[0049] For step S300: Evaluate the redundancy score of each preliminary detection box by combining the density and the nearest target to the current box, including the following steps S310 and S320.
[0050] In step S310: when the overlap between a preliminary detection box and the current box is greater than or equal to the first threshold T1, the redundancy score S = S1 * f(M, b) of each preliminary detection box is recalculated using the parameters of the target corresponding to the nearest target box and the parameters of the target corresponding to the current box. i ,d m The parameters for each target include its center position, length, and width; the score is calculated using the NMS method. The overlap between each preliminary detection box and the current box is the intersection-union ratio (IUU) of the preliminary box and the current box.
[0051] In step S310 above,
[0052] in,
[0053] in,
[0054] In the above formula, T2 represents the second threshold; σ represents the variance of the Gaussian function; M represents the current bounding box; b i c represents the initial detection box after recalculating the redundancy score; d represents the detection box of the nearest neighboring target to the current box; m This represents the density of the current bounding box relative to surrounding targets. and These represent the initial detection boxes b. i The x-coordinate of the center position, the preliminary detection box b i Center position, center ordinate, preliminary detection box b i Width and initial detection frame b i The length of x; c y c w c and h c b. The x-coordinate of the center position of the detection box c closest to the current bounding box, the y-coordinate of the center position of detection box c, the width of detection box c, and the length of detection box c; i|M Representative b i The coordinate distribution of μ relative to M M This represents the coordinate distribution of c relative to M. The first and second thresholds mentioned above are both between 0 and 1. In one specific embodiment, when the dense group scene is a dense human scene, the second threshold T2 is set to 0.3.
[0055] It is worth noting that the above f(M,b) i ,d m The formula uses a Gaussian function to measure the probability that each initial detection box corresponds to a nearby target or a redundant box in the current box. According to the above formulas (1) to (3), when ||b i|M -μ M When the values of || are close, it indicates that the initial detection box b i The closer f(M,b) is to the detection box c, the better. i ,d m The larger the value of b, the larger the score S, i.e., the higher the initial detection box value. i The higher the probability that the target in the current bounding box is another target nearby, the higher the probability of the initial detection bounding box b. i The lower the probability of a redundant detection box, the better the initial detection box b is. i It should not be suppressed; when ||b i|M -μ M The larger the difference in the values of ||, the more likely it is that the initial detection box b is...i The greater the difference between f(M,b) and the detection box c, the greater the difference between f(M,b) and the detection box c. i ,d m The smaller the value of b, the smaller the score S, i.e., the smaller the initial detection box b. i The lower the probability that the target in the current bounding box is another target nearby, the better the initial detection bounding box b is. i The higher the probability of a redundant detection box b, the higher the probability of the initial detection box b. i These should be suppressed. Therefore, in evaluating whether a detection box is redundant, this embodiment does not use cross-union ratio (CUI) to crudely suppress redundant boxes as in traditional NMS, but instead uses dynamic suppression for each preliminary detection box. Further, in step S310, when the score S of a preliminary detection box is less than a set third threshold, the preliminary detection box is deleted and not output; when the score S of a preliminary detection box is greater than or equal to the set third threshold, the preliminary detection box is retained and output. Taking a dense group of people as an example, the third threshold can be 0.5.
[0056] In step S320: when the overlap between a preliminary detection box and the current box is less than a first threshold T1, the redundancy score S of the preliminary detection box is S1, where S1 is the score calculated using the NMS method. The overlap between each preliminary detection box and the current box is the intersection-union ratio (IUU) of the two boxes. Further, in step S320, the original NMS method is used to process redundant detection boxes.
[0057] In steps S310 and S320 above, the first threshold T1 is set to a threshold T3 and a density d. m The larger of the two. The threshold T3 is set to 0.3.
[0058] It is worth noting that, considering the density d between the current bounding box and other surrounding targets... m The current bounding box is relatively small, but there are multiple and significant overlaps between it and the surrounding preliminary detection bounding boxes. This also takes into account the density of the current bounding box and other surrounding targets. m The current bounding box is relatively large, but there are multiple and significant overlaps between it and the surrounding preliminary detection bounding boxes. First threshold, second threshold, setting threshold, and density d. m Setting relationships and rules is beneficial for improving the fault tolerance rate in handling redundant detection boxes.
[0059] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A method for processing redundant detection frames in dense crowds, characterized in that, include: Select the current box from the multiple preliminary detection boxes; Detect the density of the current bounding box and surrounding objects in a dense crowd scene, and detect the nearest neighbor of the current bounding box in a dense crowd scene; as well as The redundancy score of each initial detection box is evaluated by combining the density and the nearest target to the current box, including: When the overlap between a preliminary detection box and the current box is greater than or equal to the first threshold T1, the redundancy score of each preliminary detection box is recalculated using the parameters of the target corresponding to the nearest target box and the parameters of the target corresponding to the current box. The parameters for each target include its center position, length, and width; ； ； ； in, Represents the second threshold; The variance of the Gaussian function is represented by ; M represents the current bounding box. 'c' represents the initial detection box after recalculating the redundancy score; 'c' represents the detection box of the nearest target to the current box. This represents the density of the current bounding box relative to surrounding targets. , , and These represent the preliminary detection frames. The x-coordinate of the center position, the y-coordinate of the center position, the width, and the length; , , and The x-coordinate, y-coordinate, width, and length of the center position of the detection box of the nearest target to the current box; represent The coordinate distribution relative to M, S1 represents the coordinate distribution of c relative to M; S1 is the score calculated using the NMS method.
2. The method for processing redundant detection frames in dense groups as described in claim 1, characterized in that, When the dense crowd scene is a dense human scene, set a second threshold. The value is 0.
3.
3. The method for processing redundant detection frames in dense groups as described in claim 1, characterized in that, The redundancy score of each initial detection box is evaluated by combining the density and the nearest target to the current box, including: When the overlap between a preliminary detection box and the current box is less than the first threshold T1, the redundancy score of the preliminary detection box is S=S1, where S1 is calculated using the NMS method.
4. The method for processing redundant detection frames in dense groups as described in any one of claims 1 to 3, characterized in that, The first threshold T1 is set to the threshold T3 and the density level. The larger of the two.
5. The method for processing redundant detection frames in dense groups as described in any one of claims 4, characterized in that, The threshold T3 is set to 0.
3.
6. The method for processing redundant detection frames in dense groups as described in claim 1, characterized in that, The method for selecting the current box among multiple preliminary detection boxes is as follows: select the current box in descending order of the category confidence obtained when detecting multiple preliminary detection boxes.
7. A target detection method, characterized in that, The first object detection network of deep learning is used to perform object detection on dense crowd scene images to obtain multiple preliminary detection boxes; as well as The redundant detection box processing method for dense crowds as described in any one of claims 1 to 5 is used to process multiple preliminary detection boxes to obtain the final target detection result of the dense crowd scene image.
8. The target detection method as described in claim 7, characterized in that, Detection of density and detection of nearest neighbor targets, specifically including: Construct a second object detection network: The second object detection network includes a feature extraction layer F1 that connects to the pooling layer of the first object detection network, a fully connected layer FC1 that connects to the feature extraction layer, and a detection head Head1 that is used to predict the nearest neighbor object of the current box and the density between the current box and surrounding objects; Collect the second object detection network dataset: for each object labeled in the dense crowd scene image, the corresponding nearest neighbor object labeled for each object, and multiple preliminary detection boxes obtained by the first object detection network; Multiple preliminary detection boxes obtained using labeled data and the first object detection network are used as training data for the second object detection network. The labeled data includes the data labels of the detected objects and the data labels of the nearest neighbor objects of each detected object. Smooth-L1 is used as the loss function during training.