Dense region target detection method based on density cascade clustering

Abstract

The present application relates to a kind of dense area target detection algorithm based on density cascade clustering.The detection network is composed of feature extraction network, feature classification network and feature regression network part, the improvement of the present application is mainly after the density area division of candidate frame to feature extraction network.The foreground frame obtained in RPN is clustered in density, then it is regressed to the second density area in cascade to obtain three density areas and threshold, and the NMS threshold is adaptively learned in three areas.Make the network have spatial attention mechanism, distinguish the instance with high density area high overlap area, and divide different NMS threshold according to the density area learned, so that the network reduces the situation of missing detection for high dense high overlap area.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of target detection, and particularly relates to a dense area target detection method based on density cascade clustering. BACKGROUND

[0002] The existing two-stage target detection algorithm can generate a large number of candidate boxes close to the true value box, and thus needs to use a non-maximum suppression method (NMS) for candidate box post-processing. However, if the NMS threshold is too high, a large number of false candidate boxes are generated, and if the NMS threshold is too low, candidate boxes with high overlap are'suppressed' and lost. The existing model is relatively rough in the first regression stage, resulting in a large number of candidate boxes entering the second regression stage, and a large number of candidate boxes belonging to the same detection target, causing the candidate boxes of adjacent targets in a local area to have large bias values. The network is prone to cause adjacent candidate boxes to be in local oscillation in the second regression, thereby reducing the reasoning speed of the overall network and the AP of the detection box. SUMMARY

[0003] The application aims to provide a dense area target detection algorithm based on density cascade clustering. The foreground box obtained in the RPN is subjected to density clustering through a density clustering RPN algorithm, and then is regressed to a second density area in a cascade manner to obtain three density areas and threshold values, and the NMS threshold values are adaptively learned in the three areas.

[0004] The technical scheme adopted by the application to achieve the above-mentioned purpose is as follows:

[0005] The dense area target detection method based on density cascade clustering comprises the following steps:

[0006] An image to be detected is obtained, and a target is labeled with a bounding box in the image as a training label;

[0007] A VGG network is used to extract features of the image to be detected;

[0008] The foreground box and the background box in the features are identified based on the RPN network, and the foreground box is subjected to bounding box regression to obtain NMS threshold values of different density areas as suggestions;

[0009] The suggestions output by the RPN network are mapped to a feature map of the last layer of a convolutional neural network, and a fixed-length suggestion is generated based on ROIpooling. The suggestion is subjected to a second classification and bounding box regression, and the finally obtained bounding box is taken as a detected target.

[0010] The bounding box regression of the foreground box is specifically as follows: the foreground box obtained in the RPN network is subjected to density clustering to obtain a first density area, and then is regressed to a second density area through a density cascade clustering algorithm to obtain three density areas and threshold values, and the NMS threshold values are adaptively learned in the three areas.

[0011] In the density clustering, a density area is searched in the ɛ neighborhood of the instance by the following formula:

[0012]

[0013] wherein, , represents the center point of the foreground frame conversion generated by the input image through one regression, D represents the entire instance frame coordinate set, represents the sample quantity in the sub-sample set, represents the Euclidean distance of two instance points, represents the radius of the density space.

[0014] The sample quantity in the sub-sample set The condition for the density area is:

[0015]

[0016] wherein, represents the sub-dense area radius.

[0017] The density cascade clustering algorithm is specifically: multiple density areas are divided by K-level regression, and the foreground frames of different density areas are sent into different threshold non-maximum suppression (NMS) for calculation, and the formula is:

[0018]

[0019] wherein, represents the density area of the level regression, represents the first density area, represents the second density area.

[0020] The ROIpooling judges whether the training is ended through the loss function, stops the training when the loss function is the minimum, and outputs the current proposal, and the loss function is:

[0021]

[0022] wherein, , respectively represent the total number of classification loss samples and the total number of regression samples, is a loss balancing coefficient, represents a sample real category value, represents a sample real foreground frame, represents a sample predicted category value, represents a sample predicted foreground frame, , respectively, denote cross-entropy classification loss and L2 regression loss, when p=1 denotes that the sample classification is correct, p=0 denotes that the sample classification is incorrect, t is the foreground frame offset.

[0023] The dense region target detection system based on density cascade clustering comprises:

[0024] An image acquisition module is configured to acquire a to-be-detected image and label a target in the image using a boundary frame as a training label.

[0025] A feature extraction module is configured to extract features of the to-be-detected image using a VGG network.

[0026] A density clustering module is configured to identify foreground frames and background frames in the features based on an RPN network, and perform boundary frame regression on the foreground frames to obtain NMS thresholds of different density regions as proposals.

[0027] A target recognition module is configured to map the proposals output by the RPN network to feature maps of a last layer of a convolutional neural network, generate fixed-length proposals based on ROI pooling, perform second classification and boundary frame regression on the proposals, and finally obtain boundary frames as detected targets.

[0028] The dense region target detection system based on density cascade clustering comprises a memory and a processor; the memory is configured to store a computer program; and the processor is configured to implement the dense region target detection method based on density cascade clustering when executing the computer program.

[0029] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the dense region target detection method based on density cascade clustering is implemented.

[0030] The present application has the following advantages and benefits:

[0031] 1. The present application divides the candidate frame into density regions, so that the network has a spatial attention mechanism, distinguishes instances with high density regions and high overlap regions, and divides different NMS thresholds according to the learned density region, so that the network reduces the missed detection of high dense and high overlap regions.

[0032] 2. The present application refines the selection process of the foreground frame in the first regression, so that the network learns the 'density distribution' based on the data set, thereby realizing a more refined local inhibition process in the first regression.

[0033] 3. The application proposes a Dense-RPN structure, which divides dense areas according to target density, so that the network pays attention to difficult-to-detect areas.

[0034] 4. The application does not increase network parameters, and has more obvious precision advantage under the same inference speed.

[0035] 5. The application achieves the most advanced results on the RailwayPole and CrowdHuman data sets. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 A dense density cascade clustering detection network structure diagram is provided.

[0037] Figure 2 An anchor frame point density cascade clustering visualization schematic diagram is provided.

[0038] Figure 3 is a schematic diagram of the RailwayPole data set compared with other data sets, (a) instance absolute size scale (b) instance relative size scale.

[0039] Figure 4 A comparison diagram of detection network results in different scenes is provided. DETAILED DESCRIPTION

[0040] The application will be further described in detail below in combination with the drawings and examples.

[0041] As shown in the drawings, Figure 1 The application provides a dense area target detection method based on density cascade clustering, which particularly relates to pedestrian detection and pole detection in dense area target detection. The method comprises the following steps:

[0042] 1) Collect high-resolution videos from the same camera at the same angle. Sample an image from the video every 2 frames, and finally manually annotate 10841 pole boundary boxes.

[0043] 2) Use the VGG network for feature extraction.

[0044] 3) Identify foreground and background frames based on RPN, and perform boundary box regression on the foreground frames. According to the foreground frames obtained in RPN, perform density clustering, and then regress to the second density area in a cascading manner to obtain three density areas and threshold values, and adaptively learn NMS threshold values in the three areas.

[0045] 4) Map the RPN output proposal to the feature map of the last layer of the convolutional neural network, generate a fixed-length proposal based on ROIpooling, and perform second classification and boundary box regression on the proposal.

[0046] Further, in the instance ɛThe formula for searching the dense area in the field is:

[0047] (1)

[0048] wherein , represents the center point of the foreground frame conversion generated by the input image through one regression, D represents the entire instance frame coordinate set, represents the number of samples in the sub-sample set, represents the Euclidean distance of two instance points.

[0049] Further, the density cascade clustering algorithm sends the instance anchor frame of different areas into different threshold non-maximum suppression NMS for calculation through the density area divided by the network, and the formula is:

[0050] (2)

[0051] Further, the loss function formula of the training process is:

[0052] (3)

[0053] wherein , respectively represent the total number of classification loss samples and the total number of regression samples, is a loss balancing coefficient, when p=1 represents that the sample classification is correct, p=0 represents that the sample classification is incorrect, t is the anchor frame offset.

[0054] The density clustering algorithm is based on the density of instances, and the instances are divided into dense clusters and non-dense clusters through the density of sample distribution, and the dense area in the field is searched. ɛ Firstly, different initial center points are set, which will produce distance clusters according to different Euclidean distance values. ɛ Secondly, the neighborhood size of the center point is selected to determine the minimum iteration density space. ɛ If a neighborhood containing N samples is found in the search space, the point is set as the core object, and the point and the initial point are called density direct points. The density directness only satisfies the transitivity, but not the symmetry. Therefore, the density direct always searches in the direction of increasing density of data samples. Searching the dense area in the direction of the maximum density will also have certain problems. When there is no other density direct point around the dense point, the current search stage stops, and the dense area search of the next initialization point starts, which leads to that the search result meets the local optimal solution. Therefore, the density cascade clustering algorithm is proposed to weaken the influence of local density imbalance on the clustering result.

[0055] The condition for the density region is:

[0056]

[0057]

[0058] wherein is the sub dense region radius

[0059] The density clustering algorithm is applied to the post-processing of the network head RPN to divide the instance candidate frame result set of the network once regression. The density cascade clustering algorithm sends the instance anchor frame of different regions into the non-maximum suppression NMS of different thresholds for calculation through the density region divided by the network.

[0060] Figure 2 The process of the dense region target detection method of the density cascade clustering.

[0061] Fig. 3 and Figure 4 The detection results of the dense region target detection algorithm based on the density cascade clustering are used to illustrate the application:

[0062] The density cascade clustering algorithm classifies the primary regression anchor frame according to the distribution density, and the effectiveness of the experiment is proved, Figure 3a and Figure 3b The difference between the cascade density clustering and the traditional clustering is visualized. According to the classification of different density regions, the non-maximum suppression threshold negatively related to the density is used, so that the loss of anchor frame caused by the mutual suppression of instances with high overlap of the same class due to the method of taking fixed threshold of traditional non-maximum suppression algorithm NMS is greatly reduced. Due to the foreign object occlusion and multi-scenarization of the wire pole in RailwayPole, the detection is more difficult, and data support is provided to prove the effectiveness of the algorithm. The results of the most advanced single-stage detector and the double-stage detector with the added Dense-RPN module on the RailwayPole dataset are shown in Table 1, which shows the effectiveness of the Dense module.

[0063] Table 1 Experimental results of RailwayPole double-stage detection network

[0064] Two-stage detection network Precision Recall Difference Difference Dense-RPN 78.86% 86.30% Libra-faster 79.51% 77.10% -1.29 +9.2 ga-faster 77.6% 81.40% +1.26 +4.9 dh-faster 78.41% 79.20% +0.45 +7.1 Cascade-rcnn 76.24% 84.80% +2.62 +1.5

[0065] Figure 4The detection network result comparison chart under different scenes is visualized, wherein the first row is a Dense-RPN detection network result; the second row is an RPN detection network result, and the dense module has a significant improvement on the detection result of the power pole in the dense area. The result of the two-stage detector of the Dense-RPN module on the Crowdhuman dataset is shown in Table 2, and the Dense-RPN module exceeds the basic framework RPN module by 1.4% precision and 2.11% recall on the CrowdHuman dataset, proving the effectiveness of the present application.

[0066] Table 2 Experimental results of the two-stage detection module of Crowdhuman

[0067] Module Precision Recall Dense-RPN 85.21% 93.80% Adaptive-NMS [9] ]] 91.27% 84.71% RPN 83.81% 91.69%

[0068] The density cascade clustering module Dense-RPN refines the first regression result that is most easily ignored by the two-stage detector, and re-scores the one-time regression result according to the density, that is, the higher the density of the area, the higher the score, and the lower the score of the sparse area. According to the area density, the area is divided into three different areas, and the network is given the judgment ability in the NMS stage. According to the area, the candidate frame is sent to the NMS score of different thresholds for sorting and frame deletion, which can effectively reduce the network because two different instances are too close and are mistakenly judged as the same candidate frame by the network, so as to improve the recall rate of the network. From the experimental results, it can be seen that the present application has effectiveness.

[0069] The present application adopts the two-stage detector Faster-Rcnn as the basic detection network framework, designs the Dense-RPN module to better regress the detection target, and exceeds the basic framework by 0.71% precision and 12.2% recall on the RailWayPole dataset, and exceeds the basic framework by 1.4% precision and 2.11% recall on the CrowdHuman dataset, proving the effectiveness of the present application. Compared with the latest target detection method, the present application also has different degrees of superiority.

Claims

1. A dense region object detection method based on density cascade clustering, characterized in that, The method comprises the following steps: obtaining an image to be detected and marking a target in the image using a bounding box as a training label; extracting features of the image to be detected using a VGG network; identifying foreground frames and background frames in the features based on an RPN network and performing bounding box regression on the foreground frames to obtain NMS thresholds of different density regions as suggestions; mapping the suggestions output by the RPN network to feature maps of the last layer of a convolutional neural network, generating fixed-length suggestions based on ROI pooling, performing second classification and bounding box regression on the suggestions, and obtaining the final bounding boxes as detected targets; the bounding box regression on the foreground frames comprises: performing density clustering on the foreground frames obtained in the RPN network to obtain a first density region, and then performing regression to a second density region through a density cascade clustering algorithm to obtain three density regions and thresholds, and adaptively learning NMS thresholds in the three regions; the density cascade clustering algorithm comprises: dividing multiple density regions through K-level regression, and sending foreground frames in different density regions into non-maximum suppression NMS with different thresholds for calculation, and the formula is: ； wherein represents a density region of a level regression, represents a first density region, represents a second density region.

2. The dense level-based cascade clustering method for dense region object detection according to claim 1, characterized in that, in the density clustering, the density regions are searched in the ɛ neighborhood of instances through the following formula: ； wherein, , denotes the center point of the foreground frame conversion generated by the input image through one regression, D denotes the entire instance frame coordinate set, denotes the number of samples in the sub-sample set, denotes the Euclidean distance of two instance points, denotes the radius of the density space.

3. The dense level-based cascade clustering method for dense area object detection according to claim 2, characterized in that, Number of samples in a sub-sample set The condition for being a density region is that: ； wherein denotes the sub-dense region radius.

4. The dense level-based cascade clustering method for dense region object detection according to claim 1, wherein, the ROI pooling judges whether the training is completed through a loss function, stops the training when the loss function is minimum, and outputs the current suggestions, and the loss function is: ； wherein, , respectively represent the total number of classification loss samples and the total number of regression samples, is a loss balance coefficient, represents a sample real class value, represents a sample real bounding box, represents a sample predicted class value, represents a sample predicted bounding box, , respectively represent a cross-entropy classification loss and a two-norm regression loss, when p=1 represents that a sample classification is correct, p= 0 represents that a sample classification is incorrect, t is a bounding box offset amount.

5. A dense region object detection system based on density cascade clustering, for implementing the dense region object detection method based on density cascade clustering as claimed in claim 1, characterized in that, comprises: an image acquisition module configured to obtain an image to be detected and mark a target in the image using a bounding box as a training label; a feature extraction module configured to extract features of the image to be detected using a VGG network; a density clustering module configured to identify foreground frames and background frames in the features based on an RPN network and perform bounding box regression on the foreground frames to obtain NMS thresholds of different density regions as suggestions; a target identification module configured to map the suggestions output by the RPN network to feature maps of the last layer of a convolutional neural network, generate fixed-length suggestions based on ROI pooling, perform second classification and bounding box regression on the suggestions, and obtain the final bounding boxes as detected targets.

6. A dense region object detection system based on density cascade clustering, characterized in that, comprises a memory and a processor; the memory is configured to store a computer program; and the processor is configured to implement the dense region target detection method based on the density cascade clustering when the computer program is executed.

7. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and when the computer program is executed by the processor, the dense region target detection method based on the density cascade clustering is implemented.

Images