A single-stage object detection method using anchor-free module and enhanced classifier
By combining the anchor-free frame module and enhanced classifier, the problem of anchor frame affecting positioning accuracy and high calculation cost in the existing technology is solved, and a robust and fast target detection is achieved, which improves the YOLOv3 recall rate and constructs a more robust Feature representation.
Patent Information
- Application Number
- CN202010677995.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-07-14
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2040-07-14
AI Technical Summary
In the existing object detection technology, the attributes of the anchor frame affect the positioning accuracy, the calculation cost of two-stage detectors is high, the accuracy of the single-stage detector is low, and the ability to handle small-size objects is limited, resulting in object omissions in dense scenes.
A single-stage object detection method combining anchor-free frame module and enhanced classifier is adopted. By embedding anchor-free frame module, designing a decoupling control method and training a cascade classifier, a robust feature representation is constructed to improve detection accuracy and speed.
It realizes robust and fast object detection, improves YOLOv3 recall, avoids the exclusion of redundant bounding boxes, suppresses error detection, and constructs a more robust feature representation, solving the problems of inaccurate positioning and high computing costs.
Smart Images

Figure CN111814889B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of target detection, and in particular to a single-stage target detection method using an anchor-free frame module and an enhanced classifier. Background Art
[0002] In the past few years, convolutional neural network (CNN) based object detection methods have achieved remarkable progress in many applications, such as autonomous driving, face detection and security monitoring.
[0003] Previous mainstream object detection methods usually generate anchor boxes first, then perform bounding box classification and regression, and perform two-stage reasoning. Although these two-stage detectors have shown superior performance, they still have some disadvantages. First, the properties of the anchor boxes (such as size and aspect ratio) will greatly affect the accuracy of target positioning. Second, classification, regression, and post-processing (NMS) in hundreds of bounding boxes will incur more computational costs. These shortcomings limit the application of two-stage detectors in many scenarios that require fast detection.
[0004] When anchor-based methods dominated mainstream detection algorithms, the original YOLO model established a new anchor-free method by splitting the input image into S×S grids. If the center of an object falls in a grid, this grid is responsible for predicting the object. Although each grid can predict two bounding boxes, only the bounding box with the highest IoU is selected as the detection output, that is, each grid can only predict at most one object. For a grid with multiple small objects, only one of them can be detected. YOLOv2 introduced anchor boxes and other training strategies (e.g., batch normalization) to improve the network's robustness for small target detection. However, using a single-scale feature map for detection does not work well for objects of multiple scales. The recent YOLOv3 model designed a more robust backbone network (DarkNet53), in which three levels of features are fused from top to bottom to detect large, medium, and small objects respectively.
[0005] Recently, some studies have proposed to build a single-stage detector that directly outputs the confidence score and position coordinates of the target without the need for a region proposal stage. Since the final detection result can be predicted directly on the feature map, the speed of the single-stage detector is greatly accelerated. However, due to the extreme imbalance of positive and negative training samples, the single-stage detection algorithm often cannot achieve high accuracy. In addition, the single-stage detector has limited ability to handle small-sized objects, resulting in object omissions in some dense scenes.
[0006] Some recent works have been proposed to leverage the advantages of anchor-free methods to enhance the original anchor-based models to obtain more robust results. For example, FSAF integrates an external anchor-free prediction layer with RetinaNet to achieve more accurate results without incurring too much computational cost. In order to get rid of the complex anchor design and achieve a higher recall rate, AF-RPN replaces the original anchor-based RPN in FasterR-CNN with an anchor-free module that detects more scene instances of different shapes.
[0007] The anchor-free framework can be divided into pixel estimation and keypoint estimation. FCOS uses all points in the correct annotation value to predict coordinates and suppresses those low-quality bounding boxes through a center branch. FoveaBox only uses a few pixels around the center of the correct annotation value for classification and regression. The above two pixel-based detectors are similar to anchor-based detectors, except for the difference in bounding box regression. Keypoint-based detection algorithms propose key points to detect targets. For example, CornerNet detects key points (top left and bottom right corners) and groups a pair of corner points belonging to the same object through an embedding vector. ExtremeNet detects four extreme points (top, left, bottom, right) and a center point, and groups the five key points into a bounding box through a matching algorithm. However, the complex grouping after keypoint detection slows down the speed of these two detectors. To solve this problem, CenterNet proposes to directly predict the center point as well as the height and width of each object, allowing it to run very fast.
[0008] In summary, existing object detection technologies have the following defects: First, in two-stage detectors, the properties of anchor boxes (such as size and aspect ratio) will greatly affect the accuracy of object positioning. Moreover, classification, regression, and post-processing (NMS) in hundreds of bounding boxes will incur more computational costs; second, due to the extreme imbalance of positive and negative training samples, the detection algorithm of a single-stage detector often cannot achieve high accuracy; third, the single-stage detector has limited ability to handle small-sized objects, resulting in object omissions in some dense scenes. Summary of the invention
[0009] The technical problem to be solved by the present invention is to overcome the deficiencies in the prior art and provide a single-stage target detection method using an anchor-free frame module and an enhanced classifier.
[0010] To achieve the above object, the present invention provides the following technical solution: a single-stage target detection method using an anchor-free frame module and an enhanced classifier, comprising the following steps:
[0011] a. Embedding the anchor-free module: Use the backbone network to extract feature maps, embed the anchor-free module on the feature maps, merge the prediction results of the backbone network and the anchor-free module, and perform post-processing;
[0012] b. Design a decoupling control method: construct a decoupling prediction model and embed a feature enhancement module;
[0013] c. Train enhanced classifiers: set the number of cascade classifiers, define the positive and negative sample loss functions, assign weights to different classifiers, and train the classifiers.
[0014] Preferably, after extracting the feature map in step a, a lightweight network is introduced to enhance the feature map, and the transposed convolution of the network is replaced with bilinear interpolation to reduce the model size.
[0015] Preferably, the feature map described in step a includes four scales, the first three layers of feature maps are used for backbone network prediction, and the fourth layer of feature maps is used for anchor-free frame module prediction.
[0016] Preferably, the decoupled prediction model in step b decomposes the original prediction layer into two independent branches. Four convolutions with a fixed number of channels are used in each branch.
[0017] Preferably, the feature enhancement module in step b consists of three convolutional layers, and the output of the feature enhancement module is fed back to another branch for classification.
[0018] Preferably, the number of cascade classifiers in step c is set to 6.
[0019] Preferably, the weight parameters of each layer of classifiers in step c satisfy an increasing relationship, indicating that the penalty for the classifier needs to be gradually increased. All these classifiers participate in the training, but only the last classifier is used during inference.
[0020] Compared with the prior art, the present invention has the following beneficial effects:
[0021] The present invention combines the advantages of anchor-free and anchor-based methods to propose a robust and fast single-stage object detection method. The present invention proposes a hybrid method that combines the anchor-free module with the YOLO prediction branch to generate more robust predictions and solve the problem of low recall rate of YOLOv3. The present invention proposes an enhanced classifier that uses multiple cascade classifiers to gradually extract redundant bounding boxes. The enhanced classifier can avoid certain key bounding boxes from being excluded before post-processing and suppress false detections. On this basis, the present invention proposes a decoupling method to deal with the problem of inaccurate positioning and proposes a feature enhancement module to construct a more robust feature representation. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1is a network structure diagram of the present invention;
[0023] Figure 2 It is a decoupling method diagram of the present invention;
[0024] Figure 3 It is a structural diagram of the enhanced classifier of the present invention. DETAILED DESCRIPTION
[0025] The technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0026] like Figure 1-3 As shown, the present invention provides a technical solution: a single-stage target detection method using an anchor-free frame module and an enhanced classifier, comprising the following steps:
[0027] Embedding anchor-free frame module: The present invention uses the YOLOv3 model as the basic model and selects CenterNet to implement the anchor-free frame module. The present invention directly uses DarkNet53 as the backbone network and implements three parallel sub-branches to generate prediction results. In order to save computational costs, the anchor-free frame module is embedded in a feature map with a resolution of 104×104. When two objects are close, their center points may be divided into one point in a small-scale feature map (i.e., an output with a step size of 16). To this end, the anchor-free frame module is embedded in a large-scale feature map to generate the center point and offset of the object. In addition to the center point and the corresponding height and width, the original CenterNet also predicts the offset to mitigate the impact of downsampling. In the experiment, replacing the offset prediction with a constant of 0.5 does not reduce the performance of target detection. The final result is obtained by performing NMS on the predictions of the three sub-branches and one CenterNet branch.
[0028] In terms of the output of CenterNet, P represents the set of n key point predictions for category c. Corresponding size prediction. The upper left and lower right coordinates of the bounding box can be expressed as:
[0029]
[0030]
[0031] 0.5 is used to alleviate the impact of position shift caused by downsampling, and s is the step size for mapping the predicted box back to the original image. Finally, the top 100 anchor-free box predictions with confidence scores are selected and merged with the prediction results of YOLOv3. The merged prediction results are further processed using NMS to produce the final result.
[0032] Design of decoupling control method: Classification and regression are two different tasks. Classification is to identify specific category labels, and regression is to locate objects. It is unreasonable to predict their results using unified features in the decoder network. In order to reduce the mutual influence of the two tasks, the present invention proposes four different methods to construct a decoupled prediction layer. The first method directly divides the last layer output of the original prediction layer into three branches. These three branches are used to predict the coordinate offset, classification confidence score, and target existence confidence score, respectively. The second and third methods decouple the original prediction layer into two branches, one branch ultimately predicts the classification confidence score and the target existence confidence score, and the other branch ultimately predicts the coordinate offset. The only difference between these two decoupling methods is the number of channels. Keeping the number of channels unchanged will bring huge computational complexity, while reducing the number of channels will affect the computational accuracy.
[0033] Training boosted classifiers: The boosted classifier is trained by several cascaded classifiers. The basic idea is that the first classifier predicts simpler samples, and the second classifier predicts more difficult samples, thereby producing more accurate results. In this way, the discrimination ability of subsequent classifiers is continuously improved. Specifically, the confidence scores of the predictions of the sth classifier for positive and negative samples are α and α, respectively. s and β s The confidence scores of the kth (k>s) classifier for the positive and negative samples are α k and β k When α k >α s And β k <β s s, indicating that the ability of the cascade classifier is being improved. Otherwise, the classifier will be penalized. p is the confidence score of a positive sample predicted by the first classifier, X p is the confidence score of the positive sample predicted by the second classifier. The loss function of the positive sample is defined as follows:
[0034] L p =-(1+ζ p -X p )·log(X p ) (2)
[0035] Likewise, nis the confidence score of a negative sample predicted by the first classifier, X n is the confidence score of the negative sample predicted by the second classifier. The loss function of the negative sample is defined as follows:
[0036] L n =-(1-ζ n +X n )·log(1-X n ) (3)
[0037] For a positive sample, if the confidence score predicted by the next classifier is greater than the confidence score predicted by the previous classifier (denoted as ζ p ), then the loss calculated by the designed function will be less than the original cross entropy loss function. For negative samples, the confidence score predicted by the current classifier is ζ n , the confidence score predicted by the next classifier is greater than ζ n , the loss calculated by the designed function will be greater than the original cross entropy loss function. That is, in each iteration, the classifier will focus on those difficult samples rather than simple samples.
[0038] Preferably, the present invention, embedded in the anchor-free frame module, imitates the backbone network of CenterNet and establishes a simplified DLA (S-DLA) on DarkNet. In S-DLA, the transposed convolution in the original DLA is replaced with bilinear interpolation to reduce the model size.
[0039] Preferably, the feature map embedded in the anchor-free frame module includes four scales, the first three layers of feature maps are used for backbone network prediction, and the fourth layer of feature maps is used for anchor-free frame module prediction.
[0040] Preferably, in order to strike a balance between accuracy and speed in the design of the decoupled control method, the subnet structure following RetinaNet is chosen to decompose the original prediction layer into two independent branches. And 4 convolutions with a fixed number of channels are used in each branch.
[0041] Preferably, the feature enhancement module (FEM) in the design decoupling control method is composed of three convolutional layers, two 1×1 convolutions are used to increase the number of channels, and one 3×3 convolution is used to obtain more semantic information. The output of the FEM is further fed back to another branch for classification. The FEM proposed in the present invention is similar to a compensation mechanism, and its purpose is to compensate for the reduced information in the classification branch.
[0042] Preferably, the number of the cascade classifiers in the training enhanced classifier is set to 6.
[0043] Preferably, when calculating the loss in the training enhanced classifier, different weights are assigned to different samples and different classifiers. The loss function formula of the sth classifier is as follows:
[0044]
[0045] γ s represents the weight of the sth classifier, L p and L n are positive sample loss and negative sample loss respectively. The weight parameters must satisfy γ1<γ2<…<γ s , because the penalty for classifiers needs to be gradually increased. The present invention tolerates the mistakes made by the previous classifiers, but has a lower tolerance for the mistakes made by the following classifiers. After making mistakes, they will be punished more, and this penalty is reflected in the loss. All these classifiers participate in the training, but only the last one is used in the reasoning. Therefore, the classifier designed by the present invention does not generate any additional calculations that affect the reasoning.
[0046] The present invention combines the advantages of anchor-free and anchor-based methods to propose a robust and fast single-stage object detection method. The present invention proposes a hybrid method that combines the anchor-free module with the YOLO prediction branch to generate more robust predictions and solve the problem of low recall rate of YOLOv3. The present invention proposes an enhanced classifier that uses multiple cascade classifiers to gradually extract redundant bounding boxes. The enhanced classifier can avoid certain key bounding boxes from being excluded before post-processing and suppress false detections. On this basis, the present invention proposes a decoupling method to deal with the problem of inaccurate positioning and proposes a feature enhancement module to construct a more robust feature representation.
[0047] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.
Claims
1. A single-stage object detection method using an anchor-free module and an enhanced classifier, characterized in that: The steps include: a. Embedding the anchor-free module: Use the backbone network to extract feature maps, embed the anchor-free module on the feature maps, merge the prediction results of the backbone network and the anchor-free module, and perform post-processing; Use the YOLOv3 model as the basic model, select CenterNet to implement the anchor-free module, and directly use DarkNet53 as the backbone network; b. Design decoupling control method: construct a decoupling prediction model and embed a feature enhancement module; c. Training enhanced classifiers: setting the number of cascade classifiers, defining positive and negative sample loss functions, assigning weights to different classifiers, and training classifiers; in the cascade classifiers, the classifiers from the front to the back gradually predict more difficult samples; After extracting the feature map in step a, a lightweight network is introduced into the backbone network to enhance the feature map, and the transposed convolution of the lightweight network is replaced with bilinear interpolation to reduce the model size; The feature map described in step a contains four scales, the first three layers of feature maps are used for backbone network prediction, and the fourth layer of feature maps is used for anchor-free module prediction; The decoupled prediction model described in step b decomposes the original prediction layer into two independent branches, using 4 convolutions with a fixed number of channels in each branch; The feature enhancement module in step b consists of three convolutional layers, and the output of the feature enhancement module is fed back to another branch for classification.
2. The single-stage target detection method using an anchor-free module and an enhanced classifier according to claim 1, characterized in that: The number of cascade classifiers in step c is set to 6.
3. The single-stage target detection method using an anchor-free frame module and an enhanced classifier according to claim 2, characterized in that: The weight parameters of each layer of classifiers in the cascade classifier described in step c satisfy an increasing relationship, indicating that the penalty for the classifier needs to be gradually increased. All these classifiers participate in the training, but only the last classifier is used during inference.
