A training method for a visual target detection network model
Optimizing the target detection network through a two-stage training strategy, the problem of positive and negative samples is solved, and end-to-end detection without non-maximum suppression is achieved, which improves the detection performance of the model and reduces computational overhead.
Patent Information
- Application Number
- CN202210875511.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-25
- Publication Date
- 2025-07-08
- Estimated Expiration
- 2042-07-25
AI Technical Summary
The existing object detection methods have positive and negative samples imbalances during the training process, resulting in low recall and difficult model to converge. At the same time, non-maximum suppression processing is required after training, which increases calculation overhead and deployment complexity.
A two-stage model training strategy is adopted to train the classification and regression capabilities of the model in the first stage through a multi-sample matching strategy, and the prediction branches are individually trained in the second stage to achieve end-to-end detection without non-maximum suppression.
After training is completed, the model can output a result corresponding to a subject, avoiding non-maximum suppression processing, reducing computational overhead, simplifying model deployment, and improving detection performance.
Smart Images
Figure CN115359335B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image detection, and more specifically, to a training method for a visual object detection network model. Background Art
[0002] Object detection is an important issue in the field of computer vision. In recent years, object detection methods based on deep learning have greatly improved the results of object detection. Typical object detection methods based on deep learning include two categories. One is the one-stage method, represented by methods such as YOLO and SSD. This type of method directly uses CNN (Convolutional Neural Networks) to extract image features by performing operations such as alternating convolution, pooling, and non-linear transformation on the original image. Training the CNN generally uses the backpropagation algorithm, that is, given the class label of an image, calculating the loss between the class label and the output of the CNN, and propagating this loss back to the previous layers in the CNN in the form of derivatives to predict the class and location of the target object. The other is the two-stage method, represented by R-CNN, Fast R-CNN, and Faster R-CNN. This type of method first generates some candidate boxes (proposals), then uses the CNN to extract the features of these candidate boxes, and then predicts the class and location of the objects in the candidate boxes.
[0003] In existing methods, to alleviate the imbalance between positive and negative samples during model training, increase the recall rate during model training, and accelerate model convergence, multiple positive samples are usually matched for one object for loss calculation. After the model training is completed, this type of method usually outputs multiple results for a single object, and non-maximum suppression (NMS) post-processing operations need to be performed on the output results to select the optimal prediction result. Therefore, this type of method will cause the model to require the use of an additional NMS algorithm during deployment, increasing the additional computational overhead and the complexity of algorithm deployment.
[0004] To solve the technical problem of a single object in object detection outputting multiple targets, currently, the minimum cost assignment strategy is used to adaptively match 1 positive sample for each target. This type of method has a low recall rate and the model is difficult to converge during training due to the extreme imbalance between positive and negative samples. Summary of the Invention
[0005] In view of the above problems, the present invention provides a training method for a visual object detection network model that can achieve the end-to-end detection performance of the model without non-maximum suppression processing through a two-stage model training strategy.
[0006] The technical solution adopted by the present invention to solve the above technical problems is as follows: A training method for a visual target detection network model, comprising the following steps:
[0007] S1, Obtain image samples. For each target appearing in the image samples, annotate their bounding boxes and categories according to the position, category, and boundary, and divide the data set into a training set and a validation set according to a ratio;
[0008] S2, Establish a target detection network model. The target detection network model includes a backbone network, a feature fusion network, and a decoupled prediction network. The backbone network is used for feature extraction and transmits semantic features of different scales to the feature fusion network. The feature fusion network fuses the semantic features of different scales and transmits the fused semantic features to the decoupled prediction network. The decoupled prediction network makes predictions for each anchor point according to the fused semantic features;
[0009] The decoupled prediction network includes a classification branch, a prediction branch, and a regression branch. The classification branch is used to predict the target category corresponding to the anchor point. The prediction branch is used to predict whether the anchor point contains a target. The regression branch is used to predict the position offset and size of the bounding box corresponding to the anchor point;
[0010] S3, Multi-sample first-stage matching training. Use the training set to train the target network detection model using the multi-sample matching strategy, and use the validation set to verify the model being trained;
[0011] S4, Single-sample second-stage matching training. Use the training set to train the target detection network model using the single-sample matching strategy, and use the validation set to verify the model being trained;
[0012] S5, Input the image to be detected into the trained target detection network model, detect the main body in the image, and give the category, center position, and size of each target.
[0013] Preferably, the backbone network in step S2 is a feature extractor based on a convolutional neural network, and the feature extractor includes but is not limited to Resnet and CSPDarknet.
[0014] Preferably, the feature fusion network in step S2 fuses the semantic features of different scales, so that the shallow features have deep semantic information, and the deep features have shallow semantic information, realizing the recognition of targets of different scales.
[0015] Preferably, in step S3, the multi-sample matching strategy calculates the loss by adaptively selecting the number of positive samples matching each main body, increasing the recall rate of the model.
[0016] Preferably, the single-sample matching strategy calculates the loss by adaptively selecting the optimal positive sample that matches each subject.
[0017] Preferably, the single-sample second-stage matching training is the separate training of the prediction branch.
[0018] Preferably, the specific steps of the multi-sample first-stage matching training are as follows.
[0019] S301, Initialize the object detection network model and the training set.
[0020] S302, Input a batch of data into the object detection network model for forward propagation.
[0021] S303, Calculate the cost matrix of the anchor points falling within the target area according to the output results.
[0022] S304, Adaptively calculate the number k of positive samples matched by each target according to the cost.
[0023] S305, Take the k anchor points with the highest cost as the positive samples matched by a single target, then calculate the loss, and obtain the gradient by taking the derivative of each parameter through the loss function.
[0024] S306, Backpropagate the gradient data obtained in S305 to update the parameters.
[0025] S307, Loop through S302 to S306 until the number of iterations reaches the preset number of times in the first stage or the loss value is less than a certain set threshold, then end the training.
[0026] Preferably, the specific steps of the single-sample second-stage matching training are as follows.
[0027] S401, Freeze the parameters of the backbone network, feature fusion network, classification branch, and regression branch of the object detection network model, prohibit the gradient backpropagation during the subsequent training process, separately train the prediction branch, and change the number of matched positive samples from k to 1.
[0028] S402, Input a batch of data into the object detection network model for forward propagation.
[0029] S403, Calculate the cost matrix of the anchor points falling within the target area according to the output results.
[0030] S404, Sort the elements in the cost matrix by column, select the 1 anchor point with the highest cost for a single target as the matched positive sample, then calculate the loss, and obtain the gradient by taking the derivative of each parameter through the loss function.
[0031] S405. Backpropagate the gradient data obtained in S404 to update the parameters;
[0032] S406. Loop and execute S402 to S405 until the number of iterations reaches the preset number of times in the second stage or the loss value is less than a certain set threshold, and then end the training.
[0033] Compared with the prior art, the present invention has the following beneficial effects:
[0034] 1. The present invention firstly proposes a training method for a two-stage visual object detection network model, optimizes the non-maximum suppression post-processing of the object detection algorithm during the training process, enables the model to have the ability to correspond one output to one main body after training, and after deployment, can achieve the end-to-end detection performance of the model without performing non-maximum suppression processing, effectively solving the problem of target loss existing in non-maximum suppression, saving computational overhead, and at the same time, due to the simple structure of the single model in the present invention, it is more convenient for model deployment and calculation;
[0035] 2. In the present invention, in order to make the model correspond one output to one main body when outputting, the decisive factor is the prediction branch in the decoupled prediction network. In the first stage of training, the classification ability and regression ability of the model have been trained to the optimal level. In the second stage, only the prediction branch needs to be trained, so as to realize the separate training of the prediction branch through the design of three independent branches of the decoupled prediction network. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 is a schematic structural diagram of the object detection network model of the present invention;
[0037] Figure 2 is a schematic structural diagram of the decoupled prediction network of the present invention;
[0038] Figure 3 is a flowchart of the two-stage training method of the present invention;
[0039] Figure 4 is a comparison graph of the loss changes between the two-stage training method and the single-sample matching training method shown in the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0040] The following will combine Figures 1-4 to describe the present invention in detail. Here, the illustrative embodiments of the present invention and the description are used to explain the present invention, but do not limit the present invention.
[0041] The present invention proposes a training method for a visual target detection network model. Image samples are obtained, and a target detection network model is constructed, including a backbone network, a feature fusion network, and a decoupled prediction network. Among them, the backbone network is used for feature extraction and transmits semantic features of different scales to the feature fusion network. The feature fusion network fuses the semantic features of different scales and transmits the fused semantic features to the decoupled prediction network. The decoupled prediction network predicts the category and position coordinates of each anchor point according to the fused semantic features. By training the model in two stages, the model does not require non-maximum suppression post-processing during deployment.
[0042] As Figure 1 shown, the image to be detected (shown in the lower left figure) is input into the target detection network model. After being processed by the target detection network model, the output is the prediction results of whether the anchor points in the picture contain targets, the categories of the targets corresponding to the anchor points, the position offsets and sizes of the bounding boxes corresponding to the anchor points. The specific processing steps are, from left to right, the backbone network (CSPDarknet), the feature fusion network (PAFPN), and the decoupled prediction network (Detect Head) in three parts:
[0043] The first part is the backbone network, as Figure 1 shown in the left part. This module provides three semantic features of different scales for the downstream feature fusion network. Its typical implementation is a network extractor based on a convolutional neural network. Popular ones in the academic community are, for example, Resnet, CSPDarknet, Densenet, etc. Common backbone networks are also applicable;
[0044] The second part is the feature fusion network, as Figure 1 shown in the middle part. This module fuses the semantic features of different scales transmitted by the backbone network and transmits the fused semantic information to the decoupled prediction network. Its typical implementation is a feature pyramid network based on a convolutional neural network. Popular ones in the academic community are, for example, BiFPN, PAFPN, etc. Common feature pyramid networks are also applicable;
[0045] The third part is the decoupled prediction network, as Figure 1 shown in the right part. This module decodes the fused semantic features and includes three branches: a classification branch (Class Branch), a regression branch (Regression Branch), and a prediction branch (Object Branch). Through these three branches, the anchor point category, the position offset and size of the bounding box, and whether it contains a target can be predicted respectively. During the training process, each branch can be trained independently without affecting the performance of other parts. The detailed structure of the decoupled prediction network in the present invention is as Figure 2 shown.
[0046] Figure 3 The figure shows a two-stage training process, including two stages of multi-sample matching training and single-sample matching training carried out in sequence, specifically as follows:
[0047] The first stage is multi-sample matching training, and the process is as Figure 3 shown in the left loop. After the target detection network model and the training set are initialized, a batch of data is propagated forward. According to the output results, the cost matrix of the anchor points falling within the target area is calculated. According to the cost, the number of positive samples k matched by each target is adaptively calculated. The k anchor points with the highest cost (Top-k cost) are used as the positive samples for a single target match. Then the loss is calculated, and the derivative of each parameter is obtained through the loss function to get the gradient. Thus, the parameters are updated through backpropagation according to the gradient data. The above process is executed in a loop until the number of iterations (Epoch) reaches the preset number of times (Multi-epochs) in the first stage or the loss value is less than a certain set threshold, and then the first stage of training ends;
[0048] After the first stage ends, freeze the parameters of the backbone network, feature fusion network (PAFPN), classification branch, and regression branch of the target detection network model, prohibit the gradient backpropagation in the subsequent training process, and train the prediction branch (Object Branch) alone. At the same time, modify the matching strategy in the loss function, change the number of matched positive samples from k to 1, and prepare for the second stage of training;
[0049] The second stage is single-sample matching training, and the process is as Figure 3 shown in the right loop. Propagate a batch of data forward, calculate the cost matrix of the anchor points falling within the target area according to the output results, sort the elements in the cost matrix by column, select the 1 anchor point with the highest cost (Top-1 cost) for a single target as the matched positive sample, then calculate the loss, and obtain the gradient by taking the derivative of each parameter through the loss function. Thus, the parameters are updated through backpropagation according to the gradient data. The above process is executed in a loop until the number of iterations (Epoch) reaches the preset number of times (Single-epochs) in the second stage or the loss value is less than a certain set threshold, and then the second stage of training ends.
[0050] Specific embodiments use the one-stage and two-stage methods to train the model 600 times respectively, and the obtained loss change curves are as Figure 4 shown. Figure 4 a shows that when using the single-sample matching method to train 600 times, after training is completed, the model converges to 3.25 on the test set, and the performance is poor; Figure 4b was trained 600 times in total using a two-stage method of multi-sample matching and single-sample matching. In the first stage, it was trained 500 times using multi-sample matching. After the training was completed, the model converged to 0.85 on the test set. In the second stage, it was trained 100 times using single-sample matching, and finally the model converged to 1.02, with a slight performance loss compared to the first stage, but the performance was significantly better than the result of directly training 600 times using single-sample matching.
[0051] The present invention first proposes a prediction network framework for object detection in the visual field that can be trained without non-maximum suppression (NMS-free), enabling some branches of the model to be trained independently. At the same time, a two-stage model training method is proposed, adopting different sample matching strategies at different stages, so that only one prediction result is output for one object in the object detection network model, without the need for non-maximum suppression of the output results, effectively solving problems such as object loss and increased additional overhead existing in non-maximum suppression. And through testing, the performance of the method of the present invention is significantly higher than the method of directly training using the single-sample matching strategy. At the same time, due to the simple structure of a single model, it also has obvious advantages in deployment and calculation.
[0052] The above has introduced in detail the technical solutions provided by the embodiments of the present invention. Specific examples are used in this article to elaborate on the principles and implementation manners of the embodiments of the present invention. The descriptions of the above embodiments are only applicable to helping understand the principles of the embodiments of the present invention. At the same time, for those of ordinary skill in the art, according to the embodiments of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present invention.
Claims
1. A training method for a visual target detection network model, characterized in that, It includes the following steps: S1. Obtain image samples. For each target appearing in the image samples, label their bounding boxes and categories according to the position, category, and boundary, and divide the dataset into a training set and a validation set according to a ratio; S2. Establish an object detection network model. The object detection network model includes a backbone network, a feature fusion network, and a decoupled prediction network. The backbone network is used for feature extraction and transmits semantic features of different scales to the feature fusion network. The feature fusion network fuses the semantic features of different scales and transmits the fused semantic features to the decoupled prediction network. The decoupled prediction network makes predictions for each anchor point according to the fused semantic features; The decoupled prediction network includes a classification branch, a prediction branch, and a regression branch. The classification branch is used to predict the target category corresponding to the anchor point. The prediction branch is used to predict whether the anchor point contains a target. The regression branch is used to predict the position offset and size of the bounding box corresponding to the anchor point; S3. Multi-sample first-stage matching training. Use the training set to train the object network detection model using the multi-sample matching strategy, and use the validation set to verify the model being trained; S4. Single-sample second-stage matching training. Use the training set to train the object detection network model using the single-sample matching strategy, and use the validation set to verify the model being trained; The single-sample second-stage matching training is the separate training of the prediction branch; S5. Input the image to be detected into the trained object detection network model, detect the main body in the image, and give the category, center position, and size of each target.
2. The training method of the visual target detection network model according to claim 1, characterized in that: The backbone network in step S2 is a feature extractor based on a convolutional neural network, and the feature extractor includes Resnet and CSPDarknet.
3. The training method of the visual target detection network model according to claim 1, wherein: In step S2, the feature fusion network fuses the semantic features of different scales, so that the shallow features have deep semantic information, and the deep features have shallow semantic information, realizing the recognition of targets of different scales.
4. The training method of the visual target detection network model according to claim 1, wherein: In step S3, the multi-sample matching strategy calculates the loss by adaptively selecting the number of positive samples matching each main body.
5. The training method of the visual target detection network model according to claim 1, characterized in that: The single-sample matching strategy calculates the loss by adaptively selecting the optimal positive sample matching each main body.
6. The training method of the visual target detection network model according to claim 1, characterized in that: The specific steps of the multi-sample first-stage matching training are as follows: S301. Initialize the object detection network model and the training set; S302. Input a batch of data into the object detection network model for forward propagation; S303. Calculate the cost matrix of the anchor points falling within the target area according to the output result; S304. Adaptively calculate the number of positive samples k matched by each target according to the cost; S305. Take the k anchor points with the highest cost as the positive samples matched by a single target, and then calculate the loss, and obtain the gradient by taking the derivative of each parameter through the loss function; S306. Perform backpropagation to update the parameters with the gradient data obtained in S305; S307. Loop and execute S302 to S306 until the number of iterations reaches the preset number of times in the first stage or the loss value is less than a certain set threshold to end the training.
7. The training method of the visual target detection network model according to claim 1, characterized in that: The specific steps of the single-sample second-stage matching training are as follows: S401: Freeze the parameters of the backbone network, feature fusion network, classification branch, and regression branch of the object detection network model, prohibit the gradient backpropagation during the subsequent training process, and train the prediction branch separately. Change the number of matched positive samples from k to 1. S402: Input a batch of data into the object detection network model for forward propagation. S403: Calculate the cost matrix of the anchor points falling within the target area according to the output results. S404: Sort the elements in the cost matrix by column, select the anchor point with the highest cost as the matched positive sample for a single target, and then calculate the loss. Obtain the gradient by taking the derivative of each parameter through the loss function. S405: Perform backpropagation on the gradient data obtained in S404 to update the parameters. S406: Loop through S402 to S405 until the number of iterations reaches the preset number of times in the second stage or the loss value is less than a certain set threshold, and then end the training.