Target detection model for enhancing alignment of target detection branches

Abstract

The invention discloses a method for enhancing target detection branch alignment. Firstly, structural optimization is carried out on a detection head in target detection, and information exchange between independent parallel branches is enhanced while functional key points of each branch are kept. Secondly, by redesigning a loss function, the model can concentrate on samples with good regression performance but poor classification performance, and then the ability of the model in capturing consistency key features is improved. According to the invention, a universal solution is provided for target detection in multiple scenes, the problem that the prediction result is accurate in positioning but inaccurate in classification is effectively solved, and the overall performance of target detection is further improved.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a general method for enhancing alignment between branches in target detection. Background Art

[0002] Object detection is one of the most attractive and challenging tasks in the field of computer vision. Its core lies in accurately locating and correctly classifying objects in images. With the rapid development of deep neural network technology, object detection technology has made significant progress and performance breakthroughs in recent years, and is widely used in many industries such as autonomous driving, intelligent security monitoring, industrial detection, aerospace, etc. These application scenarios demonstrate the great practical significance and production value of object detection technology in the real world.

[0003] The typical architecture of object detection consists of three parts: Backbone (feature extraction training), Neck (integration and collection of features) and Head (object detection). Among them, Backbone is responsible for extracting the basic features of the image, and Neck connects Backbone and Head to generate feature maps of different scales. Head performs object classification and bounding box regression based on the feature maps generated by Neck.

[0004] Despite significant progress, object detection technology still faces some key issues that need to be addressed, especially the inconsistency between classification and regression branches. Specifically, on some samples, the classification score may be high but the regression performance is poor, or the regression performance is good but the classification accuracy is low. This inconsistency is also called the task alignment problem in modern object detectors and is an important obstacle to further improving the performance of detectors. At present, in order to solve the inconsistency problem between classification and regression branches, two main strategies have been proposed:

[0005] 1) Task-aligned model learning strategy: This strategy aims to enhance the model's attention to sample points that perform well in both classification and regression by matching samples or adjusting sample weights. Ideally, the target detection model should perform well in both localization and classification. To achieve this goal, the model learning strategy usually gives priority to sample points with excellent performance, and adjusts the weights of these sample points in the loss function so that the model pays more attention to these key sample points during training, thereby learning related features more effectively. However, existing methods mostly focus on samples with excellent performance, ignoring samples with good regression performance but low classification scores, which limits further performance improvement.

[0006] 2) Task-aligned network structure design: In the traditional decoupled head (decoupled head, parallel head) structure, independent 4-layer convolutional layers are usually used to handle classification and regression tasks respectively. This design also easily causes the classification and regression branches to only focus on their own features, which in turn causes inconsistency problems. To solve this problem, the task-aligned network structure design aims to improve the consistency between branches by enhancing information sharing and interaction between classification and regression branches. For example, the head structure is redesigned to better coordinate classification and regression tasks, so that the model performs more consistently when processing complex scenarios. However, the current design is relatively complex, the connection between components is not intuitive enough, and it also increases the computational cost. Summary of the invention

[0007] In view of the problems existing in the prior art, the present invention proposes a method for enhancing the branch alignment of target detection: by redesigning the loss function, the model pays more attention to highly potential sample points with good regression performance but poor classification performance, so as to obtain more consistency-related features; at the same time, the Head part is reconstructed to realize information interaction between branches, strengthen the communication between classification and regression, and at the same time maintain their focus on their own features.

[0008] Technical solution of the present invention:

[0009] A target detection model for enhancing target detection branch alignment includes the following modules: a feature extraction module (Backbone), a feature integration module (Neck), a result prediction module (Head) and a model optimization module.

[0010] The feature extraction module extracts high- and low-dimensional depth features processed by the target detection model from the input image through the backbone network and provides them to the feature integration module;

[0011] The feature integration module (Neck) optimizes and combines the extracted feature information at different levels to enhance the representation of objects of different scales;

[0012] The result prediction module (Head) predicts the object position and the corresponding category according to the integrated features to obtain the result; the Head part includes a regression branch and a classification branch, the regression branch is used to predict the position of the target, and the classification branch is responsible for predicting the category of the target; the present invention introduces a dynamic convolution layer to calculate all the features of the regression branch and the classification branch, and then the results are respectively passed to the subsequent convolution layers in the regression and classification links to enhance the features;

[0013] The model optimization module compares the predicted results with the actual values ​​of the objects, calculates the loss values, and optimizes the model parameters.

[0014] Beneficial effects of the present invention:

[0015] The present invention optimizes the structure of the detection head in target detection to ensure that the functional focus of the classification and regression branches is retained, while significantly enhancing the information exchange between independent parallel branches. This optimization not only improves the synergy between branches, but also provides the model with more opportunities to capture consistent features. By redesigning the loss function, the model can focus on "sub-difficult samples" that perform well in regression but poor in classification. This design enables the model to capture consistent features more accurately, solves the problem of inconsistency between branches, and thus improves the detection performance as a whole.

[0016] The modified network structure of the present invention maintains the characteristics of simplicity and effectiveness, has low computational complexity, and still has strong scalability. At the same time, the information exchange between branches has been further enhanced, significantly improving the consistency of features and detection results. Through the improved loss function design, the model potential can be more effectively released, the training efficiency can be improved, the model can perform better when processing complex tasks, and the robustness of the model in practical applications can be further enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The overall framework diagram of the target detection model of the present invention

[0018] Figure 2 Target detection model processing flow chart of the present invention

[0019] Figure 3 Internal structure of the head of the present invention

[0020] Figure 4 Loss calculation process of the present invention DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0022] An object detection model with enhanced object detection branch alignment, such as Figure 1 As shown (the Backbone and Neck parts in the figure are simplified diagrams), it includes: feature extraction module (Backbone), feature integration module (Neck), result prediction module (Head) and model optimization module.

[0023] The feature extraction module extracts high- and low-dimensional depth features processed by the target detection model from the input image through the backbone network and provides them to the feature integration module;

[0024] The feature integration module (Neck) optimizes and combines the extracted feature information at different levels to enhance the representation of objects of different scales;

[0025] The result prediction module (Head) predicts the position of the object and the corresponding category according to the integrated features to obtain the result; the Head part includes a regression branch and a classification branch, the regression branch is used to predict the position of the target, and the classification branch is responsible for predicting the category of the target; the present invention has a plurality of branches in the regression branch and the classification branch.

[0026] The model optimization module compares the predicted results with the actual values ​​of the objects, calculates the loss values, and optimizes the model parameters.

[0027] The corresponding target detection model processing flow is as follows Figure 2 .

[0028] Further explanation is as follows:

[0029] 1. Feature extraction module: The input raw image is first processed by the Backbone part. The Backbone part consists of a series of basic convolutional layers, pooling layers, etc., which are designed to extract deep features from the raw data. These deep features are gradually extracted from the basic attributes such as image texture in the image to abstract semantics through multi-level convolution operations.

[0030] Take ResNet50 as an example. Specifically, it can be divided into 4 stages, namely P1, P2, P3, and P4 layers. The feature size is halved layer by layer, and the number of channels is doubled layer by layer. The ResNet architecture ensures that these features can preserve the key information of the original image to the greatest extent. Through this process, ResNet generates a set of preliminary feature maps representing different levels of information in the input image as the basis for subsequent processing.

[0031] 2. Feature integration module: Neck further processes and integrates the extracted features. It mainly focuses on how to optimize the combination of feature information from different levels. Technically, Neck ensures that features of different scales and resolutions can be effectively combined through operations such as feature fusion and dimensionality reduction, so that the model can understand the objects in the image more comprehensively.

[0032] As an embodiment, the Neck structure in the present invention adopts the classic FPN (Feature image pyramid), which includes feature fusion and dimensionality reduction operations. FPN strengthens the representation ability of each target at different scales, indirectly enhances the model's perception ability of targets in complex scenes, and thus provides more accurate and rich feature input for subsequent predictions.

[0033] 3. Result prediction module: After the features of the Neck part are integrated, the Head part is responsible for the final prediction of the target location and category based on the integrated features.

[0034] In traditional target detectors, the head part usually adopts a decoupled head structure, that is, the regression branch and the classification branch are processed independently. The regression branch is used to predict the position of the target, and the classification branch is responsible for predicting the category of the target. The present invention improves the traditional head structure to enhance the information interaction and feature alignment between the two branches.

[0035] Specifically, the traditional Head implementation algorithm is: get the final feature output from the Neck part of the second step, and use To represent, extract the subtask features from it to get and This process can be expressed as in and In the above diagram, the features obtained after the i-th convolutional layer are represented respectively, and cls and reg represent the corresponding classification branch or regression branch respectively; and Represents multiple stacked 3x3 convolutions. The structures of these two links in the original Head structure are consistent, but the parameters are not shared. The subtask can extract features based on the area of ​​interest.

[0036] The detailed structure of the improved Head is as follows Figure 3 , where F Neck After two layers of 3x3 ordinary convolution for classification and regression, the two sets of output features of classification and regression are passed through f cat Perform feature integration, and input the integrated features into the dynamic convolution layer with shared parameters to obtain the corresponding output The corresponding outputs are respectively subjected to a single 3x3 convolutional layer for classification and regression. The above process can be expressed as:

[0037]

[0038] Among them, f DCN represents dynamic convolution;

[0039] Finally, a layer of convolution is used to adjust the number of channels to obtain the final output F' cls and' reg ; and calculate IoU (Intersection over Union, which represents the ratio of the overlapping area of ​​the predicted box and the true box to the total area of ​​the predicted box and the true box) and the prediction score p to evaluate the performance of the model in regression and classification tasks. The calculation method of IoU and prediction score p is the existing technology.

[0040] Among them, the prediction score p is a vector containing all target categories, indicating the probability of classification into a certain category. For example, for the MSCOCO dataset containing 80 categories, p is a vector containing 80 category prediction scores. Ideally, the model's score for the correctly classified category should be close to 1, while the confidence scores for the other 79 categories should be close to 0. This design can more accurately evaluate the performance of the model in multi-classification tasks.

[0041] In the above structure, the first two convolutional layers are responsible for preliminary feature screening to adapt to the subtask. The third layer uses deformable convolution to enhance the model's ability to flexibly capture features. Deformable convolution can dynamically adjust the position of the convolution kernel, so that the model can better capture the shape and position information of the target, further strengthen the information interaction between the two branches that focus on different features, and enable the model to fully consider all features, thereby improving the accuracy of target detection. The features after the third convolution are passed to the fourth convolution channel of the regression and classification branches respectively for feature fine-tuning after integration and output. The last layer of convolution performs the final prediction calculation.

[0042] This structural improvement not only retains the advantages of the traditional decoupled head in focusing on task features, but also further strengthens the sharing of features and the interaction of information. In the above process, the number of channels of the output features remains consistent.

[0043] Furthermore, the result prediction module optimizes the structure of the Head part to keep the model simple and retain the sensitive features of each branch. It also strengthens the information interaction between the two branches, so that the model fully considers all tasks instead of just subtasks. The specific process is as follows:

[0044] a. Input the integrated features into two parallel independent convolution links, which are composed of two convolution layers. The two link tables represent the regression link and the classification link;

[0045] b. The features finally obtained from the two links are further integrated and sent to the third convolution layer. The third convolution layer uses deformable convolution to enhance the model's flexible capture of features. Deformable convolution can dynamically adjust the position of the dynamic kernel and more fully consider the feature information between classification and regression;

[0046] c. Pass the features output in b to the fourth convolution layer in the regression and classification links respectively to enhance the features.

[0047] 4. Optimization model module: The above feature extraction, feature integration and result prediction constitute a complete training cycle. In order to ensure that the model can be continuously optimized, a feedback mechanism is introduced in the training process. The loss function is calculated by comparing the difference between the predicted results and the true value, and the model parameters are updated and optimized after each training cycle.

[0048] In order to further optimize the model, the present invention designs a loss function for each sample point. First, the foreground and background are divided. For the foreground object, a more subdivided loss function design is adopted. The original classification loss is improved from only considering the classification to comprehensively assessing the classification loss according to IoU and classification score. At the same time, more attention is paid to the objects with good IoU performance but poor classification performance. For the background objects, the original design is still used, as follows (the calculation process refers to Figure 4 ):

[0049] 1) For the classification loss of negative samples (background class), Focalloss is used to calculate. The Focalloss loss function is:

[0050] L cls =-(1-p t ) γ ln(p t ),

[0051] Where (1-p t ) γ As a weight factor, γ is a hyperparameter with a value of 2, p t is a vector whose length is the number of target categories, corresponding to the probability of each category, expressed as

[0052] y=1 means the classification is correct.

[0053] This loss function can effectively deal with the problem of class imbalance and give higher weights to difficult-to-classify samples.

[0054] 2) For positive samples (foreground class), the present invention designs a loss function associated with IoU. Specifically, the loss function weight is introduced, and the expression is as follows:

[0055]

[0056] Where α and β are hyperparameters, which are 4 and 1 respectively, and the loss weight of the correctly classified item is defined as W pos , which means that for samples with good IoU performance, if their classification scores are low, the loss weight will increase, thereby guiding the model to pay more attention to the optimization of such sample points; and the classification loss weight for other incorrectly classified categories is defined as W negSuch a design can ensure that the model comprehensively considers the performance of various sample points when processing classification tasks, thereby further improving the overall target detection performance.

[0057] In summary, the loss function is as follows:

[0058]

[0059] Among them, the total loss is L cls and L reg N represents the number of samples classified as foreground, M represents the number of samples classified as background, FL represents the Focal loss function; GIoU represents the regression loss function, and the calculation method is the existing technology, b i and b GT Represent the predicted box and the true box respectively.

[0060] On the simulation platform, the method of the present invention is compared with the typical target detection method, and the results are as follows:

[0061] 1) On the CrowdHuman dataset, the proposed method is compared with FCOS (Fully Convolutional One-Stage Object Detection), ATSS (Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection) and GFL (Generalized Focal Loss) methods. The results are shown in the following table. The proposed method has the best AP (average precision) and IoU performance.

[0062]

[0063]

[0064] 2) On the MSCOCO dataset, the AP (average precision) of the proposed method is further compared with TOOD (Task-aligned One-stage Object Detection), DAL (A dynamic label assignment strategy for one-stage detectors) and other solutions, as shown in the following table:

[0065]

[0066] It can be seen that the method of the present invention achieves an AP of 47.9%, which is 1.2% and 0.8% higher than other solutions respectively.

[0067] Terminology explanation:

[0068] Classification branch & regression branch: In object detection, the final output includes the object box and the confidence of the corresponding category. The object box is generated by the regression branch to accurately locate the position of the detected object; while the confidence of the category is generated by the classification branch to accurately identify the category of the object.

[0069] Sample matching: It is used to determine the criteria for positive and negative samples in classification and regression. In practice, there are many methods for sample matching, such as dividing based on IoU (Intersection over Union).

[0070] ResNet (Residual Network): A backbone architecture, ResNet50, ResNet101, the value indicates the total depth of the current network architecture

[0071] FPN (Feature image pyramid): a commonly used architecture in Neck.

Claims

1. A target detection model with enhanced target detection branch alignment, characterized in that: Includes the following modules: Feature extraction module (Backbone), feature integration module (Neck), result prediction module (Head) and model optimization module. The feature extraction module extracts high- and low-dimensional depth features processed by the target detection model from the input image through the backbone network and provides them to the feature integration module; The feature integration module (Neck) optimizes and combines the extracted feature information at different levels to enhance the representation of objects of different scales; The result prediction module (Head) predicts the object position and corresponding category according to the integrated features to obtain the result; the Head part includes a regression branch and a classification branch, the regression branch is used to predict the position of the target, and the classification branch is responsible for predicting the category of the target; a dynamic convolution layer is introduced to calculate all the features of the regression branch and the classification branch, and then the results are respectively passed to the subsequent convolution layers in the regression and classification links to enhance the features; The model optimization module compares the predicted results with the actual values ​​of the objects, calculates the loss values, and optimizes the model parameters.

2. The target detection model for enhancing target detection branch alignment as claimed in claim 1, characterized in that: After the features of the Neck part are integrated, the result prediction module is responsible for the final prediction of the target position and category based on the integrated features; Neck part obtains feature output To represent, extract the subtask features from it to get and This process can be expressed as in and In the above figure, the features obtained after the i-th convolutional layer are represented respectively. The subscripts cls and reg respectively represent the corresponding classification branch or regression branch. and Represents multiple stacked 3x3 convolutions; F Neck After two layers of 3x3 ordinary convolution for classification and regression, the two sets of output features of classification and regression are passed through f cat Perform feature integration, and input the integrated features into the dynamic convolution layer with shared parameters to obtain the corresponding output The corresponding outputs are respectively subjected to a single 3x3 convolutional layer for classification and regression. The above process can be expressed as: Among them, f DCN represents dynamic convolution; Finally, a layer of convolution is used to adjust the number of channels to obtain the final output F' cls and F' reg ; and calculate the intersection over union (IoU) and prediction score p to evaluate the performance of the model on regression and classification tasks.

3. The target detection model for enhancing target detection branch alignment as claimed in claim 2, characterized in that: In the result prediction module structure, the first two convolutional layers are responsible for preliminary feature screening to adapt to the subtask; the third layer uses deformable convolution to enhance the model's ability to flexibly capture features.

4. The target detection model for enhancing target detection branch alignment as claimed in claim 1, characterized in that: The optimization model module introduces a feedback mechanism in the model training process, calculates the loss function by comparing the difference between the predicted results and the true value, and updates and optimizes the model parameters after each training cycle; the loss function calculation process is as follows: 1) For the classification loss of negative samples (background class), Focal loss is used to calculate; the Focal loss loss function is: L cls =-(1-p t ) γ ln(p t ), Where (1-p t ) γ As a weight factor, γ is a hyperparameter with a value of 2, p t is a vector whose length is the number of target categories, corresponding to the probability of each category, expressed as y=1 means the classification is correct; When dealing with class imbalance, give higher weights to samples that are difficult to classify; 2) For positive samples (foreground class), the loss function weight is introduced, and the expression is as follows: Where α and β are hyperparameters, which are 4 and 1 respectively, and the loss weight of the correctly classified item is defined as W pos , the classification loss weight of the incorrectly classified category is defined as W neg ; Combining 1) and 2), the loss function is as follows: Among them, the total loss is L cls and L reg The sum of ; N represents the number of samples classified as foreground, M represents the number of samples classified as background, FL represents the Focal loss function; GIoU represents the regression loss function, and the calculation method is the existing technology, b i and b BT Represent the predicted box and the true box respectively.

Images