A target detection model for enhancing alignment of target detection branches

Abstract

The application discloses a method for enhancing alignment of target detection branches. Firstly, the application optimizes the structure of a detection head in target detection, enhances information exchange between independent parallel branches while keeping the function focus of each branch, and secondly, the application redesigns a loss function so that the model can focus on samples with good regression performance but poor classification performance, thereby improving the ability of the model in capturing consistent key features. The application provides a general solution for target detection in multiple scenarios, effectively solves the problem of accurate positioning but inaccurate classification in the prediction result, and further improves the overall performance of target detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and particularly relates to a general method for enhancing alignment between branches in target detection. BACKGROUND

[0002] Target detection is one of the most attractive and challenging tasks in the field of computer vision, and its core lies in accurately locating and correctly classifying targets in images. With the rapid development of deep neural network technology, target detection technology has made significant progress and performance breakthroughs in recent years, and is widely used in automatic driving, intelligent security monitoring, industrial detection, aerospace, and many other industries. These application scenarios demonstrate the great practical significance and production value of target detection technology in the real world.

[0003] The typical architecture of target detection includes three parts: Backbone (extracting feature training), Neck (integrating and collecting features), and Head (target detection). Among them, Backbone is responsible for extracting the basic features of the image, Neck connects Backbone and Head, and is used to generate feature maps of different scales. Head generates target classification and bounding box regression according to the feature maps generated by Neck.

[0004] Although significant progress has been made, target detection technology still faces some key problems that need to be solved, especially the inconsistency between the classification and regression branches. Specifically, on some samples, the classification may have a high score but the regression performance is poor, or the regression performance is good but the classification accuracy is low. This inconsistency, also known as the task alignment problem in modern target detectors, is an important obstacle to further improving the performance of the detector. Currently, in order to solve the inconsistency problem between the classification and regression branches, two strategies have been proposed:

[0005] 1) Task alignment model learning strategy: This strategy aims to enhance the model's attention to sample points that perform well in both classification and regression by matching samples or adjusting sample weights. Ideally, a target detection model should perform well in both localization and classification. To achieve this goal, the model learning strategy usually selects sample points that perform well and adjusts the weights of these sample points in the loss function, so that the model pays more attention to these key sample points during training, thereby more effectively learning relevant features. However, existing methods focus more on samples that perform well, ignoring samples that perform well in regression but have low classification scores, limiting further performance improvement.

[0006] 2) Task-aligned network structure design: In the traditional decoupled head (decoupled head, parallel head) structure, independent 4-layer convolutional layers are usually used to handle classification and regression tasks respectively. This design also easily causes the classification and regression branches to only focus on their own features, which in turn causes inconsistency problems. To solve this problem, the task-aligned network structure design aims to improve the consistency between branches by enhancing the information sharing and interaction between the classification and regression branches. For example, the head structure is redesigned to better coordinate the classification and regression tasks, so that the model performs more consistently when processing complex scenarios. However, the current design is relatively complex, the connection between components is not intuitive enough, and it also increases the computational cost. Summary of the Invention

[0007] In response to the problems existing in the existing technology, the present invention proposes a method to enhance the branch alignment of target detection: by redesigning the loss function, the model pays more attention to highly potential sample points with good regression performance but poor classification performance, so as to obtain more consistency-related features; at the same time, the Head part is reconstructed to realize information interaction between branches, strengthen the communication between classification and regression, and at the same time maintain their focus on their own features.

[0008] Technical solution of the present invention:

[0009] An object detection model for enhancing object detection branch alignment includes the following modules: a feature extraction module (Backbone), a feature integration module (Neck), a result prediction module (Head) and a model optimization module.

[0010] The feature extraction module extracts high- and low-dimensional depth features processed by the target detection model from the input image through the backbone network and provides them to the feature integration module;

[0011] The feature integration module (Neck) optimizes and combines the extracted feature information at different levels to enhance the representation of objects at different scales;

[0012] The result prediction module (Head) predicts the object position and corresponding category based on the integrated features to obtain the result. The Head part includes a regression branch and a classification branch. The regression branch is used to predict the position of the target, and the classification branch is responsible for predicting the category of the target. The present invention introduces a dynamic convolution layer to calculate all the features of the regression branch and the classification branch. The results are then passed to the subsequent convolution layers in the regression and classification chains respectively to enhance the features.

[0013] The model optimization module compares the predicted results with the actual values ​​of the objects, calculates the loss value, and optimizes the model parameters.

[0014] Beneficial effects of the present invention:

[0015] The present invention optimizes the structure of the detection head in target detection to ensure that the functional focus of the classification and regression branches is retained, while significantly enhancing the information exchange between independent parallel branches. This optimization not only improves the synergy between branches, but also provides the model with more opportunities to capture consistent features. By redesigning the loss function, the model can focus on "sub-difficult samples" that perform well in regression but poorly in classification. This design enables the model to capture consistent features more accurately, solves the problem of inconsistency between branches, and thus improves the detection performance as a whole.

[0016] The modified network structure of the present invention maintains its simplicity and effectiveness, with low computational complexity and strong scalability. Furthermore, information exchange between branches is further enhanced, significantly improving the consistency of features and detection results. The improved loss function design can more effectively unleash the model's potential, improve training efficiency, and enable the model to perform better when handling complex tasks, further enhancing the model's robustness in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The overall framework diagram of the target detection model of the present invention

[0018] Figure 2 Target detection model processing flow chart of the present invention

[0019] Figure 3 Internal structure of the head of the present invention

[0020] Figure 4 Loss calculation process of the present invention DETAILED DESCRIPTION

[0021] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further described below with reference to the accompanying drawings and specific embodiments.

[0022] An object detection model that enhances object detection branch alignment, such as Figure 1 As shown (the Backbone and Neck parts in the figure are simplified diagrams), it includes: feature extraction module (Backbone), feature integration module (Neck), result prediction module (Head) and model optimization module.

[0023] The feature extraction module extracts high- and low-dimensional depth features processed by the target detection model from the input image through the backbone network and provides them to the feature integration module;

[0024] The feature integration module (Neck) optimizes and combines the extracted feature information at different levels to enhance the representation of objects at different scales;

[0025] The result prediction module (Head) predicts the object position and the corresponding category based on the integrated features to obtain the result; the Head part includes a regression branch and a classification branch, the regression branch is used to predict the position of the target, and the classification branch is responsible for predicting the category of the target; the present invention has a good

[0026] The model optimization module compares the predicted results with the actual values ​​of the objects, calculates the loss value, and optimizes the model parameters.

[0027] The corresponding target detection model processing flow is as follows Figure 2 .

[0028] Further explanation is as follows:

[0029] 1. Feature Extraction Module: The input raw image is first processed by the Backbone component. The Backbone component consists of a series of basic convolutional and pooling layers, designed to extract deep features from the raw data. These deep features are extracted through multi-layer convolution operations, gradually extracting basic image attributes such as texture and abstract semantics.

[0030] Taking ResNet50 as an example, it can be specifically divided into four stages: P1, P2, P3, and P4. The feature size is halved at each layer, and the number of channels is doubled. The ResNet architecture ensures that these features preserve the key information of the original image to the greatest extent possible. Through this process, ResNet generates a set of preliminary feature maps representing different levels of information in the input image, which serve as the basis for subsequent processing.

[0031] 2. Feature Integration Module: The Neck component further processes and integrates the extracted features. It focuses on optimizing the combination of feature information from different levels. Technically, Neck uses operations such as feature fusion and dimensionality reduction to ensure that features of different scales and resolutions can be effectively combined, enabling the model to more comprehensively understand the objects in the image.

[0032] As an example, the Neck structure in this invention uses the classic FPN (Feature Image Pyramid), which incorporates feature fusion and dimensionality reduction. FPN enhances the representation of objects at different scales, indirectly enhancing the model's ability to perceive objects in complex scenes, thereby providing more accurate and richer feature input for subsequent predictions.

[0033] 3. Result prediction module: After the features of the Neck part are integrated, the Head part is responsible for making the final prediction of the target location and category based on the integrated features.

[0034] Traditional object detectors typically use a decoupled head architecture, where the regression and classification branches are processed independently. The regression branch predicts the object's position, while the classification branch predicts its category. This paper improves on this traditional head architecture to enhance information exchange and feature alignment between the two branches.

[0035] Specifically, the traditional Head implementation algorithm: obtain the final feature output from the Neck part of the second step, and use To express, extract subtask features from it to get and The process can be expressed as in and In the above equations, the features obtained after the i-th convolutional layer are represented, and cls and reg represent the corresponding classification branch or regression branch respectively; and Represents multiple stacked 3x3 convolutions. The structures of these two links in the original Head structure are consistent, but the parameters are not shared. The subtask can extract features based on the area of ​​interest.

[0036] The improved Head detailed structure is as follows Figure 3 , where F Neck After two layers of 3x3 ordinary convolution for classification and regression respectively, the two sets of classification and regression output features are passed through f cat Perform feature integration, and input the integrated features into the dynamic convolution layer of shared parameters to obtain the corresponding output The corresponding outputs are respectively subjected to a single 3x3 convolutional layer for classification and regression. The above process can be expressed as:

[0037]

[0038] Among them, f DCN represents dynamic convolution;

[0039] Finally, a layer of convolution is used to adjust the number of channels to obtain the final output F' cls and' reg The model also calculates the Intersection over Union (IoU), which represents the ratio of the overlapping area between the predicted and ground-truth boxes to the total area between the predicted and ground-truth boxes, and the prediction score p, to evaluate the model's performance on regression and classification tasks. The methods for calculating IoU and the prediction score p are based on existing techniques.

[0040] The predicted score p is a vector containing all target categories, indicating the probability of being classified as a particular category. For example, in the MSCOCO dataset, which contains 80 categories, p is a vector containing the prediction scores for each of the 80 categories. Ideally, the model's score for the correctly classified category should be close to 1, while the confidence scores for the other 79 categories should be close to 0. This design allows for more accurate evaluation of the model's performance on multi-classification tasks.

[0041] In the above structure, the first two convolutional layers are responsible for preliminary feature screening to adapt to the subtask. The third layer uses deformable convolution to enhance the model's ability to flexibly capture features. Deformable convolution can dynamically adjust the position of the convolution kernel, allowing the model to better capture the shape and position of the target. It further strengthens the information exchange between the two branches focusing on different features, allowing the model to fully consider all features, thereby improving the accuracy of target detection. The features after the third convolution layer are passed to the fourth convolution channel of the regression and classification branches respectively, and the integrated output is fine-tuned. The final convolution layer performs the final prediction calculation.

[0042] This structural improvement not only retains the advantages of the traditional decoupled head in focusing on task features, but also further strengthens feature sharing and information interaction. In the above process, the number of channels of the output features remains consistent.

[0043] Furthermore, the result prediction module optimizes the structure of the head part to keep the model simple and retain the sensitive features that each branch focuses on. It also strengthens the information interaction between the two branches, allowing the model to fully consider the entire task rather than just the subtasks. The specific process is as follows:

[0044] a. Input the integrated features into two parallel independent convolutional chains. These convolutional chains consist of two convolutional layers, and the two chains represent the regression chain and the classification chain respectively.

[0045] b. The features obtained from the two links are further integrated and fed into the third convolutional layer. The third convolution layer uses deformable convolution to enhance the model's flexible feature capture. Deformable convolution can dynamically adjust the position of the dynamic kernel to more fully consider the feature information between classification and regression.

[0046] c. Pass the features output in b to the fourth convolution layer in the regression and classification links respectively to enhance the features.

[0047] 4. Optimization Model Module: The aforementioned feature extraction, feature integration, and prediction constitute a complete training cycle. To ensure continuous model optimization, a feedback mechanism is introduced during training. This calculates the loss function by comparing the predicted results with the true values, and updates and optimizes the model parameters after each training cycle.

[0048] In order to further optimize the model, this paper designs a loss function for each sample point. First, the foreground and background are divided. For the foreground objects, a more detailed loss function design is adopted. The original classification loss is improved from only considering the classification to a comprehensive evaluation of the classification loss based on IoU and classification score. At the same time, more attention is paid to the objects with good IoU performance but poor classification performance. For the background objects, the original design is still used. The details are as follows (the calculation process refers to Figure 4 ):

[0049] 1) For the classification loss of negative samples (background class), Focalloss is used to calculate. The Focalloss loss function is:

[0050] L cls =-(1-p t ) γ ln(p t ),

[0051] where (1-p t ) γ As a weight factor, γ is a hyperparameter with a value of 2, p t Is a vector whose length is the number of target categories, corresponding to the probability of each category, expressed as

[0052] y=1 indicates that the classification is correct.

[0053] This loss function can effectively deal with the problem of class imbalance and give higher weights to difficult-to-classify samples.

[0054] 2) For positive samples (foreground class), the present invention designs a loss function associated with IoU. Specifically, the loss function weight is introduced, and the expression is as follows:

[0055]

[0056] Where α and β are hyperparameters, which are 4 and 1 respectively, and the loss weight of the correctly classified item is defined as W pos , which means that for samples with good IoU performance, if their classification scores are low, the loss weight will increase, thereby guiding the model to pay more attention to the optimization of such sample points; and the classification loss weight for other incorrectly classified categories is defined as W neg,This design can ensure that the model comprehensively considers the performance of various types of sample points when processing classification tasks, thereby further improving the overall target detection performance.

[0057] In summary, the loss function is as follows:

[0058]

[0059] Among them, the total loss is L cls and L reg N represents the number of samples classified as foreground, M represents the number of samples classified as background, FL represents the Focal loss function; GIoU represents the regression loss function, and the calculation method is the existing technology, b i and b GT Represent the predicted box and the true box respectively.

[0060] On a simulation platform, the method of the present invention is compared with a typical target detection method, and the results are as follows:

[0061] 1) On the CrowdHuman dataset, the proposed method is compared with the FCOS (Fully Convolutional One-Stage Object Detection), ATSS (Bridging the Gap Between Anchor-based and Anchor-free Detection via Adaptive Training Sample Selection), and GFL (Generalized Focal Loss) methods. The results are shown in the table below. The proposed method has the best AP (average precision) and IoU performance.

[0062]

[0063]

[0064] 2) On the MSCOCO dataset, the AP (average precision) of the proposed method is further compared with that of TOOD (Task-aligned One-stage Object Detection) and DAL (A dynamic label assignment strategy for one-stage detectors), as shown in the following table:

[0065]

[0066] It can be seen that the method of the present invention achieves an AP of 47.9%, which is an improvement of 1.2% and 0.8% compared with other solutions respectively.

[0067] Explanation of terms:

[0068] Classification Branch & Regression Branch: In object detection, the final output includes the target box and the corresponding category confidence. The target box is generated by the regression branch and aims to accurately locate the location of the detected object; while the category confidence is generated by the classification branch and aims to accurately identify the target category.

[0069] Sample matching: This is used to determine the criteria for positive and negative samples in classification and regression. In practice, there are many methods for sample matching, such as segmentation based on Intersection over Union (IoU).

[0070] ResNet (Residual Network): A backbone architecture, ResNet50, ResNet101, the value represents the total depth of the current network architecture

[0071] FPN (Feature image pyramid): a commonly used architecture in Neck.

Claims

1. An object detection model with enhanced object detection branch alignment, characterized in that: It includes the following modules: feature extraction module, feature integration module, result prediction module and model optimization module; The feature extraction module extracts high- and low-dimensional depth features processed by the target detection model from the input image through the backbone network and provides them to the feature integration module; The feature integration module optimizes and combines the extracted feature information at different levels to enhance the representation of objects at different scales. The result prediction module predicts the object's location and corresponding category based on the integrated features to obtain the result. The result prediction module includes a regression branch and a classification branch. The regression branch is used to predict the target's location, while the classification branch is responsible for predicting the target's category. A dynamic convolution layer is introduced to calculate all the features of the regression branch and the classification branch. The results are then passed to subsequent convolution layers in the regression and classification chains respectively to enhance the features. The model optimization module compares the predicted results with the actual values ​​of the objects, calculates the loss value, and optimizes the model parameters; After the features are integrated by the feature integration module, the result prediction module is responsible for making the final prediction of the target position and category based on the integrated features; The feature integration module obtains feature output To express, extract subtask features from it to get and , the process can be expressed as ,in and In the figure, the features obtained after the i-th convolutional layer are represented, and the subscripts cls and reg respectively represent the corresponding classification branch or regression branch; and Represents multiple stacked 3x3 convolutions; F Neck After two layers of 3x3 ordinary convolution for classification and regression respectively, the two sets of classification and regression output features are passed through Perform feature integration, and input the integrated features into the dynamic convolution layer of shared parameters to obtain the corresponding output , and its corresponding outputs are respectively subjected to a single 3x3 convolutional layer for classification and regression. The above process can be expressed as: ， in, represents dynamic convolution; Finally, a layer of convolution is used to adjust the number of channels to obtain the final output and ; The intersection over union (IoU) and prediction score p are calculated to evaluate the performance of the model on regression and classification tasks.

2. The target detection model for enhancing target detection branch alignment according to claim 1, wherein: In the result prediction module structure, the first two convolutional layers are responsible for preliminary feature screening to adapt to the subtask; the third layer uses deformable convolution to enhance the model's ability to flexibly capture features.

3. The target detection model with enhanced target detection branch alignment according to claim 1, wherein: The model optimization module introduces a feedback mechanism during the model training process. It calculates the loss function by comparing the difference between the predicted results and the true values, and updates and optimizes the model parameters after each training cycle. The loss function calculation process is as follows: 1) For the classification loss of negative samples (background class), Focal loss is used to calculate; the Focal loss loss function is: ， in As a weight factor, γ is a hyperparameter with a value of 2. Is a vector whose length is the number of target categories, corresponding to the probability of each category, expressed as , y=1 means it is classified as the correct category; When dealing with class imbalance, give higher weights to samples that are difficult to classify; 2) For positive samples (foreground class), the loss function weight is introduced, and the expression is as follows: ， Where α and β are hyperparameters, which are 4 and 1 respectively, and the loss weight of the correctly classified item is defined as W pos , the classification loss weight of the incorrectly classified category is defined as W neg ; Combining 1) and 2), the loss function is as follows: Among them, the total loss is L cls and L reg The sum of ; N represents the number of samples classified as foreground, M represents the number of samples classified as background, FL represents the Focal loss function; GIoU represents the regression loss function, and the calculation method is the existing technology. and Represent the predicted box and the true box respectively.

Images