A classification and regression combined optimization target detection method

Abstract

The application discloses a target detection method combined with classification and regression optimization, and belongs to the technical field of computer vision and deep learning, which comprises the following steps: acquiring feature maps of each branch in a target detection network, wherein the feature maps comprise initial feature maps of a classification branch and initial feature maps of a regression branch; performing channel domain interaction on the feature maps of each branch to obtain channel-modulated features, wherein the channel-modulated features comprise first classification features and first regression features; performing spatial domain interaction on the channel-modulated features to obtain spatially enhanced features, wherein the spatially enhanced features comprise second classification features and second regression features; performing feature relationship interaction on the spatially enhanced features to obtain third regression features after final optimization; and inputting the second classification features into a classification prediction layer and inputting the third regression features into a regression prediction layer to obtain a target detection result. The application proposes a hierarchical and progressive interaction paradigm, which realizes fine collaboration from feature screening, region focusing to knowledge association, and can be integrated into mainstream target detection networks.

Description

Technical Field

[0001] This application belongs to the field of computer vision and deep learning technology, and in particular relates to a target detection method that combines classification and regression optimization. Background Technology

[0002] Object detection is a core task in computer vision, aiming to locate and identify objects of interest in images. Modern mainstream object detectors typically employ a multi-task learning framework, including a classification branch for predicting the object's category and a regression branch for regressing the object's precise bounding box location.

[0003] However, there are inherent differences and connections between classification and regression tasks: classification tasks focus on the semantic features of objects (such as texture, shape, and parts), while regression tasks focus on the geometric features of objects (such as location, scale, and aspect ratio). They are also interdependent: accurate localization helps in classification, while clear semantic understanding can improve localization accuracy.

[0004] Existing technologies typically employ a shared backbone network to extract features, then allow the two branches to operate independently or perform simple feature sharing. This approach overlooks the deep complementarity between tasks, leading to the following problems: Task conflict: The optimization objectives of the two branches are inconsistent, which may cause competition or interference at the feature level and affect the overall performance of the model.

[0005] Insufficient information utilization: The expertise of one branch (such as the semantic information of the classification branch) is not fully utilized to guide the learning of another branch (such as the regression branch).

[0006] Superficial interaction: Existing interaction methods are mostly limited to feature map addition or channel splicing, lacking structured, multi-granular, and directed information flow design, making it difficult to achieve efficient collaboration. Summary of the Invention

[0007] The purpose of this application is to overcome the shortcomings of the existing technology and provide a target detection method that jointly optimizes classification and regression. This method can systematically and hierarchically promote the deep interaction between classification and regression branches in target detection, solve the problems of insufficient interaction and weak synergy between classification and regression tasks in target detection networks, and improve detection accuracy and robustness.

[0008] The objective of this application is achieved through the following technical solution: A target detection method that combines classification and regression optimization, the method comprising: Obtain feature maps of each branch in the object detection network, including initial feature maps of the classification branch and initial feature maps of the regression branch; Channel domain interaction is performed on the feature maps of each branch to obtain channel-modulated features, which include first classification features and first regression features. Spatial domain interaction is performed on the channel-modulated features to obtain spatially enhanced features, which include second classification features and second regression features. The spatially enhanced features are subjected to feature relationship interaction to obtain the final optimized third regression features; The second classification feature is input into the classification prediction layer, and the third regression feature is input into the regression prediction layer to obtain the target detection result.

[0009] Furthermore, the process of performing channel-domain interaction on the feature maps of each branch to obtain channel-modulated features includes: Generate a first channel weight vector and a second channel weight vector, and cross-apply the first channel weight vector and the second channel weight vector to the corresponding feature map to obtain the first classification feature and the first regression feature after channel modulation.

[0010] Furthermore, the method also includes a method for generating weight vectors, specifically including: Global average pooling is performed on the initial feature map to obtain the channel description vector; The channel description vector is input into a weight generator consisting of two fully connected layers. The first layer is used for dimensionality reduction, and the second layer is used to restore the original dimension. The ReLU activation function is used in between, and the channel weight vector is generated by the Sigmoid activation function.

[0011] Furthermore, the spatial domain interaction of the channel-modulated features to obtain spatially enhanced features includes: The channel-modulated features are concatenated to obtain fused features, and a shared spatial attention map is generated through convolutional layers and activation functions. The shared spatial attention map is applied to the first classification feature and the first regression feature respectively to obtain the spatially enhanced second classification feature and second regression feature.

[0012] Furthermore, the step of performing feature relationship interaction on the spatially enhanced features to obtain the final optimized third regression features includes: Using the second regression feature as the query matrix and the second classification feature as the key matrix and value matrix, semantic enhancement features are obtained by cross-attention calculation; The semantic enhancement feature is fused with the second regression feature through residual connection to obtain the third regression feature.

[0013] Furthermore, the method also includes: The second classification feature and the second regression feature are reshaped in spatial dimension to obtain a two-dimensional matrix, and then the query matrix, key matrix and value matrix are obtained through a linear projection layer.

[0014] Furthermore, the semantically enhanced features obtained through cross-attention calculation include: The query matrix, key matrix, and value matrix are divided into multiple heads, and attention is computed in parallel on each head.

[0015] Furthermore, the step of fusing the semantically enhanced features with the second regression feature through residual connections to obtain the third regression feature includes: The output projection matrices of each head are stitched together and then fused through a linear projection layer; After reshaping and restoring the spatial dimension of the multi-head attention output, it is fused with the second regression feature through residual connection to obtain the third regression feature.

[0016] Furthermore, the method also includes: During the channel domain interaction phase, learnable coefficients are introduced for cross-weight application. These learnable coefficients are gradually learned and adjusted during training to dynamically adjust the interaction intensity. During the feature relationship interaction stage, a gating mechanism is introduced for attention output to filter out irrelevant information.

[0017] Furthermore, the method also includes: After obtaining the third regression feature, cross attention is symmetrically performed with the classification branch as the query and the regression branch as the key and value. The cross attention of the classification branch is calculated, and the result is fused into the classification branch through residual connection so that the classification branch obtains geometric information from the regression branch.

[0018] The beneficial effects of this application are as follows: (1) This application proposes a multi-level, progressive classification and regression interaction mechanism, which realizes refined collaboration from feature channel selection to spatial region focusing and then to high-level semantic association through three stages: channel interaction, spatial interaction and feature relationship interaction. This hierarchical interaction design can systematically solve the information asymmetry problem between the two tasks and significantly improve the accuracy and robustness of target detection.

[0019] (2) This application designs a directed attention flow, especially in the feature relationship interaction stage, which enables the regression branch to actively obtain semantic information from the classification branch. This design clarifies the direction of information flow, makes the interaction clear in purpose and targeting, and effectively solves the problem of positioning ambiguity caused by appearance changes, occlusion, etc., and the improvement of positioning accuracy in complex scenes is particularly significant.

[0020] (3) The shared spatial attention mechanism in this application differs from the traditional independent spatial attention mechanism. The shared attention map generated in this application is based on the joint decision-making of features from both branches, which can accurately identify spatial regions that are important for both classification and regression. This mechanism forces both branches to focus on the same spatial prior, fundamentally alleviating the task conflict problem in multi-task learning and improving the consistency of feature representations.

[0021] (4) By introducing learnable adaptive weights and a two-way interaction mechanism, this application makes the interaction process more flexible and intelligent. The network can dynamically adjust the interaction intensity and mode according to the specific task requirements and the characteristics of the input features, thereby achieving personalized feature optimization. This adaptive capability enables the method to maintain good performance in different scenarios and different target types. Attached Figure Description

[0022] Figure 1 This is a flowchart of a target detection method that combines classification and regression optimization. Figure 2 This is a schematic diagram showing the location of the multi-stage attention interaction module integrated into the YOLOv8 head network; Figure 3 This is a structural diagram of the channel interaction submodule; Figure 4 This is a structural diagram of the spatial interaction submodule; Figure 5 This is a structural diagram of the feature relationship interaction submodule. Detailed Implementation

[0023] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0024] Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] There is an urgent need for a technology that can systematically and hierarchically promote the deep interaction between classification and regression branches in target detection, so as to improve detection accuracy and robustness.

[0026] To address the aforementioned technical problems, the following embodiments of a target detection method based on joint optimization of classification and regression are proposed in this application.

[0027] This embodiment proposes a method that introduces a multi-stage attention interaction module (MSAIM) into the object detection network. This module sequentially sets up three sub-modules: channel interaction, spatial interaction, and feature relationship interaction. First, it uses global average pooling and fully connected layers to generate cross-channel weights, thereby achieving channel attention modulation between the classification and regression branches. Second, it generates a shared spatial attention map through feature concatenation and convolution, allowing the two branches to collaboratively focus on key regions. Finally, it employs a cross-attention mechanism to enable the regression branch to actively retrieve semantic information from the classification branch, and completes feature fusion through residual connections.

[0028] Based on the above scheme, a target detection method jointly optimized by classification and regression was developed, referring to... Figure 1 ,like Figure 1 The diagram shown is a flowchart of a target detection method that combines classification and regression optimization, as provided in this embodiment. The method includes the following steps: S1: Obtain the initial feature map ReLU for the classification branch and the initial feature map Sigmoid for the regression branch in the object detection network, where B is the batch size, C is the number of channels, and H and W are the height and width of the feature map, respectively.

[0029] This embodiment integrates a multi-stage attention interaction module into the YOLOv8 object detection network, referring to... Figure 2 ,like Figure 2 The diagram illustrates the integration of the multi-stage attention interaction module into the YOLOv8 head network, illustrating a specific implementation scheme for aircraft detection in airport scenarios. This method fully leverages the advantages of the MSAIM module in optimizing the interaction between classification and regression tasks, while combining it with the efficient feature extraction capabilities of the YOLOv8 network, effectively addressing the unique challenges of aircraft target detection in airport scenarios. This embodiment uses the AGVS-R dataset for testing, which was collaboratively annotated by our laboratory.

[0030] The YOLOv8-based CSPDarknet architecture is used as the basic backbone network. This network significantly reduces computational complexity while maintaining feature extraction capabilities through depthwise separable convolutions and cross-stage partial connections. The backbone network outputs three feature maps at different scales, namely P3, P4, and P5, corresponding to 1 / 8, 1 / 16, and 1 / 32 resolution of the original image, respectively.

[0031] Choosing an appropriate feature layer is crucial for ensuring module performance. Through theoretical analysis and experimental verification, we determined the following criteria for feature layer selection: Let the feature map sequence output by the backbone network be... ,in The layer index is represented by L, where L is the total number of layers. The feature map of each layer has a dimension. The best integration layer for the MSAIM module The following optimization objectives should be met: ;

[0032] in: Indicates the first The amount of semantic information in layer features. Indicates the richness of spatial detail. This represents the computational complexity. In the YOLOv8 network, through systematic evaluation of each feature layer, layer P4 (downsampling rate of 16) was determined to be the optimal choice. For a standard 640×640 input image, the feature map size of this layer is 40×40, and the number of channels C=256, achieving the best balance between semantic information and spatial detail.

[0033] S2: Perform channel domain interaction, using global average pooling and a fully connected layer respectively. and Generate channel weight vector and The weight vectors are then cross-applied to the corresponding feature maps to obtain the channel-modulated features. and . Reference Figure 3 ,like Figure 3 The diagram shown is a structural schematic of the channel interaction submodule.

[0034] As one implementation method, the specific process of channel domain interaction in this embodiment is as follows: S21: Initial feature map for classification branch Perform global average pooling to obtain the channel description vector. .

[0035] S22: Will The input consists of a weight generator with two fully connected layers. The first layer reduces the dimension from C to C / r, and the second layer restores it to C dimension. The ReLU activation function is used in between, and the channel weight vector is generated by the Sigmoid activation function. .

[0036] S23: Transfer the weight vector Initial feature map of regression branch By performing channel-by-channel multiplication, we obtain the channel-modulated regression features: .

[0037] S24: Symmetrically, for the initial feature map of the regression branch Perform the same operation to generate a weight vector And by applying it to the classification branch feature map, we get: ; Where ⊙ represents element-wise multiplication in the channel direction.

[0038] The core idea of ​​channel interaction is to use the global statistical information of one branch to guide the channel selection of another branch. This interaction is based on the following theoretical foundation: the feature channels learned by the classification branch often correspond to the semantic parts of the object (such as wings, fuselage, tail, etc.), while the feature channels learned by the regression branch focus more on the geometric characteristics of the object (such as edges, corners, contours, etc.).

[0039] Let feature map First, channel statistics are extracted using Global Average Pooling (GAP): ; The mapping from channel statistics z to channel weights W is achieved through a two-layer bottleneck structure: ; in: The first layer fully connected weight matrix is ​​used to compress the channel dimension from C to C / r. This is the first layer bias vector. Activation function For the second fully connected layer weight matrix, restore the original channel dimension. This is the second layer bias vector. This is the Sigmoid activation function.

[0040] The compression ratio *r* is a key hyperparameter, and its selection requires a balance between model capacity and computational efficiency. Theoretically, the optimal compression ratio should allow the bottleneck layer to fully learn the relationships between channels while avoiding overfitting. Through experimental analysis, we determined that *r*=16 is the optimal value, at which point the bottleneck layer dimension is C / 16=16.

[0041] The channel weights generated by the classification branch for the regression branch are: ; The channel weights generated by the regression branch for the classification branch are: ; The two attention networks share the same structure but have independent parameters. and This allows them to learn weight generation strategies suitable for their respective tasks.

[0042] The channel modulation process is achieved through channel-by-channel Hadamard product: ; ; This embodiment uses cross-application of channel weights to enable the classification and regression branches to learn from each other's important feature channels. The semantic weights of the classification branch guide the regression branch to focus on feature channels useful for localization, while the geometric weights of the regression branch guide the classification branch to focus on feature channels with a clear spatial structure, achieving preliminary semantic-geometric feature alignment.

[0043] S3: Perform spatial domain interaction, modulating the features of the channels. and Channel splicing is performed to obtain fusion features A shared-space attention map M is generated through convolutional layers and a sigmoid activation function, and this attention map is then applied to... and Above, we obtain spatially enhanced features. and . Reference Figure 4 ,like Figure 4 The diagram shown is a structural schematic of the spatial interaction submodule.

[0044] As one implementation method, the specific process of spatial domain interaction in this embodiment is as follows: S31: Features after channel modulation and By concatenating the data along the channel dimension, we obtain the fused features: .

[0045] S32: Merge features Input a 3×3 convolutional layer with a kernel output channel of 1, then pass it through a sigmoid activation function to generate a shared spatial attention map: .

[0046] S33: Connect the shared spatial attention map M with... and Element-wise multiplication yields the spatially augmented features: ; Where ⊙ represents element-wise multiplication in spatial direction.

[0047] The goal of the spatial interaction phase is to generate a spatial attention map that can simultaneously guide classification and regression tasks. The theoretical basis for this design is that in object detection tasks, certain spatial regions (such as the object center and boundary regions) are crucial for both classification and regression.

[0048] First, the two feature maps after channel modulation are concatenated: ; The concatenation operation is performed along the channel dimension, so that the fused features simultaneously contain semantic information for classification and geometric information for regression.

[0049] The shared spatial attention map is generated through a lightweight convolutional network: ; Specifically: First convolutional layer: The number of channels was reduced from 512 to 64, thus reducing the computational load.

[0050] ReLU activation: introduces nonlinearity.

[0051] Second convolutional layer: Generate a single-channel spatial response map.

[0052] Sigmoid activation: Maps the response value to the [0,1] interval to obtain the final attention.

[0053] picture .

[0054] The design consideration for this two-layer convolutional structure is that the first convolution learns the local spatial patterns of the features, and the second convolution integrates these local patterns into a global spatial importance assessment.

[0055] The generated spatial attention map is applied to the features of both the classification and regression branches simultaneously: ; ; Attention map M is broadcast along the channel dimension to ensure that the same spatial importance pattern is applied to each channel.

[0056] This embodiment generates a shared spatial attention map, enabling the classification and regression branches to collaboratively focus on spatial regions important to both tasks. Based on joint decisions using features from both branches, this attention map accurately identifies important locations such as the main body of an object and boundary regions, while effectively suppressing background noise and improving the purity and specificity of feature representations.

[0057] S4: Perform feature relationship interaction to regress branch features. As the query matrix Q, with classification branch features Using the key matrix K and value matrix V, semantic enhancement features are calculated through cross-attention, and these features are then connected via residual connections. The features are then fused to obtain the final optimized regression branch features. . Reference Figure 5 ,like Figure 5 The diagram shown is a structural schematic of the feature relationship interaction submodule.

[0058] As one implementation method, the specific process of feature relationship interaction in this embodiment is as follows: S41: First, the spatially enhanced features... and By reshaping the spatial dimensions, a two-dimensional matrix is ​​obtained: ; in This represents the total number of spatial locations. S42: Features enhanced by spatial enhancement and The query matrix Q, key matrix K, and value matrix V are obtained by passing them through a linear projection layer respectively. ; in The projection matrix is ​​learnable; S43: Calculate cross-attention: ; in Let be the dimension of the key vector. This is used to prevent the gradient from vanishing due to an excessively large inner product. S44: For the multi-head attention mechanism, the query, key, and value matrices are divided into h heads respectively: ; Attention is computed in parallel on each head: ; S45: The outputs of each head are stitched together and then fused using a linear projection layer. ; in It is the output projection matrix; S46: After restoring the spatial dimension of the multi-head attention output through reshaping, it is fused with the original regression features via residual connections: .

[0059] The feature relationship interaction stage employs a cross-attention mechanism based on the Transformer architecture. This choice is based on the following considerations: the regression branch requires accurate location information, which can benefit from the rich semantic features learned by the classification branch.

[0060] First, the spatial features are reshaped into a format suitable for attention computation. Let... , The reshaping operation is defined as: ; ; The query, key, and value matrix is ​​obtained through linear projection: ; The dimensions of the projection matrix are: ; ; This employs a multi-head attention mechanism, dividing the query, key, and value matrix into h=8 heads along the feature dimension: ; For each attention head, compute the scaled dot product attention: ; scaling factor The effect can be explained mathematically: when the dimensions of the query and key vectors... When the variance is large, the result of the dot product may have a large variance, causing the gradient of the softmax function to decrease. It can stabilize gradient propagation.

[0061] The outputs of multi-head attention are spliced ​​and linearly projected: ; Attention-enhanced features are fused with original features via residual connections: ; here This represents the inverse operation of restoring a flat format to a spatial format. It is a learnable scaling factor, initially set to 1.0, which allows the network to adaptively adjust the relative importance of the original features and attention features.

[0062] From an optimization perspective, residual connections help solve the gradient vanishing problem in deep networks and can be interpreted as making a small perturbation in the original feature space: .

[0063] This embodiment employs a cross-attention mechanism, enabling the regression branch to proactively retrieve the most relevant semantic context information from the classification branch. Specifically, each spatial location (Query) in the regression branch calculates its relevance to all locations (Key) in the classification branch, and then aggregates the most relevant semantic features from the classification branch (Value). This mechanism provides rich semantic prior support for the localization process, effectively addressing localization ambiguity caused by target rotation, scale changes, and partial occlusion. The residual connection design maintains the integrity of the original geometric features while incorporating beneficial semantic information, achieving optimized feature fusion.

[0064] S5: Optimize the features and Input the data into the classification prediction layer and the regression prediction layer respectively to obtain the target detection results.

[0065] The features processed by the MSAIM module need to be seamlessly integrated with subsequent parts of the original detection network. For the YOLOv8 network, the processing flow is as follows: Classification branch output features: ; Regression branch output features: ; The MSAIM module is designed with engineering practicality in mind. Its standardized input / output interfaces allow for seamless integration into various inspection frameworks. The module's internal computational flow has been meticulously optimized to minimize computational overhead while ensuring performance.

[0066] From a software engineering perspective, the implementation of the MSAIM module in this embodiment adopts object-oriented design principles, provides a clear API interface, supports flexible parameter configuration, and facilitates adjustments and extensions by researchers and engineers according to specific needs.

[0067] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A target detection method that combines classification and regression optimization, characterized in that, The method includes: Obtain feature maps of each branch in the object detection network, including initial feature maps of the classification branch and initial feature maps of the regression branch; Channel domain interaction is performed on the feature maps of each branch to obtain channel-modulated features, which include first classification features and first regression features. Spatial domain interaction is performed on the channel-modulated features to obtain spatially enhanced features, which include second classification features and second regression features. The spatially enhanced features are subjected to feature relationship interaction to obtain the final optimized third regression features; The second classification feature is input into the classification prediction layer, and the third regression feature is input into the regression prediction layer to obtain the target detection result.

2. The target detection method jointly optimized by classification and regression as described in claim 1, characterized in that, The process of performing channel-domain interaction on the feature maps of each branch to obtain channel-modulated features includes: Generate a first channel weight vector and a second channel weight vector, and cross-apply the first channel weight vector and the second channel weight vector to the corresponding feature map to obtain the first classification feature and the first regression feature after channel modulation.

3. The target detection method based on joint optimization of classification and regression as described in claim 2, characterized in that, The method also includes a method for generating weight vectors, specifically including: Global average pooling is performed on the initial feature map to obtain the channel description vector; The channel description vector is input into a weight generator consisting of two fully connected layers. The first layer is used for dimensionality reduction, and the second layer is used to restore the original dimension. The ReLU activation function is used in between, and the channel weight vector is finally generated by the Sigmoid activation function.

4. The target detection method for joint optimization of classification and regression as described in claim 1, characterized in that, The spatial domain interaction of the channel-modulated features to obtain spatially enhanced features includes: The channel-modulated features are concatenated to obtain fused features, and a shared spatial attention map is generated through convolutional layers and activation functions. The shared spatial attention map is applied to the first classification feature and the first regression feature respectively to obtain the spatially enhanced second classification feature and second regression feature.

5. The target detection method jointly optimized by classification and regression as described in claim 1, characterized in that, The step of performing feature relationship interaction on the spatially enhanced features to obtain the final optimized third regression features includes: Using the second regression feature as the query matrix and the second classification feature as the key matrix and value matrix, semantic enhancement features are obtained by cross-attention calculation; The semantic enhancement feature is fused with the second regression feature through residual connection to obtain the third regression feature.

6. The target detection method jointly optimized by classification and regression as described in claim 5, characterized in that, The method further includes: The second classification feature and the second regression feature are reshaped in spatial dimension to obtain a two-dimensional matrix, and then the query matrix, key matrix and value matrix are obtained through a linear projection layer.

7. The target detection method jointly optimized by classification and regression as described in claim 5, characterized in that, The semantically enhanced features obtained through cross-attention calculation include: The query matrix, key matrix, and value matrix are divided into multiple heads, and attention is computed in parallel on each head.

8. The target detection method jointly optimized by classification and regression as described in claim 7, characterized in that, The step of fusing the semantically enhanced features with the second regression feature through residual connections to obtain the third regression feature includes: The output projection matrices of each head are stitched together and then fused through a linear projection layer; After reshaping and restoring the spatial dimension of the multi-head attention output, it is fused with the second regression feature through residual connection to obtain the third regression feature.

9. The target detection method for joint optimization of classification and regression as described in claim 1, characterized in that, The method further includes: During the channel domain interaction phase, learnable coefficients are introduced for cross-weight application. These learnable coefficients are gradually learned and adjusted during training to dynamically adjust the interaction intensity. During the feature relationship interaction stage, a gating mechanism is introduced for attention output to filter out irrelevant information.

10. The target detection method for joint optimization of classification and regression as described in claim 1, characterized in that, The method further includes: After obtaining the third regression feature, cross attention is symmetrically performed with the classification branch as the query and the regression branch as the key and value. The cross attention of the classification branch is calculated, and the result is fused into the classification branch through residual connection so that the classification branch obtains geometric information from the regression branch.

Images