Object detection method based on spatial decoupling and boundary feature aggregation

Abstract

The present invention discloses a target detection method based on spatial decoupling and boundary feature aggregation, including: a significant dependency supplementation module; a spatially sensitive area perception module; and a boundary feature aggregation module. The present invention first uses the spatially sensitive area perception module on different task branches to predict the spatially sensitive areas of the respective branches. Since the classification branch pays more attention to the significant area, the spatially sensitive area of ​​the classification task can be adjusted to the significant area more quickly through the significant dependency supplementation module. On the basis of the predicted spatially sensitive area, the boundary feature aggregation module is used to supplement each anchor point with sufficient boundary features for accurately predicting the classification and positioning scores. Based on the above modules, the present invention proposes a better task alignment network, which can quickly and accurately achieve spatial decoupling with minimal prediction burden, and then aggregate sufficient boundary information from the respective task spatially sensitive areas to the anchor points to align the task prediction results.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a target detection method based on spatial decoupling and boundary feature aggregation. Background Art

[0002] In the field of object detection, the R-CNN family and FPN have achieved breakthroughs in performance, but further performance improvements for these methods face bottlenecks. Subsequently, methods using a shared head for object classification and localization have gained attention and have been applied to most advanced detectors, including two-stage methods, single-stage methods, and anchor-free methods. Given that classification and regression are two independent branches with different learning focuses, this can lead to misalignment between the predicted classification scores and localization accuracy.

[0003] To address this issue, single-stage object detectors enhance the classification score by using an additional branch to predict a confidence score. Although this method alleviates the problem of misalignment between the two tasks, it is suboptimal because the classification score and confidence score predictions may be inaccurate, and a simple combination of the two may lead to worse ranking results.

[0004] The GFLv2 and VFNet methods use a joint representation approach, fusing regression confidence with classification results to eliminate task misalignment. However, this approach is limited in that it does not consider the spatial preferences of different tasks. Classification tasks focus more on features in salient areas, while regression tasks focus more on features in boundary areas. These methods all extract features from the same area, resulting in an inadequate solution to the misalignment problem between the two tasks. Furthermore, the star-shaped bounding box feature representation proposed by VFNet simply aggregates features from fixed boundary points into anchor points. While this can supplement the anchor points with the geometric shape of the bounding box and some boundary context information, the anchor points still lack representative boundary information, which can affect the accuracy of positioning predictions to a certain extent.

[0005] SALT-Net mitigates the problems of spatial coupling and task misalignment by predicting a set of perception points as the spatially sensitive regions of each task, and then aligning them to the same anchor point. However, the large number of predictions required for the two sets of perception points can easily lead to prediction bias. In the regression branch, the prediction area for the perception point set covers the entire object, without paying special attention to the boundary areas. As a result, the anchor points lack the boundary geometry and contextual information needed to predict positioning accuracy. Summary of the Invention

[0006] Aiming at the problem of spatial coupling and task misalignment in existing target detection, the present invention proposes a target detection method based on spatial decoupling and boundary feature aggregation.

[0007] First, in order to solve the spatial coupling problem caused by extracting features from the same area for the classification task and the regression task, a scheme for perceiving spatially sensitive areas for different tasks is proposed. The regression task pays more attention to the features of the boundary area, so no perception behavior is performed on this task. Instead, the four boundaries of the initial bounding box generated initially are directly used as the spatially sensitive areas of the regression task. The classification task focuses on the features of the salient area, takes the spatially sensitive area of ​​the regression task as the shape hypothesis, predicts four perception scaling factors, and uses them to adjust the spatially sensitive area of ​​the regression task to obtain the spatially sensitive area of ​​the classification task. The present invention uses a scheme for perceiving spatially sensitive areas for different tasks, so it has two advantages: reducing the impact of regional prediction deviation on positioning accuracy in the regression branch, and achieving relatively accurate perception prediction with a very small prediction burden in the classification branch.

[0008] Second, to address the low accuracy of perceptual scaling factor prediction, a salient dependency supplementation module is proposed. This module supplements the dependency relationship between each position in the feature map and the global salient features. This approach, by adding a salient dependency supplementation module, offers the advantage of rapidly adjusting the spatially sensitive regions of the classification task to salient regions.

[0009] Third, to address the problem of suboptimal bounding box prediction accuracy caused by a lack of sufficient boundary information in anchor points, a boundary feature aggregation module is proposed. This module consists of two submodules: a boundary representative feature aggregation module and a boundary supplementary feature aggregation module. The boundary representative feature aggregation module uses a boundary alignment module, which adaptively extracts boundary extreme points and then aggregates representative features extracted from these extreme points into anchor points. The boundary supplementary feature aggregation module uses a star-shaped bounding box feature representation. Based on the initial bounding box, this module selects nine sampling points and aggregates the features at their corresponding positions into anchor points through deformable convolution, thereby supplementing the anchor points with the bounding box geometry and sufficient boundary context information. Combining these two submodules, sufficient boundary information is supplemented for the anchor points, enabling accurate bounding box regression. The use of the boundary feature aggregation module in this invention offers the advantage of providing sufficient boundary information for each anchor point, thereby improving the accuracy of bounding box prediction. Furthermore, the boundary feature aggregation module is used in both task branches, thereby simultaneously optimizing classification scores and localization accuracy, i.e., the prediction results of the alignment task.

[0010] Fourth, to address the shortcomings of current task alignment networks, we propose a better task alignment network based on the three modules above. This improved task alignment network enables rapid and accurate spatial decoupling with minimal prediction overhead. It then aggregates sufficient boundary information from sensitive regions of the respective task spaces into anchor points, better aligning task predictions.

[0011] The specific steps of the target detection method based on spatial decoupling and boundary feature aggregation include:

[0012] Step 1. Input the feature map of each layer of FPN into the classification tower and regression tower to obtain the third layer feature map F of the classification tower c , the third layer feature map F of the regression tower r1 And the 4th layer feature map F of the regression tower r The classification tower contains three 3×3 convolutional layers with ReLU activation, and the regression tower contains four 3×3 convolutional layers with ReLU activation.

[0013] Step 2. F r As input, the initial bounding box B is generated by 3×3 convolution c .

[0014] Step 3. F c Input to the significant dependency supplement module S re This module adds the dependency relationship between each position of the feature map and the global saliency feature, and finally obtains the feature map F s The calculation formula is:

[0015] F s =S re (F c )

[0016] Step 4. Set the initial bounding box B c With the feature map F s Input to the spatially sensitive region perception module S area This module will F s As input, the perceptual scaling factor is generated and then compared with the initial bounding box B c Multiply to get the adjusted bounding box B c_adjust , where its four boundaries are the spatially sensitive areas of the classification branches; the calculation formula is:

[0017] B c_adjust =S area (B c , F s )

[0018] Step 5. In the regression branch, the initial bounding box B c With the feature map F r1Input to the boundary feature aggregation module. This module is supplemented by the boundary feature aggregation module G sup and boundary representation feature aggregation module G dep Composition. dep Use the boundary alignment module to c With F r1 Input to this module generates a feature map containing boundary representation information. sup Using the star-shaped bounding box feature representation module, B c With F r1 Input to this module to generate a feature map containing sufficient boundary information and geometric shape information. Finally, the feature maps generated by the two modules are added together to obtain a feature map F containing sufficient boundary information. r_border The calculation formula is:

[0019] F r_border =G sup (B c , F r1 )+G dep (B c , F r1 )

[0020] Step 6. In order to better achieve task alignment, the present invention also uses the boundary feature aggregation module of step 5 in the classification branch. c_adjust With the feature map F c Input into this module to generate feature map F c_border The calculation formula is:

[0021] F c_border =G sup (B c_adjust , F c )+G dep (B c_adjust , F c )

[0022] Step 7. In the regression branch, replace F r_border As input, a 1×1 convolutional layer is used to generate an adjustment scaling factor F for the initial bounding box. scale , then F scale With B c Multiply to get the precise bounding box B ref In the classification branch, F c_border As input, a 1×1 convolutional layer is used to generate the classification score C.

[0023] Step 8. Calculate the loss and optimize the entire network. In order to improve the prediction accuracy of the classification score, the present invention uses the Varifocal loss; the calculation formula is:

[0024]

[0025] Where c is the predicted classification score, u is the IoU score between the predicted precise bounding box and the true bounding box; α is a hyperparameter that suppresses the imbalance in the number of positive and negative samples, with a value of 0.75; β is a hyperparameter that controls the imbalance in the number of easy / difficult to distinguish samples, with a value of 2.0.

[0026] In order to improve the accuracy of the predicted bounding box positioning, the present invention adopts GIoU loss for the initial bounding box and the precise bounding box respectively; the calculation formula is:

[0027]

[0028] Integrate the above loss functions together to obtain the overall loss function L; the calculation formula is:

[0029]

[0030] Among them, λ0, λ1, and λ2 are hyperparameters for balancing loss, which are set to 1.0, 1.5, and 2.0 respectively. pos is the number of foreground points, i represents all positions in the feature map; C i is the classification score corresponding to position i; is the ground truth bounding box corresponding to position i, B c，i is the initial bounding box corresponding to position i; if i is the foreground point, w c，i is the IoU score between the initial bounding box and the real bounding box corresponding to position i, w ref，i is the IoU score between the precise bounding box and the real bounding box corresponding to position i, otherwise w c，i With w ref，i All are 0.

[0031] The network architecture of the present invention consists of a spatially sensitive region perception module, a salient dependency supplementation module, and a boundary feature aggregation module. The spatially sensitive region perception module adjusts the initial bounding box by predicting a perception scaling factor to obtain the spatially sensitive region of the classification branch. The regression branch directly uses the four boundaries of the initial bounding box as the spatially sensitive region of the regression branch, thus eliminating the problem of spatial coupling. The salient dependency supplementation module is used before the spatially sensitive region perception module. It supplements the dependency relationship between each position in the feature map and the global salient features, thereby more quickly adjusting the spatially sensitive region of the classification branch to the salient region. The boundary feature aggregation module consists of a boundary supplementation feature aggregation module and a boundary representation feature aggregation module. The boundary supplementation feature aggregation module supplements each position in the feature map with sufficient boundary context and geometric shape information; the boundary representation feature aggregation module supplements each position in the feature map with boundary representation information. In addition, the present invention uses the boundary feature aggregation module for each spatially sensitive region in both task branches to achieve alignment between classification scores and localization accuracy. The present invention adopts multiple loss functions, including Varifocal loss and GIoU loss, to optimize the parameters of the overall network, so that the model can simultaneously predict accurate classification scores and positioning accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The network architecture diagram of target detection based on spatial decoupling and boundary feature aggregation of the present invention;

[0033] Figure 2 The significant dependency supplement module structure diagram of the present invention;

[0034] Figure 3 The structural diagram of the spatial sensitive area perception module of the present invention;

[0035] Figure 4 The structural diagram of the boundary feature aggregation module of the present invention;

[0036] Figure 5 The structural diagram of the boundary supplementary feature aggregation module of the present invention;

[0037] Figure 6 The boundary of the present invention represents a feature aggregation module structure diagram. DETAILED DESCRIPTION

[0038] Example 1

[0039] This invention proposes a target detection method based on spatial decoupling and boundary feature aggregation. Figure 1As shown, the overall network architecture consists of a spatially sensitive region perception module, a salient dependency supplementation module, and a boundary feature aggregation module. First, on the regression branch, the fourth-layer feature map of the regression tower is used to predict the initial bounding box, and its four boundaries are used as the spatially sensitive regions of the regression branch. Then, the third-layer feature map of the regression tower and the initial bounding box are input into the boundary feature aggregation module to obtain a feature map containing sufficient boundary information. The feature map is used to predict four perceptual scaling factors to adjust the position of the initial bounding box and thus obtain an accurate bounding box. On the classification branch, the third-layer feature map of the classification tower is input into the salient dependency supplementation module to obtain a feature map containing global salient information relationships. The perceptual scaling factors predicted by the feature map are used to adjust the initial bounding box to the salient region, i.e., the spatially sensitive region of the classification branch. In order to better achieve task alignment, the present invention also uses a boundary feature aggregation module in the classification branch. The adjusted bounding box and the third-layer feature map of the classification tower are input into the module to obtain a feature map containing sufficient salient information, and then the classification score is predicted by the feature map. Finally, the loss is calculated for the initial bounding box, the refined bounding box, and the classification score. This step is only used during training to optimize the parameters of the overall network so that the model can simultaneously predict accurate classification scores and positioning accuracy.

[0040] Next, the implementation steps will be described in detail with reference to the accompanying drawings.

[0041] Step 1. Input the feature map of each layer of FPN into the classification tower and regression tower to obtain the third layer feature map F of the classification tower c , the third layer feature map F of the regression tower r1 And the 4th layer feature map F of the regression tower r The classification tower contains three 3×3 convolutional layers with ReLU activation, and the regression tower contains four 3×3 convolutional layers with ReLU activation.

[0042] Step 2. F r As input, the initial bounding box B is generated by 3×3 convolution c .

[0043] Step 3. F c Input to the significant dependency supplement module. Figure 2 As shown in the figure, the module will go through three processing parts: context module, feature conversion module and fusion module.

[0044] Step 3.1. In the context module, F c It will go through 1×1 convolution layer and SoftMax processing in sequence to form context-related feature map F c_m The calculation formula is:

[0045]

[0046] Among them, W k represents a 1×1 convolutional layer, N p Represents the total number of pixels in the feature map; F c，j Indicates F c The eigenvalue of the jth position in the feature map; F c，n Represents the feature value at the nth position in the feature map.

[0047] Step 3.2. In the feature conversion module, F c_m It will be processed in sequence through 1×1 convolution layer, layer normalization, ReLU and 1×1 convolution layer to obtain the feature map F containing channel-wise dependency. t The calculation formula is:

[0048]

[0049] in, represents a 1×1 convolutional layer, δ represents ReLU, and LN represents layer normalization.

[0050] Step 3.3. The fusion module fuses the global context features into the features of all positions to obtain the feature map F s The calculation formula is:

[0051] F s =F c +F t

[0052] Step 4. Set the initial bounding box B c With F s Input to the spatially sensitive region perception module. Figure 3 As shown, this module will F s As input, after 3×3 convolution layer and Sigmoid processing, the perceptual scaling factor is generated, which is then compared with the initial bounding box B c Multiply to get the adjusted bounding box B c_adjust , where its four boundaries are the spatially sensitive areas of the classification branches; the calculation formula is:

[0053] B c_adjust =B c ×σ(f conv (F s ))

[0054] where f conv Represents the 3×3 convolution layer processing function; σ represents Sigmoid, B c ∈R H×W×4 , B c_adjust ∈R H×W×4 .

[0055] Step 5. Set the initial bounding box Bc And the third layer feature map F of the regression tower r1 Input to the boundary feature aggregation module. Figure 4 As shown in the figure, the boundary feature aggregation module is divided into a boundary representative feature aggregation module and a boundary supplementary feature aggregation module. The feature maps output by the two sub-modules are added together to form an output feature map containing sufficient boundary information.

[0056] Step 5.1. The boundary representation feature aggregation module structure is as follows Figure 5 As shown. A 1×1 convolutional layer with a feature map F with a channel number of C is used. r1 Expand it to a boundary-sensitive feature map with 5C channels (the first C channels represent single-point features, and the last 4C channels correspond to 4 edges). Then, combined with the initial bounding box, the boundary extreme points are extracted from the boundary-sensitive feature map through the border alignment module (BorderAlign) as the representative features of the boundary. Then, a 1×1 convolutional layer is used to restore the module input dimension, and finally a feature map containing boundary representative information is generated. The calculation formula is:

[0057] G dep (B c , F r1 )=f conv_c (BorderAlign(f conv_5c (F r1 ), B c ))

[0058] where f conv_c represents a function that compresses a feature map with 5C channels into a feature map with C channels through a 1×1 convolutional layer; f conv_5c It represents a function that expands a feature map with C channels into a feature map with 5C channels through a 1×1 convolutional layer.

[0059] Step 5.2. The boundary supplement feature aggregation module structure is as follows Figure 6 As shown. According to the initial bounding box B c Manually select 9 fixed sampling points, use their relative offsets from the origin to replace the offset predicted by the deformable convolution, and then r1 The projected point features of the nine relative offset positions are deformed and convolved to generate a feature map containing sufficient boundary information and geometric shape information. The calculation formula is:

[0060] G sup (B c , F r1 )=f convd (F r1 , Δp n )

[0061] where Δp n ∈RH×W×18 , represents the relative offset of the 9 sampling points in the x and y directions; f convd Represents the variable convolution processing function.

[0062] Step 6. To better achieve task alignment, the present invention also uses the boundary feature aggregation module of step 5 in the classification branch, except that the input of the boundary feature aggregation module becomes the adjusted bounding box and the third layer feature map of the classification tower.

[0063] Step 7. In the regression branch, the feature map generated by the boundary feature aggregation module is used as input. A 1×1 convolutional layer is used to generate a feature map with the adjusted scaling factor for the initial bounding box. This feature map is then multiplied with the initial bounding box to obtain the refined bounding box. In the classification branch, the feature map generated by the boundary feature aggregation module is used as input. A 1×1 convolutional layer is used to generate the classification score.

[0064] Step 8. Calculate the loss and optimize the entire network so that the model can simultaneously predict accurate classification scores and positioning accuracy.

[0065] In order to improve the prediction accuracy of the classification score, the present invention adopts Varifocal loss; the calculation formula is:

[0066]

[0067] Where c is the predicted classification score, u is the IoU score between the predicted precise bounding box and the true bounding box; α is a hyperparameter that suppresses the imbalance in the number of positive and negative samples, with a value of 0.75; β is a hyperparameter that controls the imbalance in the number of easy / difficult to distinguish samples, with a value of 2.0.

[0068] In order to improve the accuracy of the predicted bounding box positioning, the present invention adopts GIoU loss for the initial bounding box and the precise bounding box respectively; the calculation formula is:

[0069]

[0070] Integrate the above loss functions together to obtain the overall loss function L; the calculation formula is:

[0071]

[0072] Among them, λ0, λ1, and λ2 are hyperparameters for balancing loss, which are set to 1.0, 1.5, and 2.0 respectively. pos is the number of foreground points, i represents all positions in the feature map; C i is the classification score corresponding to position i; is the ground truth bounding box corresponding to position i, B c，i is the initial bounding box corresponding to position i; if i is the foreground point, wc，i is the IoU score between the initial bounding box and the real bounding box corresponding to position i, w ref，i is the IoU score between the precise bounding box and the real bounding box corresponding to position i, otherwise w c，i With w ref，i All are 0.

Claims

1. Target detection method based on spatial decoupling and boundary feature aggregation, characterized by The steps include: Step 1. Input the feature map of each layer of FPN into the classification tower and regression tower to obtain the third layer feature map F of the classification tower c , the third layer feature map F of the regression tower r1 And the 4th layer feature map F of the regression tower r ; The classification tower contains three 3×3 convolutional layers with ReLU activation, and the regression tower contains four 3×3 convolutional layers with ReLU activation; Step 2. F r As input, the initial bounding box B is generated by 3×3 convolution c ; Step 3. F c Input to the significant dependency supplement module S re ; This module will go through three parts of processing: context module, feature conversion module and fusion module, and obtain the feature map F containing the dependency relationship with the global saliency feature s The calculation formula is: F s =S re (F c ) Step 4. Set the initial bounding box B c With F s Input to the spatially sensitive region perception module S area ; This module will F s As input, the perceptual scaling factor is generated and then compared with the initial bounding box B c Multiply to get the adjusted bounding box B c_adjust , where its four boundaries are the spatially sensitive areas of the classification branches; The calculation formula is: B c_adjust =S area (B c ,F s ) Step 5. In the regression branch, the initial bounding box B c With the feature map F r1 Input to the boundary feature aggregation module; this module is supplemented by the boundary feature aggregation module G sup and boundary representation feature aggregation module G dep Composition; G dep Use the boundary alignment module to c With F r1 Input to this module to generate a feature map containing boundary representation information; G sup Using the star-shaped bounding box feature representation module, B c With F r1 Input into this module to generate a feature map containing boundary information and geometric shape information; finally, the feature maps generated by the two modules are added together to obtain a feature map F containing boundary information r_border The calculation formula is: F r_border =G sup (B c ,F r1 )+G dep (B c ,F r1 ) Step 6. To align the classification and localization tasks, the boundary feature aggregation module of step 5 is also used in the classification branch; The adjusted bounding box B c_adjust With the feature map F c Input into this module to generate feature map F c_border The calculation formula is: F c_border =G sup (B c_adjust ,F c )+G dep (B c_adjust ,F c ) Step 7. In the regression branch, replace F r_border As input, a 1×1 convolutional layer is used to generate the adjusted scaling factor feature map F for the initial bounding box. scale ; Then F scale With B c Multiply to get the precise bounding box B ref ; In the classification branch, F c_border As input, a 1×1 convolutional layer is used to generate the classification score C; Step 8. Calculate the loss and optimize the overall network.

2. The target detection method based on spatial decoupling and boundary feature aggregation according to claim 1 is characterized in that The step 3 includes the following sub-steps: Step 3.

1. In the context module, F c It will go through 1×1 convolution layer and SoftMax processing in turn to obtain the context-related feature map F c_m The calculation formula is: Among them, W k represents a 1×1 convolutional layer, N p Represents the total number of pixels in the feature map; F c,j Indicates F c The eigenvalue of the jth position in the feature map; F c,n Represents the eigenvalue of the nth position in the feature map; Step 3.

2. In the feature conversion module, F c_m It will be processed in sequence through 1×1 convolution layer, layer normalization, ReLU and 1×1 convolution layer to obtain the feature map F containing channel-wise dependency. t The calculation formula is: in, represents a 1×1 convolutional layer, δ represents ReLU, and LN represents layer normalization; Step 3.

3. The fusion module will fuse the global context features into the features of all positions to obtain the feature map F s The calculation formula is: F s =F c +F t 。 3. The target detection method based on spatial decoupling and boundary feature aggregation according to claim 1 is characterized in that The step 4 includes the following sub-steps: Step 4.

1. Initial bounding box B c With F s Input to the spatially sensitive area perception module; This module will first s As input, after 3×3 convolution layer and Sigmoid processing, the perceptual scaling factor is generated, which is then compared with the initial bounding box B c Multiply to get the adjusted bounding box B c_adjust , where its four boundaries are the spatially sensitive areas of the classification branches; The calculation formula is: B c_adjust =B c ×σ(f conv (F s )) where f conv represents the 3×3 convolution layer processing function; σ represents Sigmoid; B c ∈R H×W×4 , B c_adjust ∈R H×W×4 .

4. The target detection method based on spatial decoupling and boundary feature aggregation according to claim 1 is characterized in that The step 5 includes the following sub-steps: Step 5.

1. Boundary representation feature aggregation module structure Use the feature map F with a 1×1 convolution layer and a channel number of C r1 Expanded to a boundary-sensitive feature map with 5C channels, Combined with the initial bounding box, the boundary alignment module BorderAlign extracts the extreme points of the boundary from the boundary-sensitive feature map as the representative features of the boundary. Use a 1×1 convolutional layer to restore the module input dimension and generate a feature map containing boundary representation information; the calculation formula is: G dep (B c ,F r1 )=f conv_c (BorderAlign(f conv_5c (F r1 ),B c )) where f conv_c represents a function that compresses a feature map with 5C channels into a feature map with C channels through a 1×1 convolutional layer; f conv_5c Represents a function that expands a feature map with C channels into a feature map with 5C channels through a 1×1 convolutional layer; Step 5.

2. Boundary Supplementary Feature Aggregation Module Structure According to the initial bounding box B c Manually select several fixed sampling points and use their relative offsets from the origin to replace the offsets predicted by the deformable convolution. In the feature map F r1 Perform deformable convolution on the projection point features of these relative offset positions. Generate a feature map containing boundary information and geometric shape information; the calculation formula is: G sup (B c ,F r1 )=f convd (F r1 ,Δp n ) where Δp n ∈R H×W×18 , represents the relative offset of the 9 sampling points in the x and y directions; f convd Represents the variable convolution processing function.

5. The target detection method based on spatial decoupling and boundary feature aggregation according to claim 1 is characterized in that The step 8 includes the following sub-steps: Step 8.

1. In order to improve the prediction accuracy of the classification score, the Varifocal loss is used; the calculation formula is: Where c is the predicted classification score, u is the IoU score between the predicted accurate bounding box and the true bounding box; α is a hyperparameter to suppress the imbalance in the number of positive and negative samples; β is a hyperparameter to control the imbalance in the number of easy / difficult to distinguish samples; Step 8.

2. To improve the accuracy of bounding box positioning, GIoU loss is used for the initial bounding box and the refined bounding box respectively; the calculation formula is: Step 8.

3. Combine the loss functions to get the overall loss function L; the calculation formula is: Where λ0, λ1, λ2 are hyperparameters used to balance the loss, N pos is the number of foreground points, i represents all positions in the feature map; C i is the classification score corresponding to position i; is the ground truth bounding box corresponding to position i, B c,i is the initial bounding box corresponding to position i; If i is the foreground point, w c,i is the IoU score between the initial bounding box and the real bounding box corresponding to position i, w ref,i is the IoU score between the precise bounding box and the real bounding box corresponding to position i, otherwise w c,i With w ref,i All are 0.

Images