Arbitrary angle target detection method based on coarse mask smoothing label supervision

Abstract

This invention relates to the field of artificial intelligence deep learning and computational visual object detection and segmentation technology. It utilizes the GLSMask-RCNN model to detect non-horizontal targets at arbitrary angles in visible light remote sensing images, exhibiting excellent detection performance for targets with large aspect ratios and dense distribution. Because this invention is a pixel-level mask segmentation-based detection method and introduces adaptive 2D Gaussian label smoothing to optimize coarse label supervision, it solves common boundary problems in non-horizontal target detection tasks during both training and actual detection phases, resulting in more stable model performance. This invention provides an effective new solution for target detection in visible light remote sensing images and is applicable to non-horizontal target detection scenarios in other image types.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence deep learning and computing, as well as visual target detection and segmentation technology, and specifically relates to an arbitrary angle target detection method based on coarse mask smoothing label supervision. Background Technology

[0002] With the continuous development of remote sensing imaging technology, remote sensing images are widely used in major fields such as military strategy, urban planning, public security investigation, and traffic control, serving people's production and life, social stability, and national security from different levels and directions. According to the different electromagnetic wavebands recorded, remote sensing images can be divided into several major categories: ultraviolet, visible light, near-infrared, thermal infrared, microwave images, and multi-band and ultra-multi-band images. Although visible light remote sensing images are greatly affected by factors such as illumination and fog, under clear weather and good sea conditions, they better reflect the shape of targets and are easier for the human eye to identify. Therefore, research on target detection methods based on visible light remote sensing images has received much attention in recent years. Because visible light remote sensing images contain many non-horizontal targets with large aspect ratios, such as ships, vehicles, ports, and aircraft, common horizontal bounding boxes cannot effectively and accurately locate these targets. Therefore, in visible light remote sensing images, the coordinates of the four corner points of a quadrilateral are often used to represent the bounding box of a target in any direction. Therefore, how to effectively regress non-horizontal bounding boxes in any direction has become the primary problem in the field of target detection in visible light remote sensing images. To address this problem, researchers have proposed numerous solutions, with the most effective ones falling into the following categories: 1. Five-parameter methods based on angle parameter regression; 2. Eight-parameter methods based on the regression of four corner coordinates; 3. Angle-based classification methods; 4. Methods based on center point and vector prediction; 5. Methods based on rotating regions of interest. Currently, methods for detecting non-horizontal targets at arbitrary angles are gradually maturing. However, due to the periodicity of angles and the order of corner points or vectors, boundary problems cause most models to exhibit performance instability or difficulty in training at angular boundaries. Summary of the Invention

[0003] The purpose of this invention is to provide an arbitrary-angle target detection method based on coarse mask smoothing label supervision. The main structure of this method is based on Cascade Mask R-CNN, primarily comprising a Backbone network, a Neck feature pyramid network, an RPN region proposal network, a BBOX head network, and a GLSMask head network. Before training the network, adaptive augmentation and enhancement of the data are performed, and corresponding network hyperparameter configurations are set. During training, a loss function is calculated for each output, and the gradient obtained from the derivative is backpropagated for gradient descent parameter updates, allowing the model to learn the corresponding detection capabilities.

[0004] The arbitrary angle target detection method based on coarse mask smoothing label supervision of the present invention includes the following steps:

[0005] 1.1 Obtaining and preprocessing the dataset includes the following steps:

[0006] 1.1.1 Based on the official labeling standard of the DOTA dataset, which specifies that each target contains a category and is labeled with x-axis 1, y-axis 1, x-axis 2, y-axis 2, x-axis 3, y-axis 3, x-axis 4, and y-axis 4, target detection bounding boxes are assigned to the visible light remote sensing image dataset to be processed. The visible light remote sensing image dataset is divided into training, validation, and test sets in a 7:2:1 ratio. If there is a limited amount of training data, random sampling can be performed on the official DOTA dataset, and the visible light remote sensing image dataset can be padded to a size comparable to the DOTA dataset.

[0007] 1.1.2 Enhancement preprocessing is performed on the visible light remote sensing image dataset with detection box labels obtained in step 1.1.1, including random flipping, random angle rotation, random cropping, Gaussian blur, Gaussian noise, contrast enhancement, color dithering, and brightness enhancement, in order to enhance the feature diversity and scale of the dataset; among them, enhancement schemes involving image coordinate position transformation need to perform the same transformation on the target detection box labels with the same bias.

[0008] 1.2 Constructing the GLSMask-RCNN Network

[0009] The parameter settings for the GLSMask-RCNN network include the following steps:

[0010] 1.2.1 Constructing the Backbone Feature Extraction Network: The Backbone network is used to extract features from the input image. Its overall structure is ResNet101, and it includes the following steps:

[0011] 1.2.1.1 ResNet101 is a deep convolutional neural network. Its feature is the introduction of residual structure to form a residual network. During the forward and backward propagation of the model, it has a skip connection information propagation path, which can effectively avoid the gradient vanishing and gradient explosion phenomena common in deep neural networks, and can increase the depth of the model to perform efficient feature extraction.

[0012] 1.2.1.2 The most basic structure of a residual network is a residual block, which is divided into two branches, including the main path and residual edges. By stacking residual block structures, residual networks can process the input RGB image and finally obtain deep features with 2048 channels and a downsampling rate of 32.

[0013] 1.2.1.3 The main path includes three convolution operations, with the following sequential structure: a convolutional layer with a kernel size of 1×1, a batch normalization layer, a ReLU activation function, a convolutional layer with a kernel size of 3×3, a batch normalization layer, a ReLU activation function layer, a convolutional layer with a kernel size of 1×1, a batch normalization layer, and a ReLU activation function layer.

[0014] 1.2.1.4 The residual edge is a summation structure that jumps between the input and output of the main path. The residual network downsamples the input image 5 times, which is accomplished by adjusting the stride of the first convolutional layer of the residual block to 2 and adding a convolutional layer with a stride of 2 to the residual edge.

[0015] 1.2.2 Constructing the Neck Feature Pyramid Feature Fusion Structure: Feature fusion is performed using the Feature Pyramid Network (FPN), including the following steps:

[0016] 1.2.2.1 In general, FPN upsamples the intermediate features generated by ResNet101 during feature extraction and fuses them with the corresponding features from the previous layer to achieve effective fusion of features at different levels of the input image.

[0017] 1.2.2.2 FPN takes the features from the second to fifth layers of ResNet101 as input. The processing of each layer consists of two stages: upsampling and fusion. The upsampling operation first performs a 1×1 convolution on the deep feature map to reduce the number of channels, and then enlarges the feature map size using bilinear interpolation. The fusion operation first performs a simple pixel-by-pixel summation between the upsampled feature map and the feature map from the previous layer, and then further refines the fusion of deep and shallow features by performing a 3×3 convolution on the fused result.

[0018] 1.2.2.3 The feature map of the fifth-level backbone network is directly obtained from the feature map of the fourth-level FPN through the convolutional layers with 1×1 kernel and 3×3 kernel in the FPN; the feature map of the fifth-level FPN is obtained by worst-case pooling downsampling from the feature map of the fourth-level FPN.

[0019] 1.2.2.4 The output of the FPN network consists of five layers, each with a different feature map size and 256 channels.

[0020] 1.2.3 Region Proposal Network (RPN): Inheriting the Region Proposal Network from the Mask-RCNN model, its function is to generate region of interest (ROI) location suggestions for the target based on the input feature map, including the following steps:

[0021] 1.2.3.1 The input features of the Region Proposal Network are the output features of each layer of the FPN; structurally, the RPN consists of a convolutional layer with a kernel size of 3×3 and two parallel convolutional layers with a kernel size of 1×1.

[0022] 1.2.3.2 The two output channels of RPN are 18 and 36 respectively. This means that for each input feature pixel position, it matches 9 preset anchor boxes with different aspect ratios and sizes. For each anchor box, it outputs 2 prediction scores representing the probabilities of the foreground and background respectively, and 4 bias parameters representing the adjustment of the anchor box to the proposal box: x, y, h, w.

[0023] 1.2.3.3 Select 2000 proposal boxes according to the foreground score and perform non-maximum suppression to obtain the final proposal box parameters;

[0024] 1.2.3.4 By truncating the output features of each layer of FPN according to the suggestion box parameters, the location suggestion of the region of interest of the target can be obtained;

[0025] 1.2.4 ROIAlign: For the region of interest features of different sizes generated by RPN, the bilinear interpolation algorithm is used to downsample them to obtain region of interest features with uniform spatial sizes of 7×7 and 14×14.

[0026] 1.2.5 BBOX Head Network: The BBOX head network predicts the target category and bounding box parameters based on the features of the input region of interest; this part includes the following steps:

[0027] 1.2.5.1 The input features of the BBOX head network are regions of interest features of size 7×7;

[0028] 1.2.5.2 The sequential structure of the BBOX head network consists of a 3×3 convolutional layer, a 1×1 convolutional layer, and a parallel classifier and regressor. The classifier consists of a global average pooling layer and a fully connected layer for BBOX categories. The regressor consists of a global average pooling layer and a fully connected layer for BBOX parameter regression.

[0029] 1.2.5.3 The number of neurons in the BBOX category fully connected layer is equal to the number of categories. The output category prediction vector is then processed by softmax activation to obtain the normalized prediction probability of each category.

[0030] 1.2.5.4 BBOX Parameters: The number of neurons in the fully connected layer of the regression is equal to the number of categories multiplied by 4, which is used to progressively refine the extraction of the region of interest;

[0031] 1.2.6 GLSMask Header Network: The GLSMask head network predicts a pixel-level segmentation mask for the target based on the input region of interest features, including the following steps:

[0032] 1.2.6.1 The input features of the GLSMask head network are region of interest features of size 14×14;

[0033] 1.2.6.2 The sequential structure of the GLSMask header network consists of a convolutional layer with a kernel size of 3×3, a transposed convolutional layer with a kernel size of 2×2, and a convolutional layer with a kernel size of 1×1.

[0034] 1.2.6.3 The final output size of the GLSMask header network is 28×28×number of classes, that is, the number of channels equals the number of classes, and a segmentation mask is predicted for each class corresponding to each target;

[0035] 1.2.7 Head Stacking Optimization Structure: The parallel head structure composed of the BBOX head network and the GLSMask head network is stacked three times. For the head structure of stage one, the input is the region of interest feature obtained by the RPN based on the proposal box. For the head structures of stages two and three, the region of interest feature is obtained based on the detection box output by the previous head structure.

[0036] 1.2.8 Generation of Arbitrary Angle Detection Boxes Based on Masks: A three-layer cascaded stacked header structure is used to obtain the optimal target mask and category prediction at the final level. First, the 28×28 final mask prediction features are upsampled using bilinear interpolation to obtain a pixel-wise segmentation mask prediction probability map with spatial dimensions consistent with the detection box size output by the third-level header. According to a preset threshold, the probability map is binarized to obtain the final target segmentation mask. The segmentation mask is denoised and made continuous, and the corresponding minimum bounding rectangle is generated. This minimum bounding rectangle is the detection box for any angle of the target.

[0037] 1.3 Model Training:

[0038] 1.3.1 Configure relevant data iteration parameters for model training, including batchsize=8, max_epoch=100, num_workers=4;

[0039] 1.3.2 The Adam optimizer is used with an initial learning rate of 0.002, momentum of 0.5, smoothing parameters β1 = 0.9, β2 = 0.999, and weight_decay = 0.0001. The learning rate is increased linearly during the warm-up phase and adjusted using the StepLR strategy during the training phase.

[0040] 1.3.2 GSL Tag Segmentation Coarse Tag Optimization:

[0041] In the dataset of the arbitrary angle target detection task, the label type is the coordinates of the four corner points of the quadrilateral detection box. In this method, for each target, the segmentation mask information is generated based on the four corner points. However, for most targets, such as ships, ports, and aircraft, the mask generated in this way covers the texture information of the background around the edges, which is a coarse label. This method provides a two-dimensional Gaussian label smoothing technique with adaptive width and height, which reduces and smooths the edges of the 0 / 1 segmentation mask label, so that the processed segmentation mask label discards useless information and retains information that is more conducive to the semantic and directional representation of the target itself.

[0042] 1.4 Loss Function

[0043] The model's loss function is generally divided into two parts: the RPN multi-task loss function and the multi-task loss function of the head network output at each layer, which are described in detail below:

[0044] 1.4.1 The RPN multi-task loss function is expressed as follows:

[0045]

[0046] Where: p i This represents the probability that the i-th anchor is predicted to be the true label; This represents the true class label of the i-th anchor, which is 1 when it is a positive sample and 0 when it is a negative sample; t i This represents the bounding box regression parameters for predicting the i-th anchor; N represents the regression parameters of the ground truth bounding box corresponding to the i-th anchor; cls N represents the total number of samples in a mini-batch, i.e., the number of positive and negative samples at the final sampling point, which is 256; reg This indicates the number of anchor positions, approximately 2400; The binary cross-entropy loss is expressed as:

[0047]

[0048] The smooth L1 loss function is expressed as follows:

[0049]

[0050] 1.4.2 The multi-task loss function output by the head network at each level is expressed as follows:

[0051] L(p,u,t) u ,v)=Lcls (p, u) + λ[u≥1]L loc (t u ,v)+L GSLMask

[0052] Where: p is the softmax probability distribution predicted by the classifier, p = (p0, ..., p... k ); u represents the corresponding target's true category label; t u The regression parameters for the corresponding category u predicted by the corresponding bounding box regressor v is the bounding box regression parameter corresponding to the true target (v x ,v y ,v w ,v h );L cls (p,u) is the cross-entropy loss function, expressed as:

[0053] L cls (p,u)=-logp u

[0054] L loc (t u (v) is the Smooth L1 loss function, expressed as:

[0055]

[0056] Where: L GSLMask The expression for the two-dimensional cross-entropy loss function optimized using adaptive two-dimensional Gaussian label smoothing is:

[0057]

[0058] Where: p i,j N represents the predicted probability of the feature points output by the GSLMask head network; N is the size of the feature map output by the GSLMask head network. W(i,j) is the window function for adaptive two-dimensional Gaussian label smoothing, and its expression is:

[0059]

[0060] Where: θ is the included angle calculated based on the coordinates of the four corner points of the coarse label, specifically the angle between the line connecting the midpoints of the opposite longer sides of the quadrilateral and the x-axis; x, y are the pixel coordinates of the feature map output by the GSLMask head network;

[0061]

[0062]

[0063] Where: d{·} represents the distance between two points; D is a hyperparameter, defaulted to 4; m * ,n * These are the x-coordinates and y-coordinates of the midpoints of each side, calculated using the four corner points.

[0064] This invention employs a non-horizontal arbitrary-angle target detection method based on coarse mask-supervised training using GLSMask-RCNN (Self-adaptive Gaussian Label SmoothingMask R-CNN). The model structure used in this invention includes a ResNet101 backbone network, a Neck Feature Pyramid Network (FPN), a Region Proposal Network (RPN), a BBOX head network, and a GLSMask head network. The visible light remote sensing image to be detected is input into the model. First, features are extracted through the ResNet101 backbone network, and the features at each level are then input into the Feature Pyramid Network for feature enhancement and fusion, resulting in a feature map containing rich information at both deep and shallow layers. The feature map is input into the RPN to generate corresponding region proposal boxes, and based on the region proposal boxes and the feature map, a Region of Interest (ROI) feature is generated. The RPI feature is then input into the BBOX head network and the GLSMask head network for target generation category prediction, detection box regression parameters, and segmentation mask prediction, respectively. The segmentation mask prediction map is binarized to obtain a binary segmentation mask map. The minimum bounding rectangle is generated from the segmented mask binary image, and the detection of non-horizontal targets in any direction is completed.

[0065] This invention utilizes the GLSMask-RCNN model to detect non-horizontal targets at arbitrary angles in visible light remote sensing images, demonstrating excellent detection performance for targets with large aspect ratios and dense distribution. Because this invention employs a pixel-level mask segmentation-based detection method and introduces adaptive 2D Gaussian label smoothing to optimize coarse label supervision, it effectively addresses common boundary issues in non-horizontal target detection during both training and actual detection phases, resulting in more stable model performance. This invention provides an effective new solution for target detection in visible light remote sensing images and is applicable to non-horizontal target detection scenarios in other image types. Attached Figure Description

[0066] Figure 1 This is a flowchart of an arbitrary angle target detection method based on coarse mask smoothing label supervision;

[0067] Figure 2 The flowchart for the GLSMask-RCNN model;

[0068] Figure 3 This is a structural diagram of the GLSMaskHead network;

[0069] Figure 4 A schematic diagram of the segmentation mask label marking points;

[0070] Figure 5 A planar coordinate graph of the adaptive two-dimensional Gaussian label smoothing window function;

[0071] Figure 6 A three-dimensional coordinate graph of the adaptive two-dimensional Gaussian label smoothing window function;

[0072] Figure 7 A flowchart and visualization of the adaptive 2D Gaussian label smoothing process;

[0073] Figure 8 This is a demonstration of the detection performance of GSL-MaskRCNN after 100 iterations of training. Detailed Implementation

[0074] The present invention will now be described in conjunction with the accompanying drawings.

[0075] like Figure 1 As shown, the arbitrary angle target detection method based on coarse mask smoothing label supervision of the present invention includes the following steps:

[0076] 1.1 Obtaining and preprocessing the dataset includes the following steps:

[0077] 1.1.2 Based on the official labeling standard of the DOTA dataset, which specifies that each target contains a category and is labeled with x-axis 1, y-axis 1, x-axis 2, y-axis 2, x-axis 3, y-axis 3, x-axis 4, and y-axis 4, target detection bounding boxes are assigned to the visible light remote sensing image dataset to be processed. The visible light remote sensing image dataset is divided into training, validation, and test sets in a 7:2:1 ratio. If there is a limited amount of training data, random sampling can be performed on the official DOTA dataset, and the visible light remote sensing image dataset can be padded to a size comparable to the DOTA dataset.

[0078] 1.1.2 Enhancement preprocessing is performed on the visible light remote sensing image dataset with detection box labels obtained in step 1.1.1, including random flipping, random angle rotation, random cropping, Gaussian blur, Gaussian noise, contrast enhancement, color dithering, and brightness enhancement, in order to enhance the feature diversity and scale of the dataset; among them, enhancement schemes involving image coordinate position transformation need to perform the same transformation on the target detection box labels with the same bias.

[0079] 1.2 Constructing the GLSMask-RCNN Network

[0080] The structure and flow of the GLSMask-RCNN network are as follows: Figure 2As shown, the flow of each part of the network structure includes the following steps:

[0081] 1.2.1 Constructing the Backbone Feature Extraction Network: The Backbone network is used to extract features from the input image. Its overall structure is ResNet101, and it includes the following steps:

[0082] 1.2.1.1 ResNet101 is a deep convolutional neural network. Its feature is the introduction of residual structure to form a residual network. During the forward and backward propagation of the model, it has a skip connection information propagation path, which can effectively avoid the gradient vanishing and gradient explosion phenomena common in deep neural networks, and can increase the depth of the model to perform efficient feature extraction.

[0083] 1.2.1.2 The most basic structure of a residual network is a residual block, which is divided into two branches, including the main path and residual edges. By stacking residual block structures, residual networks can process the input RGB image and finally obtain deep features with 2048 channels and a downsampling rate of 32.

[0084] 1.2.1.3 The main path includes three convolution operations, with the following sequential structure: a convolutional layer with a kernel size of 1×1, a batch normalization layer, a ReLU activation function, a convolutional layer with a kernel size of 3×3, a batch normalization layer, a ReLU activation function layer, a convolutional layer with a kernel size of 1×1, a batch normalization layer, and a ReLU activation function layer.

[0085] 1.2.1.4 The residual edge is a summation structure that jumps between the input and output of the main path. The residual network downsamples the input image 5 times, which is accomplished by adjusting the stride of the first convolutional layer of the residual block to 2 and adding a convolutional layer with a stride of 2 to the residual edge.

[0086] 1.2.2 Constructing the Neck Feature Pyramid Feature Fusion Structure: Feature fusion is performed using the Feature Pyramid Network (FPN), including the following steps:

[0087] 1.2.2.1 In general, FPN upsamples the intermediate features generated by ResNet101 during feature extraction and fuses them with the corresponding features from the previous layer to achieve effective fusion of features at different levels of the input image.

[0088] 1.2.2.2 FPN takes the features from the second to fifth layers of ResNet101 as input. The processing of each layer consists of two stages: upsampling and fusion. The upsampling operation first performs a 1×1 convolution on the deep feature map to reduce the number of channels, and then enlarges the feature map size using bilinear interpolation. The fusion operation first performs a simple pixel-by-pixel summation between the upsampled feature map and the feature map from the previous layer, and then further refines the fusion of deep and shallow features by performing a 3×3 convolution on the fused result.

[0089] 1.2.2.3 The feature map of the fifth-level backbone network is directly obtained from the feature map of the fourth-level FPN through the convolutional layers with 1×1 kernel and 3×3 kernel in the FPN; the feature map of the fifth-level FPN is obtained by worst-case pooling downsampling from the feature map of the fourth-level FPN.

[0090] 1.2.2.4 The output of the FPN network consists of five layers, each with a different feature map size and 256 channels.

[0091] 1.2.3 Region Proposal Network (RPN): Inheriting the Region Proposal Network from the Mask-RCNN model, its function is to generate region of interest (ROI) location suggestions for the target based on the input feature map, including the following steps:

[0092] 1.2.3.1 The input features of the Region Proposal Network are the output features of each layer of the FPN; structurally, the RPN consists of a convolutional layer with a kernel size of 3×3 and two parallel convolutional layers with a kernel size of 1×1.

[0093] 1.2.3.2 The two output channels of RPN are 18 and 36 respectively. This means that for each input feature pixel position, it matches 9 preset anchor boxes with different aspect ratios and sizes. For each anchor box, it outputs 2 prediction scores representing the probabilities of the foreground and background respectively, and 4 bias parameters representing the adjustment of the anchor box to the proposal box: x, y, h, w.

[0094] 1.2.3.3 Select 2000 proposal boxes according to the foreground score and perform non-maximum suppression to obtain the final proposal box parameters;

[0095] 1.2.3.4 By truncating the output features of each layer of FPN according to the suggestion box parameters, the location suggestion of the region of interest of the target can be obtained;

[0096] 1.2.4 ROIAlign: For the region of interest features of different sizes generated by RPN, the bilinear interpolation algorithm is used to downsample them to obtain region of interest features with uniform spatial sizes of 7×7 and 14×14.

[0097] 1.2.5 BBOX Head Network: The BBOX head network predicts the target category and bounding box parameters based on the features of the input region of interest; this part includes the following steps:

[0098] 1.2.5.1 The input features of the BBOX head network are regions of interest features of size 7×7;

[0099] 1.2.5.2 The sequential structure of the BBOX head network consists of a 3×3 convolutional layer, a 1×1 convolutional layer, and a parallel classifier and regressor. The classifier consists of a global average pooling layer and a fully connected layer for BBOX categories. The regressor consists of a global average pooling layer and a fully connected layer for BBOX parameter regression.

[0100] 1.2.5.3 The number of neurons in the BBOX category fully connected layer is equal to the number of categories. The output category prediction vector is then processed by softmax activation to obtain the normalized prediction probability of each category.

[0101] 1.2.5.4 BBOX Parameters: The number of neurons in the fully connected layer of the regression is equal to the number of categories multiplied by 4, which is used to progressively refine the extraction of the region of interest;

[0102] 1.2.6 GLSMask Header Network: The structure of the GLSMask header network is as follows: Figure 3 As shown. The GLSMask head network predicts a pixel-level segmentation mask for the target based on the input region of interest features, including the following steps:

[0103] 1.2.6.1 The input features of the GLSMask head network are region of interest features of size 14×14;

[0104] 1.2.6.2 The sequential structure of the GLSMask header network consists of a convolutional layer with a kernel size of 3×3, a transposed convolutional layer with a kernel size of 2×2, and a convolutional layer with a kernel size of 1×1.

[0105] 1.2.6.3 The final output size of the GLSMask header network is 28×28×number of classes, that is, the number of channels equals the number of classes, and a segmentation mask is predicted for each class corresponding to each target;

[0106] 1.2.7 Head Stacking Optimization Structure: The parallel head structure composed of the BBOX head network and the GLSMask head network is stacked three times. For the head structure of stage one, the input is the region of interest feature obtained by the RPN based on the proposal box. For the head structures of stages two and three, the region of interest feature is obtained based on the detection box output by the previous head structure.

[0107] 1.2.8 Generation of Arbitrary Angle Detection Boxes Based on Masks: A three-layer cascaded stacked header structure is used to obtain the optimal target mask and category prediction at the final level. First, the 28×28 final mask prediction features are upsampled using bilinear interpolation to obtain a pixel-wise segmentation mask prediction probability map with spatial dimensions consistent with the detection box size output by the third-level header. According to a preset threshold, the probability map is binarized to obtain the final target segmentation mask. The segmentation mask is denoised and made continuous, and the corresponding minimum bounding rectangle is generated. This minimum bounding rectangle is the detection box for any angle of the target.

[0108] 1.3 Model Training:

[0109] 1.3.1 Configure relevant data iteration parameters for model training, including batchsize=8, max_epoch=100, num_workers=4;

[0110] 1.3.2 The Adam optimizer is used with an initial learning rate of 0.002, momentum of 0.5, smoothing parameters β1 = 0.9, β2 = 0.999, and weight_decay = 0.0001. The learning rate is increased linearly during the warm-up phase and adjusted using the StepLR strategy during the training phase.

[0111] 1.3.2 GSL Tag Segmentation Coarse Tag Optimization:

[0112] In the dataset of the arbitrary angle target detection task, the label type is the coordinates of the four corner points of the quadrilateral detection box. In this method, for each target, the segmentation mask information is generated based on the four corner points. However, for most targets, such as ships, ports, and aircraft, the mask generated in this way covers the texture information of the background around the edges, which is a coarse label. This method provides a two-dimensional Gaussian label smoothing technique with adaptive width and height, which reduces and smooths the edges of the 0 / 1 segmentation mask label, so that the processed segmentation mask label discards useless information and retains information that is more conducive to the semantic and directional representation of the target itself.

[0113] 1.4 Loss Function

[0114] The model's loss function is generally divided into two parts: the RPN multi-task loss function and the multi-task loss function of the head network output at each layer, which are described in detail below:

[0115] 1.4.1 The RPN multi-task loss function is expressed as follows:

[0116]

[0117] Where: p i This represents the probability that the i-th anchor is predicted to be the true label; This represents the true class label of the i-th anchor, which is 1 when it is a positive sample and 0 when it is a negative sample; t i This represents the bounding box regression parameters for predicting the i-th anchor; N represents the regression parameters of the ground truth bounding box corresponding to the i-th anchor; cls N represents the total number of samples in a mini-batch, i.e., the number of positive and negative samples at the final sampling point, which is 256; reg This indicates the number of anchor positions, approximately 2400; The binary cross-entropy loss is expressed as:

[0118]

[0119] The smooth L1 loss function is expressed as follows:

[0120]

[0121] 1.4.2 The multi-task loss function output by the head network at each level is expressed as follows:

[0122] L(p,u,t) u ,v)=L cls (p, u) + λ[u≥1]L loc (t u ,v)+L GSLMask

[0123] Where: p is the softmax probability distribution predicted by the classifier, p = (p0, ..., pk); u is the true class label of the corresponding target; t u The regression parameters for the corresponding category u predicted by the corresponding bounding box regressor v is the bounding box regression parameter corresponding to the true target (v x ,v y ,v w ,v h );L cls (p,u) is the cross-entropy loss function, expressed as:

[0124] L cls (p, u) = -logp u

[0125] L loc (t u (v) is the Smooth L1 loss function, expressed as:

[0126]

[0127] Where: L GSLMask The expression for the two-dimensional cross-entropy loss function optimized using adaptive two-dimensional Gaussian label smoothing is:

[0128]

[0129] Where: p i,j N represents the predicted probability of the feature points output by the GSLMask head network; N is the size of the feature map output by the GSLMask head network. W(i,j) is the window function for adaptive two-dimensional Gaussian label smoothing, and its expression is:

[0130]

[0131] Among them: the parameters involved in the coarse label are as follows Figure 4 As shown, θ is the included angle calculated based on the coordinates of the four corner points of the coarse label, specifically the angle between the line connecting the midpoints of the longer opposite sides of the quadrilateral and the x-axis; x, y are the pixel coordinates of the feature map output by the GSLMask head network.

[0132]

[0133]

[0134] Where: d{·} represents the distance between two points; D is a hyperparameter, defaulted to 4; m * ,n * These are the x-coordinates and y-coordinates of the midpoints of each side, calculated using the four corner points.

[0135] like Figure 5 As shown, based on the above formula, the adaptive two-dimensional Gaussian label smoothing window function W(x,y) can be calculated for the coarse mask label. Figure 6 This is a schematic diagram of the window function W(x,y) in three-dimensional space. Based on this window function W(x,y) and the aforementioned formula, the mask supervision loss L for adaptive high-dimensional Gaussian label smoothing can be calculated. GSLMask , specifically Figure 7 As shown.

[0136] like Figure 8 As shown, after 100 iterations of training, the network can detect targets of different scales, different rotation angles, and dense areas in remote sensing images with excellent results, especially when the angle is close to the horizontal boundary, it still has relatively stable detection performance.

Claims

1. An arbitrary angle object detection method based on coarse mask smoothing label supervision, characterized in that it comprises the following steps: 1.1 Obtaining a data set and preprocessing, comprising the following steps: 1.1.1 According to the official label standard of the DOTA dataset, that is, each target contains a class, the horizontal coordinates 1, the vertical coordinates 1, the horizontal coordinates 2, the vertical coordinates 2, the horizontal coordinates 3, the vertical coordinates 3, the horizontal coordinates 4, and the vertical coordinates 4, the target detection box label of the visible light remote sensing image dataset to be processed is labeled; the visible light remote sensing image dataset is divided into a training set, a validation set, and a test set according to a ratio of 7:2:1; if there is a limited training data condition, sampling can be performed in the DOTA official dataset by means of random sampling, and the visible light remote sensing image dataset is filled to the size of the DOTA dataset; 1.1.2 Perform enhanced preprocessing on the visible light remote sensing image dataset with bounding box labels obtained in step 1.1.1, including random flipping, random angle rotation, random cropping, Gaussian blur, Gaussian noise, contrast enhancement, color jittering, and brightness enhancement, to enhance the feature diversity and scale of the dataset; wherein, The enhancement scheme related to the image coordinate position transformation needs to transform the target detection box label by the same bias; 1.2 Constructing a GLSMask-RCNN network The parameter settings of the GLSMask-RCNN network include the following steps: 1.2.1 Constructing a Backbone backbone feature extraction network: the Backbone backbone network is used for feature extraction of the input image, and the overall structure of this part is ResNet101, including the following steps: 1.2.1.1 ResNet101 is a deep convolutional neural network, which introduces a residual structure to form a residual network. In the process of forward and backward propagation of the model, there is a jump connection information transmission path, which can effectively avoid the phenomenon of gradient disappearance and gradient explosion in deep neural networks, so that the depth of the model can be deepened for efficient feature extraction; 1.2.1.2 The most basic structure of the residual network is the residual block, which is divided into two branches, including the main path and the residual edge; the residual network can process the input RGB image by stacking the residual block structure, and finally obtain deep features with a channel number of 2048 and a down-sampling rate of 32; 1.2.1.3 The main path includes three convolution operations in the order of a convolution layer with a convolution kernel size of 1x1, a batch normalization layer, a ReLU activation function, a convolution layer with a convolution kernel size of 3x3, a batch normalization layer, a ReLU activation function layer, a convolution layer with a convolution kernel size of 1x1, a batch normalization layer, and a ReLU activation function layer; 1.2.1.4 The residual edge is a jump connection addition structure between the input and output of the main path. The residual network has undergone 5 times of down-sampling, which is completed by adjusting the step length of the first convolution layer of the residual block to 2 and adding a convolution layer with a step length of 2 to the residual edge; 1.2.2 Constructing a Neck feature pyramid feature fusion structure: using a feature pyramid FPN for feature fusion, including the following steps: 1.2.2.1 FPN generally performs up-sampling on the intermediate features of each layer generated during the feature extraction of ResNet101, and fuses the corresponding features of the previous layer to effectively fuse the features of different levels of the input image; 1.2.2.2 FPN takes the features of the second to fifth levels in ResNet101 as input; in the processing of each level, it is divided into two operation stages: upsampling and fusion. The upsampling operation first performs a convolution operation with a convolution kernel size of 1x1 on the deep feature map to reduce the number of channels of the feature map, and then enlarges the size of the feature map through the bilinear interpolation algorithm. The fusion operation first simply fuses the feature map obtained by upsampling and the feature map of the previous level in a pixel-by-pixel addition manner, and then further refines the fusion result using a convolution operation with a convolution kernel size of 3x3 to further improve the fusion of deep and shallow layer features; 1.2.2.3 The backbone network feature map of the fifth level directly obtains the FPN feature map of the fourth level through the convolution layer with a convolution kernel of 1x1 and the convolution layer with a convolution kernel of 3x3 in FPN; the FPN feature map of the fifth level is obtained by downsampling the worst pooling from the FPN feature map of the fourth level; 1.2.2.4 The output of the FPN network includes five levels in total, each level has a different feature map size, and the number of channels is 256; 1.2.3 Region Proposal Network (RPN): inherits the region proposal network in the Mask-RCNN model, which generates target region of interest position suggestions based on the input feature map, including the following steps: 1.2.3.1 The input feature of the region proposal network is the output feature of each level of FPN; in structure, RPN is composed of a convolution layer with a convolution kernel size of 3x3, and two convolution layers with a convolution kernel size of 1x1 in parallel; 1.2.3.2 The channel numbers of the two outputs of RPN are 18 and 36 respectively, which means that for each input feature pixel position, 9 anchor boxes with different aspect ratios and sizes are matched, and for each anchor box, 2 prediction scores representing the foreground and background probabilities are output, as well as 4 bias parameters representing the adjustment of the anchor box to the suggestion box: x, y, h, w; 1.2.3.3 Select 2000 suggestion boxes according to the foreground score and perform non-maximum suppression processing to obtain the final suggestion box parameters; 1.2.3.4 According to the suggestion box parameters, the output features of each level of FPN are intercepted, and the region of interest position suggestions for the target are obtained; 1.2.4 ROIAlign: uses the bilinear interpolation algorithm to downsample the region of interest features of different sizes generated by RPN to obtain region of interest features with uniform spatial size of 7x7 and 14x14; 1.2.5 BBOX Head Network: BBOX head network outputs the prediction of target class and detection box parameters according to the input region of interest features; this part includes the following steps: 1.2.5.1 The input feature of the BBOX head network is the region of interest feature with a size of 7x7; 1.2.5.2 The sequential structure of the BBOX header network is a convolutional layer with a convolution kernel size of 3x3, a convolutional layer with a convolution kernel size of 1x1, and a parallel classifier and regressor; the classifier consists of a global average pooling layer and a BBOX category fully connected layer; the regressor consists of a global average pooling layer and a BBOX parameter regression fully connected layer; 1.2.5.3 The number of neurons of the BBOX category fully connected layer is equal to the number of categories, and the output category prediction vector is successively processed by a softmax activation to obtain normalized prediction probabilities for each category; 1.2.5.4 The number of neurons of the BBOX parameter regression fully connected layer is equal to the number of categories multiplied by 4, which is used to gradually refine the extraction of the region of interest; 1.2.6 GLSMask header network: The GLSMask header network outputs a pixel-level segmentation mask prediction based on the input region of interest feature, including the following steps: 1.2.6.1 The input feature of the GLSMask header network is a region of interest feature with a size of 14x14; 1.2.6.2 The sequential structure of the GLSMask header network is a convolutional layer with a convolution kernel size of 3x3, a transposed convolutional layer with a convolution kernel size of 2x2, and a convolutional layer with a convolution kernel size of 1x1; 1.2.6.3 The final output size of the GLSMask header network is 28x28xcategory number, i.e., the number of channels is equal to the number of categories, and a segmentation mask is predicted for each category corresponding to each target; 1.2.7 Header stacking optimization structure: The parallel header structure composed of the BBOX header network and the GLSMask header network is stacked three times; for the first stage of the header structure, the input is the region of interest feature obtained by the RPN based on the proposal box; for the second and third stages of the header structure, the region of interest feature is obtained based on the detection box output by the previous layer of the header structure; 1.2.8 Mask-based arbitrary angle detection box generation: The three-layer stacked header structure obtains the optimal target mask and category prediction in the final layer; first, the 28x28 final mask prediction feature is upsampled using bilinear interpolation to obtain a pixel-by-pixel segmentation mask prediction probability map with a spatial size consistent with the detection box size output by the third layer of the header; the probability map is binarized based on a predetermined threshold to obtain the final target segmentation mask; the segmentation mask is denoised and continuous, and the corresponding minimum bounding rectangle is generated; the minimum bounding rectangle is the arbitrary angle detection box of the target; 1.3 Model training: 1.3.1 Configure relevant data iteration parameters for model training, including batchsize=8, max_epoch=100, num_workers=4; 1.3.2 Use the Adam optimizer with an initial learning rate of 0.002, a momentum of 0.5, a smoothing parameter β1=0.9, β2=0.999, and a weight decay of 0.0001; use a linear increase strategy for the learning rate in the warm-up stage and a StepLR strategy to adjust the learning rate in the training stage; 1.3.2 GSL label segmentation coarse label optimization: In the data set of the arbitrary angle target detection task, the label type is the four corner point coordinates of the quadrilateral detection frame; in this method, for each target, the segmentation mask information is generated based on the four corner points; however, for most targets, such as ships, ports, and aircraft, this way of generating the mask covers the texture information of the background at the four corners and edges, which belongs to the coarse label; this method provides a long-width adaptive two-dimensional Gaussian label smoothing technique to reduce and smooth the four corners of the 0 / 1 segmentation mask label, so that the processed segmentation mask label discards useless information and retains information that is more conducive to the semantic and directional representation of the target itself; 1.4 Loss function The loss function of the model is divided into two parts, including the RPN multi-task loss function and the multi-task loss function of each level head network output, which are described in detail as follows: 1.4.1 The expression of the RPN multi-task loss function is: wherein: p i pi represents the probability of the ith anchor being the true label; ti represents the true class label of the ith anchor, which is 1 when it is a positive sample and 0 when it is a negative sample; i pi represents the regression parameter of the bounding box of the ith anchor; ti represents the regression parameter of the true bounding box corresponding to the ith anchor; cls N represents the number of all samples in a mini-batch, i.e. the number of positive and negative samples in the final sampling, which is 256; reg N represents the number of anchor positions, which is about 2400; is a binary cross-entropy loss, and its expression is: smooth L1 loss function, expressed as: 1.4.2 The expression of the multi-task loss function of each level head network output is: wherein: p is the softmax probability distribution predicted by the classifier p = (p0,..., p k ); u is the corresponding target real class label; t u regression parameters for the corresponding class u predicted by the corresponding bounding box regressor v are the regression parameters for the bounding box of the corresponding real target (v x ,v y ,v w ,v h ); L cls (p, u) is a cross-entropy loss function expressed as: L cls (p, u) = -log p u L loc (t u v) is a Smooth L1 loss function, expressed as: where: L GSLMask is the two-dimensional cross-entropy loss function optimized using adaptive two-dimensional Gaussian label smoothing, expressed as: where: p i,j is the predicted probability of the feature point of the GSLMask head network output; N is the size of the feature map output by the GSLMask head network; W(i,j) is a window function of adaptive two-dimensional Gaussian label smoothing, and the expression is: Where: θ is the included angle calculated according to the four corner point coordinates of the coarse label, specifically the included angle between the midpoint connecting line of the longer side of the quadrilateral and the x-axis; x, y are the pixel point position coordinates of the feature map output by the GSLMask head network. wherein: d{·} represents the distance between two points; D is a hyperparameter, preset as 4; m * ,n * are the horizontal and vertical coordinates of the midpoint of each side calculated through the four corner points.

Images