A posteriori instance information guided anchor-free remote sensing rotating target detection method

Abstract

The present application relates to a kind of posteriori instance information guiding anchorless frame remote sensing rotating target detection method, belong to remote sensing rotating target detection technical field.First, the different resolution feature layers output by skeleton network are carried out multi-scale semantic feature depth fusion using multi-scale adjacent layer feature fusion network.Second, pixel-level instance prediction network is designed after multi-scale adjacent layer feature fusion network, for obtaining the shape and angle information of remote sensing rotating target.Then, pixel-level instance prediction result with target shape and angle information is injected into the output feature of multi-scale adjacent layer feature fusion network, finally, the remote sensing rotating target in image is located using anchorless frame detection algorithm based on center point.The present application significantly improves the detection effect of anchorless frame detection algorithm on any angle target in optical remote sensing scene, not only improves target recall rate, but also greatly reduces false alarm rate, and has good practical application value.

Description

TECHNICAL FIELD

[0001] The present application relates to a remote sensing rotating target detection method, in particular to a posteriori instance information guided anchor-free remote sensing rotating target detection method, and belongs to the technical field of remote sensing rotating target detection. BACKGROUND

[0002] With the continuous improvement of the resolution of optical remote sensing images, it has been widely used in various fields such as agricultural production, environmental monitoring and military reconnaissance. Target detection technology, as an important means of extracting remote sensing image information, has become a research hotspot in the field of remote sensing. In recent years, the rapid development of deep learning has helped machine vision to open up a new road, and the target detection algorithm based on convolutional neural network has changed with each passing day, which has stronger learning ability and generalization ability than traditional target detection algorithms. However, in the face of complex background and target distribution in optical remote sensing scene, how to effectively use convolutional neural network features to interpret the targets with arbitrary angle distribution in optical remote sensing scene is an important research direction at present.

[0003] At present, most of the remote sensing rotating target detectors are based on anchor box design. This kind of method can generate candidate regions by manually setting several rotating prior anchor boxes, and then determine the target of the candidate region through convolutional network. If it is preliminarily determined as a target, the prior box will be regressed to a more accurate position through further network learning. However, this kind of method has problems such as low calculation efficiency, manual adjustment of hyperparameters, etc. When the preset anchor box is unreasonable, it will seriously affect the detection performance, and the algorithm has poor universality. Therefore, in recent years, the anchor-free target detection technology based on key points has been gradually widely used in the field of remote sensing due to its simple and efficient network framework. However, due to the lack of anchor box information with angle prior, the learning ability of anchor-free detection method for rotating targets with arbitrary angles in remote sensing scene is weaker than that of anchor-based detection method. In the case of only angle supervision information, it is difficult to make the anchor-free rotating target detector have good angle prediction ability through implicit feature optimization, therefore, the performance upper limit of the anchor-free rotating target detector is always lower than that of the anchor-based detector. In view of this problem, a target angle guiding method for anchor-free algorithm should be explored to help the anchor-free detector better perceive the angle and target shape information, so as to improve the detection performance of remote sensing rotating targets. SUMMARY

[0004] The purpose of the present application is to solve the problem of low detection accuracy of anchor-free detection method for arbitrary angle remote sensing rotating targets in optical remote sensing scene, and to propose a posteriori instance information guided anchor-free remote sensing rotating target detection method.

[0005] The method of the present application is realized by the following technical scheme.

[0006] A posteriori instance information guided anchor-free remote sensing rotating target detection method, comprising the following steps:

[0007] Step 1: extract the remote sensing image features through the skeleton network, and then use the multi-scale adjacent layer feature fusion network to perform multi-scale semantic feature depth fusion on the remote sensing image features to obtain the fusion features;

[0008] Step 2: input the fusion features obtained in step 1 into the pixel-level instance prediction network to output the pixel-level instance prediction results with target angle and shape information;

[0009] Step 3: inject the pixel-level instance prediction results obtained in step 2 into the fusion features obtained in step 1 to generate posterior information injection features;

[0010] Step 4: input the posterior information injection features generated in step 3 into the center point prediction network, the width-height prediction network and the angle prediction network in parallel to obtain the center point prediction value, the width-height prediction value and the angle prediction value of the target.

[0011] Step 5: use the loss function to continuously correct the pixel-level instance prediction results in step 2, the center point prediction value, the width-height prediction value and the angle prediction value in step 4 to finally determine the bounding box information of the target and realize the detection of the remote sensing rotating target.

[0012] In step 1, the remote sensing image features extracted by the skeleton network are a multi-scale feature group, which includes a total of 4 features arranged in descending order of resolution, which are 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the original remote sensing image resolution.

[0013] In step 1, the method for obtaining the fusion features by performing multi-scale semantic feature depth fusion is:

[0014] Step 1.1: unify the channel dimension of all features in the multi-scale feature group, and unify the channel dimension of the four feature layers to 256 by using 1x1 convolution operation with a step of 1, and the processing result is denoted as F1, F2, F3, F4, arranged in descending order of resolution;

[0015] Step 1.2: send the processing results F1, F2, F3, F4 to the multi-scale adjacent layer feature fusion network for semantic feature enhancement between adjacent scale layers to complete multi-scale semantic feature depth fusion.

[0016] Among them, the specific method for performing semantic feature enhancement between adjacent scale layers is:

[0017] The processed results F1, F2, F3, and F4 are divided into three groups: the first group of features {F1, F2}, the second group of features {F1, F2, F3}, and the third group of features {F2, F3, F4}. Then, feature fusion is performed between adjacent layers for each group of features, specifically as follows:

[0018] For the first set of features, the resolution of F1 is used as the baseline resolution. A 3×3 convolution operation with a stride of 1 and padding of 1 is performed on F1. A 3×3 deformable convolution with a stride of 1 and padding of 1 is performed on F2, followed by bilinear interpolation with an upsampling factor of 2. Finally, the processing results of F1 and F2 are added together to obtain the features.

[0019] For the second set of features, F2 is used as the baseline resolution. A 3×3 convolution with stride 2 and padding 1 is performed on F1, a 3×3 convolution with stride 1 and padding 1 is performed on F2, and a deformable 3×3 convolution with stride 1 and padding 1 is performed on F3. Then, a bilinear interpolation operation with an upsampling factor of 2 is performed. Finally, the processing results of F1, F2, and F3 are summed to obtain the feature.

[0020] For the third set of features, F3 is used as the baseline resolution. F2 is subjected to a 3×3 convolution with a stride of 2 and padding of 1, F3 is subjected to a 3×3 convolution with a stride of 1 and padding of 1, and F4 is subjected to a 3×3 deformable convolution with a stride of 1 and padding of 1, followed by bilinear interpolation with an upsampling factor of 2. The results of processing F2, F3, and F4 are then summed to obtain the feature.

[0021] Next, the three features obtained and The group was regrouped into a fourth group based on its characteristics. Fifth group of features

[0022] For the fourth set of features, The resolution is used as the baseline resolution for... Perform a 3×3 convolution operation with a stride of 1 and padding of 1. Perform a 3×3 deformable convolution with a stride of 1 and padding of 1, followed by bilinear interpolation with an upsampling factor of 2, and finally... and The processing results are added together to obtain the features.

[0023] For the fifth set of features, As a baseline resolution, for Perform a 3×3 convolution with a stride of 2 and padding of 1. performing a 3x3 deformable convolution operation with a stride of 1 and padding of 1 and then performing a bilinear interpolation operation with an up-sampling factor of 2, and then adding the processing results of and , to obtain a feature F

[0024] Finally, the two obtained features F and are combined to obtain a sixth group of features F

[0025] For the sixth group of features, the resolution of F is taken as a reference resolution, a 3x3 convolution operation with a stride of 1 and padding of 1 is performed on F , a 3x3 deformable convolution operation with a stride of 1 and padding of 1 is performed on F , a bilinear interpolation operation with an up-sampling factor of 2 is performed, and finally the processing results of F and F are added to obtain a feature F out .

[0026] In the step 2, the method for outputting the pixel-level instance prediction result with the target angle and shape information is as follows:

[0027] First, the feature map F out is input into a 3x3 convolution layer with a stride of 1, padding of 1, and 64 output channels, and then the output result of the 3x3 convolution layer is input again into a 1x1 convolution layer with a stride of 1, padding of 0, and the number of output channels being the number of categories, and finally a pixel-level instance prediction result P mask is output, wherein the dimension of P mask is CxHxW, and C represents the number of categories, and HxW represents the resolution of the feature map.

[0028] In the step 3, the method for generating the posterior information injection feature map is as follows:

[0029] Step 3.1: performing a global average pooling operation on the pixel-level instance prediction result P mask obtained in the step 2 according to the category dimension, and then performing a Softmax operation on the category dimension to obtain the category weight, as shown in the following formula (1):

[0030]

[0031] In the formula, C represents the number of categories, represents the pixel-level instance prediction result of the cth category, and W c represents the weight of the cth category.​

[0032] Step 3.2: Apply the category weights W obtained in Step 3.1 c The pixel-level instance prediction result P obtained in step 2 is considered as channel weight. mask Treating them as spatial weights, the fusion features obtained in step 1 are then sequentially compared with the channel weights W. c and spatial weight P mask Multiply, specifically as shown in equation (2), and the generation of the post-hoc information injection feature map is now complete:

[0033]

[0034] Where C represents the number of categories; c = 1, 2, 3, ..., C; W represents the spatial weight of the c-th category; c F represents the channel weight of the c-th category; out F represents the fusion features obtained in step 1; e This indicates that the generated posterior information is injected into the feature map.

[0035] Predicting results based on pixel-level instances for each category For F out By weighting the spatial dimensions, the shape and angle information of the remotely sensed rotating target can be effectively injected into the feature map F. out In addition, it can also filter out background interference information in a spatially weighted manner, reducing attention to false alarm targets. Furthermore, it utilizes channel weights W... c In F out Weighting the channel dimension can reduce the weight of target categories that do not appear, thereby improving the feature description capability for target categories that have appeared.

[0036] In step 4, the method for obtaining the predicted values ​​of the target's center point, width and height, and angle is as follows:

[0037] The method for obtaining the target center point prediction value is as follows: The posterior information generated in step 3 is injected into the feature map and input into a 3×3 convolutional layer with a stride of 1, padding of 1, and output channels of 64. Then, the output of the above 3×3 convolutional layer is input into a 1×1 convolutional layer with a stride of 1, padding of 0, and output channels equal to the number of categories. Finally, the target center point prediction value is output.

[0038] The method for obtaining the target width and height prediction values ​​is as follows: The posterior information generated in step 3 is injected into the feature map and input into a 3×3 convolutional layer with a stride of 1, padding of 1, and output channels of 64. Then, the output of the above 3×3 convolutional layer is input into a 1×1 convolutional layer with a stride of 1, padding of 0, and output channels of 2. Finally, the target width and height prediction values ​​are output.

[0039] The method for obtaining the target angle prediction value is: injecting the posterior information generated in step 3 into the feature map input into a 3*3 convolution layer with a step of 1, padding of 1 and 64 output channels, then inputting the output result of the 3*3 convolution layer into a 1*1 convolution layer with a step of 1, padding of 0 and 1 output channel again, and finally outputting the target angle prediction value;

[0040] The specific method for continuously correcting the pixel-level instance prediction result of step 2, the center point prediction value, the width-height prediction value and the angle prediction value of step 4 by using the loss function in step 5 is:

[0041] Step 5.1: calculating the center point, width-height, angle and pixel-level instance true value of the target according to the true boundary box of the remote sensing rotating target.

[0042] Step 5.2: taking the center point, width-height, angle and pixel-level instance true value of the target calculated in step 5.1 as the supervision signal, then calculating the loss value of the target center point prediction value and the target center point true value by using the center point loss function, calculating the loss value of the pixel-level instance prediction result and the pixel-level instance true value by using the pixel-level instance loss function, calculating the loss value of the target width-height prediction value and the target width-height true value by using the width-height loss function, and calculating the loss value of the target angle prediction value and the target angle true value by using the angle loss function. Finally, adding the loss values together, and then updating the parameters of the network by using the back propagation algorithm to correct the prediction result.

[0043] Beneficial effects

[0044] Compared with the prior art, the method has the following advantages:

[0045] (1) The method can effectively solve the problem of low detection accuracy of the anchor-free detection method in the optical remote sensing scene for the remote sensing rotating target with any angle, and can also greatly reduce the false alarm rate, and has good practical application value. The method of the application first uses a multi-scale adjacent layer feature fusion network to fuse the multi-scale semantic feature information of the remote sensing image with the target with any angle, and provides guarantee for generating posterior instance information. Then, a pixel-level instance prediction network is constructed after the multi-scale adjacent layer feature fusion network to obtain the angle and shape information of the remote sensing rotating target from the target true boundary box in a supervised learning manner. Secondly, a posterior information injection module is used to inject the pixel-level instance prediction result with the target shape and angle information into the output feature of the multi-scale adjacent layer feature fusion network, which is used to guide the network feature to learn the angle and shape information of the target. Finally, a center point-based anchor-free detection algorithm is used to detect the remote sensing rotating target in the image.

[0046] (2) The present application relates to a kind of posteriori instance information guiding anchorless frame remote sensing rotating target detection method, belong to remote sensing rotating target detection technical field.First, the feature layer of different resolution that the skeleton network exports is carried out multi-scale semantic feature depth fusion using multi-scale adjacent layer feature fusion network.Second, pixel-level instance prediction network is designed after multi-scale adjacent layer feature fusion network, for obtaining the shape and angle information of remote sensing rotating target.Then, design posteriori information injection module, pixel-level instance prediction result with target shape and angle information is injected into the output feature of multi-scale adjacent layer feature fusion network, generates posteriori information injection feature.Finally, on the basis of generated posteriori information injection feature, the remote sensing rotating target in image is located using the anchorless frame detection algorithm based on center point.This method significantly improves the detection effect of anchorless frame detection algorithm to arbitrary angle target in optical remote sensing scene, not only improves target recall rate, also greatly reduces false alarm rate, with good practical application value. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 It is the overall flow chart of the present method.

[0048] Figure 2 It is the skeleton network feature extraction schematic diagram in the present method.

[0049] Figure 3 It is the multi-scale adjacent layer feature fusion module schematic diagram in the present method.

[0050] Figure 4 It is the network overall framework structure diagram proposed in the present method.

[0051] Figure 5 It is the posteriori instance information injection module schematic diagram proposed in the present method, Figure 5 a indicates the preparation schematic diagram of the supervision signal corresponding to pixel-level instance prediction head network, Figure 5 b is the network structure of pixel-level instance prediction network, Figure 5 c is the structure diagram of posteriori instance information injection module. DETAILED DESCRIPTION

[0052] The method of the present application will be further described in detail below in conjunction with the drawings and examples.

[0053] Examples

[0054] A kind of posteriori instance information guiding anchorless frame remote sensing rotating target detection method, overall flow as Figure 1 Shown, specifically includes the following steps:

[0055] Step 1: input remote sensing images, extract remote sensing image features through a skeleton network, and output fused features F after multi-scale semantic feature depth fusion of remote sensing image features by using a multi-scale adjacent layer feature fusion network out .

[0056] The application gives a specific implementation method, including the following steps:

[0057] Step 1.1: scan the original image, slice the input high-resolution remote sensing image to generate a plurality of 512*512 resolution slices; use a skeleton network to extract remote sensing image features of the image slices to obtain a multi-scale feature group with resolution sizes of 128*128, 64*64, 32*32 and 16*16 in turn.

[0058] In the embodiment, the mentioned skeleton network is selected as a ResNet-101 network, as shown in Figure 2 , the network structure has four Block modules, and the outputs of the Blocks constitute a multi-scale feature group.

[0059] Step 1.2: in order to ensure the use of subsequent modules, first, the multi-scale feature group is subjected to channel dimension unification processing, and four feature maps are unified to 256 channels by using a 1*1 convolution layer with a step of 1 and padding of 0, and the output result of this step is recorded as F1, F2, F3 and F4, which are arranged in descending order of resolution.

[0060] Step 1.3: the processing results F1, F2, F3 and F4 are sent to a multi-scale adjacent layer feature fusion network for semantic feature enhancement between adjacent scales, as shown in Figure 3 .

[0061] Since a pixel-level instance prediction task is used in the subsequent prediction task, and considering the multi-scale distribution characteristics of remote sensing targets, higher requirements are put forward for the semantic feature description ability of the network. Therefore, the application designs adjacent layer fusion for multi-scale feature fusion, which not only avoids the problem of semantic information misalignment caused by direct fusion of shallow features and deep features, but also can well guarantee the interaction of various scale information and the multi-scale semantic description ability, and provides feature description guarantee for the subsequent pixel-level instance prediction.

[0062] The following examples are given.

[0063] The processing results F1, F2, F3 and F4 are divided into three groups, namely a first group of features {F1, F2}, a second group of features {F1, F2, F3} and a third group of features {F2, F3, F4}, and then adjacent layer feature fusion is performed on each group of features, as shown in Figure 4 , specifically:

[0064] For the first group of features, taking the resolution of F1 as the reference resolution, a 3x3 convolution operation with a step size of 1 and a padding of 1 is performed on F1, a 3x3 deformable convolution with a step size of 1 and a padding of 1 is performed on F2, and then a bilinear interpolation operation with a upsampling factor of 2 is performed, and finally the processing results of F1 and F2 are added to obtain the feature

[0065] For the second group of features, taking F2 as the reference resolution, a 3x3 convolution with a step size of 2 and a padding of 1 is performed on F1, a 3x3 convolution with a step size of 1 and a padding of 1 is performed on F2, and a 3x3 deformable convolution with a step size of 1 and a padding of 1 is performed on F3, and then a bilinear interpolation operation with a upsampling factor of 2 is performed, and then the processing results of F1, F2 and F3 are added to obtain the feature

[0066] For the third group of features, taking F3 as the reference resolution, a 3x3 convolution with a step size of 2 and a padding of 1 is performed on F2, a 3x3 convolution with a step size of 1 and a padding of 1 is performed on F3, and a 3x3 deformable convolution with a step size of 1 and a padding of 1 is performed on F4, and then a bilinear interpolation operation with a upsampling factor of 2 is performed, and then the processing results of F2, F3 and F4 are added to obtain the feature

[0067] Next, the three features obtained and are grouped again into a fourth group of features a fifth group of features

[0068] For the fourth group of features, taking the resolution of as the reference resolution, a 3x3 convolution operation with a step size of 1 and a padding of 1 is performed on , a 3x3 deformable convolution with a step size of 1 and a padding of 1 is performed on , and then a bilinear interpolation operation with a upsampling factor of 2 is performed, and finally the processing results of and are added to obtain the feature

[0069] For the fifth group of features, taking as the reference resolution, a 3x3 convolution with a step size of 2 and a padding of 1 is performed on , a 3x3 convolution with a step size of 1 and a padding of 1 is performed on , a 3x3 deformable convolution with a step size of 1 and a padding of 1 is performed on , and then a bilinear interpolation operation with a upsampling factor of 2 is performed, and then the processing results of and The processing results are added together to obtain the features.

[0070] Finally, the two features obtained and By combining them, we obtain the sixth set of features.

[0071] For the sixth set of features, The resolution is used as the baseline resolution for... Perform a 3×3 convolution operation with a stride of 1 and padding of 1. Perform a 3×3 deformable convolution with a stride of 1 and padding of 1, followed by bilinear interpolation with an upsampling factor of 2, and finally... and The processing results are added together to obtain feature F. out .

[0072] Step 2: Combine the fused features F obtained in Step 1 out The input is fed into a pixel-level instance prediction network, and the output is a pixel-level instance prediction result P containing target angle and shape information. mask .

[0073] This invention provides a specific implementation method:

[0074] First, the feature map F out The input is passed to a 3×3 convolutional layer with a stride of 1, padding of 1, and 64 output channels. The output of this 3×3 convolutional layer is then passed to a 1×1 convolutional layer with a stride of 1, padding of 0, and output channels equal to the number of classes. The final output is the pixel-level instance prediction result P. mask P mask The dimensions are C×H×W, where C represents the number of categories, and H and W represent the width and height of the feature map.

[0075] Step 3: Using the posterior instance information injection module, inject the pixel-level instance prediction results obtained in Step 2 into the fusion feature F obtained in Step 1. out Above, such as Figure 5 As shown in (b) and (c), posterior information is injected into the feature map.

[0076] This invention provides a specific implementation method, including the following steps:

[0077] Step 3.1: Calculate the pixel-level instance prediction result P obtained in Step 2. mask Global average pooling is performed along the category dimension, and then softmax is performed along the category dimension to obtain the category weights, as shown in equation (3):

[0078]

[0079] In the formula, C represents the number of categories. W represents the pixel-level instance prediction result for the c-th category. c This represents the weight of the c-th category;

[0080] Step 3.2: Apply the category weights W obtained in Step 3.1 c The pixel-level instance prediction result P obtained in step 2 is considered as channel weight. mask The spatial weights are then multiplied sequentially by the channel weights and spatial weights obtained in step 1, as shown in equation (4). This completes the generation of the post-hoc information injection feature map.

[0081]

[0082] Where C represents the number of categories; c = 1, 2, 3, ..., C; W represents the spatial weight of the c-th category; c F represents the channel weight of the c-th category; out F represents the fusion features obtained in step 1; e This indicates that the generated posterior information is injected into the feature map.

[0083] Predicting results based on pixel-level instances for each category For F out By weighting the spatial dimensions, the shape and angle information of the remotely sensed rotating target can be effectively injected into the feature map F. out In addition, it can also filter out background interference information in a spatially weighted manner, reducing attention to false alarm targets. Furthermore, it utilizes channel weights W... c In feature map F out Weighting the channel dimension can reduce the weight of target categories that do not appear, thereby improving the feature description capability for target categories that have appeared.

[0084] Step 4: Inject the posterior information generated in Step 3 into the feature map and input it in parallel into the center point prediction network, width and height prediction network, and angle prediction network to obtain the center point prediction value, width and height prediction value, and angle prediction value of the target.

[0085] This invention provides a specific implementation method:

[0086] The method for obtaining the target center point prediction value is as follows: The posterior information generated in step 3 is injected into the feature map and input into a 3×3 convolutional layer with a stride of 1, padding of 1, and output channels of 64. Then, the output of the above 3×3 convolutional layer is input into a 1×1 convolutional layer with a stride of 1, padding of 0, and output channels equal to the number of categories. Finally, the target center point prediction value is output.

[0087] The method for obtaining the target width-height prediction value is that: the posterior information generated in step 3 is injected into a feature map to a 3*3 convolution layer with a step of 1, padding of 1 and 64 output channels, then the output result of the 3*3 convolution layer is input again to a 1*1 convolution layer with a step of 1, padding of 0 and 2 output channels, and finally the target width-height prediction value is output.

[0088] The method for obtaining the target angle prediction value is that: the posterior information generated in step 3 is injected into a feature map to a 3*3 convolution layer with a step of 1, padding of 1 and 64 output channels, then the output result of the 3*3 convolution layer is input again to a 1*1 convolution layer with a step of 1, padding of 0 and 1 output channel, and finally the target angle prediction value is output.

[0089] Step 5: the pixel-level instance prediction result in step 2, the center point prediction value, the width-height prediction value and the angle prediction value are constantly corrected by using a loss function, and finally the bounding box information of the target is determined to realize the detection of the remote sensing rotating target.

[0090] The application provides a specific implementation method, which comprises the following steps:

[0091] Step 5.1: the real values of the center point, width-height, angle and pixel-level instance of the remote sensing rotating target are calculated according to the real bounding box of the remote sensing rotating target.

[0092] The preparation of the supervision signal of the pixel-level instance prediction network proposed in the application is described as follows, as shown in Figure 5 (a).

[0093] According to the rotating bounding box provided by the remote sensing rotating target detection data set, first, all the rotating bounding boxes are divided into C groups according to the category number C of the data set, then a feature map with the same resolution as the remote sensing image is defined for each group, and the values on the feature map are all 0 by default; secondly, according to the rotating bounding box in each group, the values of the corresponding region inside the bounding box on the feature map are set to 1, and the values outside the bounding box remain unchanged. Finally, the resolution size of the pixel-level instance prediction result obtained in step 2 is adjusted to ensure that the resolution size of the pixel-level instance prediction result and the pixel-level instance real value is consistent.

[0094] Step 5.2: The center point, width and height, angle and pixel-level instance of the target calculated in step 5.1 are taken as the supervision signal, and then the loss value of the predicted center point of the target and the real value of the target center point is calculated by using the center point loss function; the loss value of the pixel-level instance prediction result and the real value of the pixel-level instance is calculated by using the pixel-level instance loss function; the loss value of the target width and height prediction value and the real value of the target width and height is calculated by using the width and height loss function; the loss value of the target angle prediction value and the real value of the target angle is calculated by using the angle loss function. Finally, the loss values are added together, and then the parameters of the network are updated by using the back propagation algorithm, so as to correct the prediction result.

[0095] The loss function is illustrated as follows.

[0096] The loss function of the center point is shown in the following formula (5):

[0097]

[0098] Wherein, C represents the number of categories, H and W represent the width and height of the output feature map; The predicted value at the (x, y) position on the channel of the cth category; Y xyc The real value at the (x, y) position on the channel of the cth category; N represents the number of positive samples. α and β represent weight adjustment factors, which are respectively set to 2 and 4. L ct The loss value calculated by the center point loss function.

[0099] The loss function of the pixel-level instance prediction adopts a modified cross-entropy loss function, as shown in the following formula (6):

[0100]

[0101] Wherein, C represents the number of categories, H and W represent the width and height of the output feature map; Z represents the number of positive samples in the pixel-level instance real value map; The predicted value at the (x, y) position on the channel of the cth category; M xyc The real value at the (x, y) position on the channel of the cth category; α represents a weight adjustment factor, which is set to 2; L m The loss value between the prediction value of the pixel-level instance prediction network and the supervision signal.

[0102] In addition, the loss functions of the width and height and the angle all adopt the Smooth-L1 norm loss function, which is specifically shown in the following formula (7):

[0103]

[0104] Wherein, x represents the difference value between the real value and the predicted value, i.e., hp -h g , w p -w g , a p -a g wherein h, w and a represent width, height, angle, respectively; and SmoothL1(x) represents a loss value calculated.

[0105] Numerous modifications can be made to the present application without departing from the principles thereof, and such modifications are intended to be within the scope of the application.

Claims

1. A method for anchor-free bounding box remote sensing rotating target detection with posterior instance information guidance, characterized in that The method comprises the following steps: Step 1: extracting remote sensing image features through a skeleton network, and then performing multi-scale semantic feature deep fusion on the remote sensing image features by using a multi-scale adjacent layer feature fusion network to obtain fused features; Step 2: inputting the fused features obtained in step 1 into a pixel-level instance prediction network to output pixel-level instance prediction results with target angle and shape information; Step 3: injecting the pixel-level instance prediction results obtained in step 2 into the fused features obtained in step 1 to generate posterior information injection features; Step 4: inputting the posterior information injection features generated in step 3 into a center point prediction network, a width-height prediction network and an angle prediction network in parallel to obtain a center point prediction value, a width-height prediction value and an angle prediction value of the target; Step 5: correcting the pixel-level instance prediction results in step 2 and the center point prediction value, the width-height prediction value and the angle prediction value of the target obtained in step 4 by using a loss function to obtain the bounding box information of the target, and realizing the detection of the remote sensing rotating target. In step 1, the method for obtaining the fused features through multi-scale semantic feature deep fusion is: Step 1.1: Uniformize the channel dimension of all features in the multi-scale feature group, and use 1x1 convolution operation with a step size of 1 to unify the channel dimension of the four feature layers to 256, and the processing result is denoted as , arranged in descending order of resolution; Step 1.2: send the processing result into the multi-scale adjacent layer feature fusion network to enhance the semantic features between adjacent scale layers and complete the multi-scale semantic feature depth fusion.

2. The remote sensing rotating target detection method without anchor box guided by posterior instance information according to claim 1, wherein: In step 1, the remote sensing image features extracted by the skeleton network are a multi-scale feature group, which includes a total of 4 features arranged in descending order of resolution, i.e., 1 / 4, 1 / 8, 1 / 16 and 1 / 32 of the resolution of the original remote sensing image.

3. The remote sensing rotating target detection method without anchor box guided by posterior instance information according to claim 2, wherein: In step 1.2, the specific method for performing semantic feature enhancement between adjacent scale layers is: The processing result are divided into three groups, respectively, the first group of features , the second group of features , the third group of features , then the features of each group are fused with adjacent layers, specifically: For the first set of features, F A resolution of 1 is used as the baseline resolution for... F 1. Perform a 3×3 convolution operation with a stride of 1 and padding of 1. F 2. Perform a 3×3 deformable convolution with a stride of 1 and padding of 1, followed by bilinear interpolation with an upsampling factor of 2, and finally... F 1 and F The results of processing step 2 are added together to obtain the features. ; For the second group of features, the following is performed: F 2 is convolved with a 3x3 kernel with stride 2 and padding 1, and F 1 is convolved with a 3x3 kernel with stride 1 and padding 1, and F 2 is convolved with a 3x3 kernel with stride 1 and padding 1, and F 3 is convolved with a 3x3 kernel with stride 1 and padding 1, and then upsampled by a factor of 2 using bilinear interpolation, and then F 1, F 2, and F 3 are added together to obtain feature . For the third group of features, the following is performed: F 3 is convolved with a 3x3 kernel with stride 2 and padding 1, F 2 is convolved with a 3x3 kernel with stride 1 and padding 1, F 3 is convolved with a 3x3 deformable convolution kernel with stride 1 and padding 1, and then bilinearly upsampled by a factor of 2, F 4 is convolved with a 3x3 deformable convolution kernel with stride 1 and padding 1, and then bilinearly upsampled by a factor of 2, F 2, F 3 and F 4 are added together to obtain the feature .

4. The remote sensing rotating target detection method without anchor box guided by posterior instance information according to claim 3, wherein: The three features obtained , The grouping is performed again, into a fourth group of features }, a fifth group of features } For the fourth set of features, The resolution is used as the baseline resolution for... Perform a 3×3 convolution operation with a stride of 1 and padding of 1. Perform a 3×3 deformable convolution with a stride of 1 and padding of 1, followed by bilinear interpolation with an upsampling factor of 2, and finally... and The processing results are added together to obtain the features. ; For the fifth group of features, the following is performed: As the reference resolution, the following is performed: A 3x3 convolution with a stride of 2 and padding of 1 is performed on A 3x3 convolution operation with a stride of 1 and padding of 1 is performed on A 3x3 deformable convolution with a stride of 1 and padding of 1 is performed on , and The processing results of are added to obtain the feature 5. The remote sensing rotating target detection method without anchor box guided by posterior instance information according to claim 4, wherein: The two features obtained are combined to obtain a sixth set of features } For the sixth group of features, the resolution of the feature is taken as the reference resolution, a 3x3 convolution operation with a stride of 1 and padding of 1 is performed on the feature , a 3x3 deformable convolution with a stride of 1 and padding of 1 is performed on the feature , and then a bilinear interpolation operation with an up-sampling factor of 2 is performed, and finally the processing results of the features and are added to obtain the feature .

6. The remote sensing rotating target detection method without anchor box guided by posterior instance information according to claim 1, wherein: In step 2, the method for outputting the pixel-level instance prediction results with target angle and shape information is: First, the feature map is input to a 3x3 convolution layer with a step size of 1, padding of 1, and 64 output channels, and then the output of the 3x3 convolution layer is input again to a 1x1 convolution layer with a step size of 1, padding of 0, and 1x1 output channels, and finally the pixel-level instance prediction result is output , The dimension of the pixel-level instance prediction result is CxHxW, where C represents the number of categories, and HxW represents the resolution of the feature map.

7. The remote sensing rotating target detection method without anchor box guided by posterior instance information according to claim 1, wherein: In step 3, the method for generating the posterior information injection feature map is: Step 3.1: obtaining pixel-level instance prediction results of the step 2 The global average pooling operation is performed according to the category dimension, and then the Softmax operation is performed in the category dimension to obtain the category weight, as shown in the following formula (1): （1） In the formula, C represents the number of categories, represents the pixel-level instance prediction result of the first c category, represents the weight of the first c category; Step 3.2: Obtain the category weight of step 3.1 Step 2: Obtain the pixel-level instance prediction result of step 1 as a channel weight Step 1: Obtain the fusion feature of step 0 as a spatial weight, and then multiply the fusion feature obtained in step 1 with the channel weight and the spatial weight in turn, as shown in the following formula (2), to complete the generation of the posterior information injection feature map Step 1: Obtain the fusion feature of step 0 as a spatial weight, and then multiply the fusion feature obtained in step 1 with the channel weight and the spatial weight in turn, as shown in the following formula (2), to complete the generation of the posterior information injection feature map​ （2） wherein C denotes the number of categories; c = 1, 2, 3…, C; denotes the spatial weight of the c th category; denotes the channel weight of the c th category; denotes the fusion feature obtained in step 1; denotes the generated posterior information injection feature map.

8. The remote sensing rotating target detection method without anchor box guided by posterior instance information according to claim 1, wherein: In step 4, the method for obtaining the center point prediction value, the width-height prediction value and the angle prediction value of the target is: The target center point prediction value is obtained by injecting the posterior information generated in step 3 into a feature map, inputting the feature map into a 3*3 convolution layer with a step length of 1, a padding of 1 and an output channel number of 64, inputting the output of the 3*3 convolution layer into a 1*1 convolution layer with a step length of 1, a padding of 0 and an output channel number of the category number, and finally outputting the target center point prediction value; The target width and height prediction value is obtained by injecting the posterior information generated in step 3 into a feature map, inputting the feature map into a 3*3 convolution layer with a step length of 1, a padding of 1 and an output channel number of 64, inputting the output of the 3*3 convolution layer into a 1*1 convolution layer with a step length of 1, a padding of 0 and an output channel number of 2, and finally outputting the target width and height prediction value; The target angle prediction value is obtained by injecting the posterior information generated in step 3 into a feature map, inputting the feature map into a 3*3 convolution layer with a step length of 1, a padding of 1 and an output channel number of 64, inputting the output of the 3*3 convolution layer into a 1*1 convolution layer with a step length of 1, a padding of 0 and an output channel number of 1, and finally outputting the target angle prediction value.

9. The posterior instance information guided anchor-free remote sensing rotating target detection method according to claim 1, characterized in that: In step 5, the specific method for correcting the pixel-level instance prediction result in step 2, the center point prediction value, the width and height prediction value and the angle prediction value by using the loss function is as follows: Step 5.1: calculating the center point, width and height, angle and pixel-level instance true value of the target according to the true boundary box of the remote sensing rotating target; Step 5.2: taking the center point, width and height, angle and pixel-level instance true value of the target calculated in step 5.1 as a supervision signal, then calculating the loss value of the target center point prediction value and the target center point true value by using the center point loss function, calculating the loss value of the pixel-level instance prediction result and the pixel-level instance true value by using the pixel-level instance loss function, calculating the loss value of the target width and height prediction value and the target width and height true value by using the width and height loss function, calculating the loss value of the target angle prediction value and the target angle true value by using the angle loss function, and finally adding the loss values together, then updating the parameters of the network by using the back propagation algorithm to correct the prediction result.

Images