A false alarm removal method for SAR vehicle target detection

Abstract

The invention discloses a method for removing false alarms for assisting SAR vehicle target detection, relates to the technical field of radar target detection, and solves the problem that when detecting a target in the prior art, the false alarm rate is high, a low-quality detection frame is easily generated, and it is difficult to accurately detect the target; the method comprises: obtaining a SAR image and an optical remote sensing image that are registered with each other; inputting the SAR image into a trained target detection network to obtain a preliminary target detection result; inputting the optical remote sensing image into a trained target segmentation network to obtain a target segmentation result; mapping the bounding box position in the target detection result to the segmented image, calculating the sum of all non-vehicle target pixel points inside the bounding box, and removing false alarms in the preliminary target detection result according to a certain threshold. The invention enhances the feature representation capability of the prediction feature layer on the basis of the original target detection framework, realizes the function of removing false alarms in complex scenes, and improves the target detection effect of the SAR image.

Description

Technical Field

[0001] The present invention relates to the technical field of radar target detection, and in particular to a false alarm elimination method for assisting vehicle target detection. Background Art

[0002] Synthetic Aperture Radar (SAR) is a high-resolution sensor capable of all-weather, all-day Earth observation. Its imaging process is independent of external light sources, enabling it to operate in all kinds of extreme climates. It is not susceptible to interference from natural phenomena such as weather, lighting, and fog, and can, to a certain extent, identify hidden targets. As SAR imaging technology continues to mature, acquiring SAR images has become easier, and image quality and resolution have become increasingly higher, leading to the development of SAR image target detection technology.

[0003] In the early stages, target detection technology mainly relied on traditional methods. Among them, the landmark method was the Constant False Alarm Rate (CFAR) detection method proposed by Finn et al. based on Bayesian theory. This method is mainly applicable to radar signal target detection tasks under Gaussian background noise. Subsequently, many researchers carried out research on SAR image target detection. In the 1990s, Burl, Novak and other scholars from the Lincoln Laboratory in the United States proposed the famous dual-parameter CFAR detection method, which had an important impact on the subsequent development of SAR image target detection research. A large number of constant false alarm detection algorithms based on various background noise distributions have emerged, such as mean statistic CFAR, ordered statistic CFAR, maximum statistic CFAR and minimum statistic CFAR. However, although traditional target detection methods can achieve good detection performance in simple scenarios, they still have the following shortcomings and room for improvement: (1) In practical applications, traditional methods require artificially setting thresholds and a large number of hyperparameters based on image scenes, and lack adaptability and robustness. (2) The detection process of traditional methods is relatively complex and requires pixel-by-pixel calculations. This results in low efficiency for object detection in large images, and poor detection performance in complex scenes, such as areas with dense buildings and trees. Therefore, the performance and efficiency of traditional object detection methods are no longer able to meet the needs and challenges of practical applications.

[0004] With the development and maturity of deep learning, two types of target detection algorithms, anchor-based and anchor-free, have been widely used in SAR image target detection tasks, such as Faster R-CNN, SSD (Single Shot MultiBox Detector, SSD), YOLO (You only look once, YOLO), CornerNet and FCOS (Fully Convolutional One-Stage Object Detection, FCOS), which have significant performance advantages over traditional target detection methods. For example, patent CN202210269829.6 provides a dual-stream SSD SAR image target detection method based on CFAR guidance. Its main innovations include: (1) utilizing the strong scattering property of SAR image targets, fusing the original amplitude features of SAR images and CFAR indicator map features to improve feature extraction capabilities; (2) introducing the CFAR binary indicator map into the loss function to alleviate the problem of imbalance between positive and negative samples; (3) improving the post-processing algorithm NMS for target detection and proposing an AR-NMS algorithm based on area threshold, which further improves the accuracy of SAR target detection.

[0005] However, for SAR image vehicle target data, there are relatively few vehicle targets and the detection scene is relatively complex. There is a lot of natural and artificial clutter with similar scattering characteristics to the target, such as trees and buildings, resulting in a high false alarm rate in vehicle target detection tasks. Among existing deep network algorithms, candidate box-based target detection algorithms require tiling a large number of boxes in the feature map and then performing subsequent processing on these candidate boxes. This is computationally intensive and complex, and can easily produce a large number of low-quality detection boxes. For frameless target detection algorithms, the receptive field of each feature point is limited, and information about the target scene cannot be fully utilized. This can easily lead to false alarms when detecting scenes similar to the target, making it difficult to accurately detect vehicle targets. Therefore, the above methods still have room for improvement. Summary of the Invention

[0006] The present invention provides a method for removing false alarms for assisting vehicle target detection, thereby solving the problems in the prior art of high false alarm rate and easy occurrence of low-quality detection frames when detecting targets, making it difficult to accurately detect targets. The present invention combines target scene segmentation with target detection, improves the ability to remove false alarms, and enhances the feature representation capability of the prediction feature layer on the basis of the original target detection framework, increases the receptive field of the prediction feature layer and the context information contained therein, enables the network to adapt to targets of more sizes and shapes, and improves the detection effect.

[0007] The present invention provides a method for removing false alarms for assisting vehicle target detection, the method comprising:

[0008] Acquire mutually registered SAR images and optical remote sensing images;

[0009] The SAR image slice is input into the trained target detection network to obtain the classification category and bounding box regression position corresponding to the predicted feature layer; wherein, the target detection network specifically includes: a feature extraction module, a feature fusion module and a classification regression prediction module; the overall target detection framework uses patent number 202310153318.2, and the patent name is the target detection framework of the anchor-free SAR target detection method based on the supervised attention mechanism, and the feature fusion module therein is improved, using the deep feature layer to learn the weight of the context information, and performing weighted multiplication on the shallow feature layer to obtain more accurate features with richer semantic information. The feature extraction module is used to use the ResNet network to extract network features from the SAR image to obtain multiple network feature layers; the feature fusion module is used to use multiple identical feature fusion units based on attention context information to perform feature fusion on the multiple network feature layers to obtain fused features as prediction feature layers; the classification and regression prediction module is used to use the classification prediction unit and the regression prediction unit to predict the category and position of the prediction feature layer to obtain the classification category and bounding box regression position corresponding to the prediction feature layer, and finally integrate the slice detection results into the complete large image to obtain the final target detection result, that is, the target classification category score and bounding box regression position in the large image;

[0010] Inputting the optical remote sensing image slices into a trained target segmentation network to obtain segmented images of multiple slices, and stitching the multiple segmented images into a complete large image to obtain a final stitched segmented image;

[0011] Based on the bounding box regression position, the bounding box is mapped to the final stitched segmented image, the sum of all non-vehicle target pixels within the bounding box in the stitched segmented image is calculated, and the sum of the pixels is compared with a threshold. False alarms in the target detection result are removed based on the comparison result.

[0012] In a possible implementation, the feature fusion unit based on attention context information includes: an FFM_AC1 unit, an FFM_AC2 unit, and an FFM_AC3 unit;

[0013] The multiple network feature layers include: a C1 network feature layer, a C2 network feature layer, and a C3 network feature layer;

[0014] The FFM_AC1 unit is specifically used to perform a convolution operation on the C3 network feature layer with a convolution kernel size of 1×1 and a step size of 1 to obtain a size of Feature layer Where H and W represent the SAR image slice size of the input target detection network, at this time H = 800, W = 1333; the neural network is used to detect the feature layer Perform attention map extraction and obtain a size of Feature layer For the feature layer Perform four times upsampling to get a size of Feature layer P3;

[0015] The FFM_AC2 unit is specifically used to perform a convolution operation on the C2 network feature layer with a convolution kernel size of 1×1 and a step size of 1 to obtain a size of Feature layer Where H and W represent the SAR image slice size of the input target detection network, at this time H = 800, W = 1333; the neural network is used to detect the feature layer Perform attention map extraction and obtain a size of Feature layer For the feature layer Perform double upsampling to get a size of Feature layer P2;

[0016] The FFM_AC3 unit is specifically used to perform a convolution operation on the C1 network feature layer with a convolution kernel size of 1×1 and a step size of 1 to obtain a size of Feature layer Where H and W represent the SAR image slice size of the input target detection network, at this time H = 800, W = 1333; the feature layer Multiply and add the feature map P2 to obtain the feature layer P1 1 , which is expressed as The feature layer P1 1 Multiply and add the feature layer P3 to obtain the feature layer P1, which is expressed as P1=P1 1 +P1 1 ×P3.

[0017] In one possible implementation, the method for extracting the attention map specifically includes:

[0018] Obtaining the spatial attention weight and the channel attention weight from the input feature map from the spatial dimension and the channel dimension respectively, to obtain the spatial attention weight and the channel attention weight;

[0019] The channel attention weight and the spatial attention weight are multiplied with the original feature map in sequence to obtain the feature layer after the attention map is extracted.

[0020] In a possible implementation, the classification prediction unit is specifically configured to:

[0021] Performing a convolution operation on the prediction feature layer using four convolutions to obtain a prediction classification feature layer;

[0022] Performing a convolution operation on the prediction classification feature layer using convolution to obtain a prediction classification score layer;

[0023] The predicted classification score layer is normalized using Sigmoid normalization to obtain a classification feature map, that is, the classification category.

[0024] In a possible implementation, the regression prediction unit is specifically configured to:

[0025] Performing a convolution operation on the predicted feature layer using four convolutions to obtain a feature layer for predicting the regression position of the bounding box;

[0026] A convolution operation is performed on the feature layer of the predicted bounding box regression position using convolution to obtain the bounding box regression position.

[0027] In a possible implementation, the target segmentation network includes: an encoding module and a decoding module;

[0028] The encoding module specifically includes a feature extraction module and a dilated spatial pyramid pooling unit. The feature extraction module is used to extract features from the optical remote sensing image to obtain shallow features and deep features. The dilated spatial pyramid pooling unit is used to use dilated convolutions of different sizes to obtain feature maps of different receptive fields for the deep features, and to splice the feature maps of different receptive fields to obtain convolution deep features.

[0029] The decoding module includes two convolution modules, two upsampling layers, a splicing layer and a convolution layer. The decoding module is used to perform channel splicing and fusion of the shallow features and the convolution deep features to obtain the slice segmentation result.

[0030] In a possible implementation, the atrous spatial pyramid pooling unit specifically includes: five parallel branches, a splicing layer, and a convolutional layer;

[0031] The first four branches each include a convolution layer, a batch normalization layer and an activation function. The void rates of the convolution in the convolution layers in the first four branches are set to 1, 6, 12, and 18, respectively. The fifth branch consists of a global pooling layer, a convolution layer, a batch normalization layer, an activation function layer and an upsampling layer.

[0032] In one possible implementation, calculating the sum of all pixels within the bounding box in the spliced ​​segmented image, comparing the sum of the pixels with a threshold, and removing false alarms in the target detection result based on the comparison result specifically includes:

[0033] Obtaining the sum of all non-vehicle target pixel points within the bounding box in the spliced ​​segmented image;

[0034] The relationship between the sum of the non-vehicle target pixel points and a threshold is determined. If the sum of the non-vehicle target pixel points is greater than or equal to the threshold, the bounding box is a false alarm, and the false alarm is removed from the target detection result. If the sum of the non-vehicle target pixel points is less than the threshold, the bounding box is retained in the target detection result.

[0035] In a possible implementation, after mapping the bounding box onto the spliced ​​segmented image based on the bounding box regressed position, before performing the false alarm removal operation, the method further includes:

[0036] Using the NMS algorithm to remove the overlapping bounding boxes;

[0037] The spliced ​​segmented image is subjected to a binarization process and an erosion operation in morphology.

[0038] In a possible implementation manner, both the SAR image and the optical remote sensing image contain vehicles.

[0039] One or more technical solutions provided in the present invention have at least the following technical effects or advantages:

[0040] The present invention adopts a method for removing false alarms for assisting vehicle target detection, which comprises the following steps: obtaining mutually registered SAR images and optical remote sensing images, utilizing the same scene information in the optical remote sensing images and the SAR images, and utilizing the prior knowledge of clutter scenes that are unlikely to contain vehicle targets, thereby achieving the function of removing false alarms in vehicle target detection in complex scenes with artificial or natural clutter such as buildings and trees, and effectively improving the accuracy of SAR target detection; inputting SAR image slices into a trained target detection network to obtain classification categories and bounding box regression positions corresponding to the predicted feature layer; wherein the target detection network specifically comprises: a feature extraction module; The feature extraction module is used to extract features from SAR images using the ResNet network to obtain multiple network feature layers; the feature fusion module is used to use multiple identical feature fusion units based on attention context information to fuse multiple network feature layers and obtain fused features as prediction feature layers; the classification regression prediction module is used to use the classification prediction unit and the regression prediction unit to predict the category and position of the prediction feature layer to obtain preliminary target detection results, namely the classification category and bounding box regression position, and adopt the feature fusion module based on attention context information FFM_AC (Feature fusion module based on attention context information The present invention adopts the deep information to shallow information fusion under the attention mechanism. Instead of using the original pyramid structure to directly upsample the high-level information and add it with the shallow information at the feature layer, it adopts the improved attention-based context information. The deep feature layer is used to learn the weight of the context information, and the shallow feature layer is weighted multiplied to obtain more accurate and semantically rich features. The optical remote sensing image is input into the trained target segmentation network to obtain the target segmentation result, i.e., the spliced ​​segmentation image. The bounding box is mapped to the spliced ​​segmentation image according to the bounding box regression position, and the spliced ​​segmentation is calculated. The sum of all non-vehicle target pixel points within the bounding box in the image is calculated, and the sum of the non-vehicle target pixel points is compared with the threshold, and the false alarms in the target detection results are removed according to the comparison results; this solves the problem that when detecting targets in the existing technology, the false alarm rate is high and low-quality detection frames are easily generated, making it difficult to accurately detect targets. It realizes the combination of target scene segmentation and target detection, improves the ability to remove false alarms, and enhances the feature representation ability of the prediction feature layer on the basis of the original target detection framework, enhances the receptive field of the prediction feature layer and the context information contained therein, can adapt to targets of more sizes and shapes, and improves the detection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments of the present invention or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 A flowchart of the steps of a method for assisting vehicle target detection and removing false alarms provided by an embodiment of the present invention;

[0043] Figure 2 Schematic diagram of an attention feature fusion module based on attention context information provided by an embodiment of the present invention;

[0044] Figure 3 SAR images provided by the embodiments of the present invention;

[0045] Figure 4 The optical remote sensing image provided by the embodiment of the present invention;

[0046] Figure 5 A schematic diagram of a target segmentation framework provided by an embodiment of the present invention;

[0047] Figure 6 A schematic diagram of the Xception network framework of the feature extraction module provided in an embodiment of the present invention;

[0048] Figure 7 A schematic diagram of the network framework of a spatial diaphanous pyramid pooling unit provided in an embodiment of the present invention;

[0049] Figure 8 A schematic diagram of a network framework of a decoding module provided in an embodiment of the present invention;

[0050] Figure 9 False alarms in target detection removed using segmented images provided by an embodiment of the present invention; DETAILED DESCRIPTION

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present invention.

[0052] The present invention provides a method for assisting vehicle target detection in removing false alarms. Figure 1 As shown, the method includes the following steps S101 to S104.

[0053] S101, obtain mutually registered SAR images and optical remote sensing images. Based on the registered SAR image and optical image data, respectively design a vehicle target detection dataset for SAR images and a scene segmentation dataset for optical images, and respectively use a target detection algorithm to perform vehicle target detection in SAR images, and use the designed target segmentation algorithm to perform pixel-level segmentation of buildings and tree scenes in optical images. Finally, map the vehicle position of the target detection result with the scene splicing segmentation image, and reduce the false alarm phenomenon of target detection based on the relationship between the vehicle target and the segmented scene, screen out the real vehicle target, and reduce the false alarm phenomenon. In a specific embodiment proposed by the present invention, see Figure 3 SAR images and participation Figure 4 All optical remote sensing images contain vehicles.

[0054] The present invention combines target scene segmentation with target detection, and can utilize the same scene information in optical remote sensing images and SAR images. It also uses the prior knowledge that clutter scenes are unlikely to contain vehicle targets. This allows the removal of false alarms in vehicle target detection in complex scenes with artificial or natural clutter such as buildings and trees, effectively improving the accuracy of SAR target detection.

[0055] S102: Input the SAR image slice into the trained target detection network to obtain the target detection result. The target detection result includes the classification score and bounding box regression position corresponding to the predicted feature layer. The target detection network specifically includes a feature extraction module, a feature fusion module, and a classification regression prediction module. The feature fusion module in the target detection framework of the anchor-free SAR target detection method based on the supervised attention mechanism is improved. The feature fusion module of the present invention adopts the attention feature fusion module FFM_CA based on attention context information, see Figure 2 As shown, the deep information is fused with the shallow information under the attention mechanism. The present invention does not use the original pyramid structure to directly upsample the high-level information and add it with the shallow information at the feature layer. Instead, it adopts the improved attention-based context information, uses the deep feature layer to learn the weight of the context information, and performs weighted multiplication on the shallow feature layer to obtain more accurate features with richer semantic information, so that the shallow feature layer can contain more accurate global and local deep semantic information. At the same time, it also enhances the receptive field of the shallow feature, can adapt to targets of more sizes and shapes, and improves the detection effect.

[0056] The feature extraction module is used to extract features from SAR images using the ResNet18 network to obtain multiple network feature layers. Figure 3As shown in the figure, specifically, ResNet18 is used as the network structure of the feature extraction module. The input image size of this module is H×W×3, and the output size is feature maps, namely the C1 feature layer, the C2 feature layer, and the C3 feature layer.

[0057] Specifically, the ResNet18 structure mainly includes five blocks: block1, block2, block3, block4 and block5, and the module input is an image of size H×W×3.

[0058] Block 1: consists of a convolution layer L1, a batch normalization layer L2, an activation function layer L3 and a pooling layer L4. The convolution layer L1 is used to extract image features. Its convolution window size is 7×7, the sliding step is 2, and the padding attribute is 3. It is used to output Feature map of size X 1 ; Batch normalization layer L2, input is the feature layer X output by L1 1 , which normalizes each feature map to accelerate network training and improve the robustness of the model for output Feature map of size X 2 ; The activation function layer L3 uses ReLU, and its input is the feature layer X output by L2 2 , set the negative values ​​in the network to 0 and retain the positive values, and perform nonlinear transformation on the feature map for output Feature map of size X 3 ; Pooling layer L4 adopts the maximum pooling method, and its input is the feature layer X output by L3 3 , the pooling window size is 3×3, the sliding step is 2, and the padding attribute is 1, which is used to output Feature map of size X 4 , the output of block1 serves as the input of block2.

[0059] Block2: consists of two convolutional modules M1 and M2 and two residual connections R1 and R2. Specifically, the output of block1 is the input of M1 and R1; M1 includes two convolutional layers L5 and L6, which are used to output a size of The feature map X 5 , where the convolution window size of L5 and L6 is 3×3, the sliding step is 2, and the padding attribute is 1. The stacking of convolution is to increase the receptive field of the feature map and improve the target representation ability of the feature map. The output of M1 is another input of R1; R1 converts the output of block1 X 4 and the output X of M1 5 The corresponding elements are added to output a size of The feature map X 6 , the output of R1 is used as the input of M2 and R2; similarly, M2 includes two convolutional layers L7 and L8, which are used to output a size of The feature map X 7 , where the convolution window size of L7 and L8 is 3×3, the sliding step is 2, the padding attribute is 1, and the output of M2 is another input of R2; R2 converts the output of R1 to X 6 and the output X of M2 7 The corresponding elements are added to output a size of The feature map X 8 , the output of R2 serves as the input of block3.

[0060] Block3: includes two convolution modules M3 and M4 and two residual connections R3 and R4. Specifically, the output of block2 is the input of M3 and R3; M3 includes two convolution layers L9 and L 10 , for output size The feature map X 9 , among which L9, L 10 The convolution window size is 3×3, the sliding step is 2, and the padding attribute is 1. The stacking of convolutions is to increase the receptive field of the feature map and improve the target representation ability of the feature map. The output of M3 is another input of R3; R3 converts the output of block2 X 8 and the output X of M3 9 The corresponding elements are added to output a size of The feature map X 10 , the output of R3 is used as the input of M4 and R4; similarly, M4 includes two convolutional layers L 11 、L 12 , for output size The feature map X 11 , where L 11 、L 12 The convolution window size is 3×3, the sliding step is 2, the padding attribute is 1, and the output of M4 is another input of R4; R4 converts the output of R3 X 10 and the output X of M4 11 The corresponding elements are added to output a size of The feature map X 12 , the output of R4 is used as the input of block4. In addition, the output of R4 is the C1 feature layer of the feature extraction module, and it is also an input of the corresponding feature fusion module.

[0061] Block 4: includes two convolution modules M5 and M6 and two residual connections R5 and R6. Specifically, the output of block 3 is the input of M5 and R5; M5 includes two convolution layers L 13 、L 14 , for output size The feature map X 13 , where L 13 、L 14 The convolution window size is 3×3, the sliding step is 2, and the padding attribute is 1. The stacking of convolutions is to increase the receptive field of the feature map and improve the target representation ability of the feature map. The output of M5 is another input of R5; R5 converts the output of block3 X 12 and the output X of M5 13 The corresponding elements are added to output a size of The feature map X 14 , the output of R5 is used as the input of M6 and R6; similarly, M6 includes two convolutional layers L 15 、L 16 , for output size The feature map X 15 , where L 15 、L 16 The convolution window size is 3×3, the sliding step is 2, the padding attribute is 1, and the output of M6 is another input of R6; R6 converts the output of R5 X 14 and the output X of M6 15 The corresponding elements are added to output a size of The feature map X 16 , the output of R6 serves as the input of block5. In addition, the output of R6 is the C2 feature layer output of the feature extraction module, and it is also an input of the corresponding feature fusion module.

[0062] Block5: includes two convolution modules M7 and M8 and two residual connections R7 and R8. Specifically, the output of block4 is the input of M7 and R7; M7 includes two convolution layers L 17 、L 18 , for output size The feature map X 17 , where L 17 、L 18 The convolution window size is 3×3, the sliding step is 2, and the padding attribute is 1. The stacking of convolutions is to increase the receptive field of the feature map and improve the target representation ability of the feature map. The output of M7 is another input of R7; R7 converts the output of block4 X 16 and the output X of M7 17 The corresponding elements are added to output a size of The feature map X 18 , the output of R7 is used as the input of M8 and R8; similarly, M8 includes two convolutional layers L 19 、L 20 , for output size The feature map X 19 , where L 19 、L 20 The convolution window size is 3×3, the sliding step is 2, the padding attribute is 1, and the output of M8 is another input of R8; R8 converts the output of R7 to X 18 and M8 output X 19 The corresponding elements are added to output a size of The feature map X 20 , the output of R8 is the C3 feature layer of the feature extraction module, and it is also an input of the corresponding feature fusion module.

[0063] In S102, the feature fusion module is used to use multiple identical feature fusion units based on attention context information to fuse features of multiple network feature layers to obtain fused features. The feature fusion units based on attention context information specifically include: FFM_AC1 unit, FFM_AC2 unit and FFM_AC3 unit.

[0064] The multiple network feature layers include: C1 network feature layer, C2 network feature layer and C3 network feature layer;

[0065] The FFM_AC1 unit is specifically used to perform a convolution operation on the C3 network feature layer with a convolution kernel size of 1×1 and a step size of 1, and obtain a size of Feature layer Where H and W represent the SAR image slice size of the input target detection network, in this case H = 800, W = 1333; the neural network is used to detect the feature layer Perform attention map extraction and obtain a size of Feature layer Feature layer Perform four times upsampling to get a size of Feature layer P3.

[0066] The FFM_AC2 unit is specifically used to perform a convolution operation on the C2 network feature layer with a convolution kernel size of 1×1 and a step size of 1, and obtain a size of Feature layer Where H and W represent the SAR image slice size of the input target detection network, in this case H = 800, W = 1333; the neural network is used to detect the feature layer Perform attention map extraction and obtain a size of Feature layer Feature layer Perform double upsampling to get a size of Feature layer P2.

[0067] The FFM_AC3 unit is specifically used to perform a convolution operation on the C1 network feature layer with a convolution kernel size of 1×1 and a step size of 1, and obtain a size of Feature layer Where H and W represent the SAR image slice size of the input target detection network, in this case H = 800, W = 1333; the feature layer Multiply and add with the feature map P2 to get the feature layer P1 1 , which is expressed as The feature layer P1 1 Multiply and add the feature layer P3 to obtain the feature layer P1, which is expressed as P1=P1 1 +P1 1 ×P3.

[0068] According to the proposed feature fusion method of context information, the feature fusion of C1 and C2 is performed first, and then the feature fusion of C1 and C3 is performed. The interaction of context information in this example is mainly reflected in the above-mentioned P1=P1 1 +P1 1 ×P3, first perform feature fusion of C1 C2, and then perform feature fusion of C1 C3.

[0069] In S102, the method for extracting the attention map in the feature fusion unit based on attention context information specifically includes the following steps S201 to S202.

[0070] S201, obtain the spatial attention weight and channel attention weight of the input feature map from the spatial dimension and channel dimension respectively, and obtain the spatial attention weight and channel attention weight. The method for extracting the attention map includes a channel attention module and a spatial attention module, wherein the channel attention module is specifically composed of two pooling layers, wherein the pooling layer here is the pooling of the spatial dimension to remove redundant information in the spatial dimension and retain more accurate channel information features, two fully connected layers, one element addition layer, one activation function layer, and one element multiplication layer. That is, the first pooling layer P1, the second pooling layer P2, the third fully connected layer P3, the fourth fully connected layer P4, the fifth element addition layer P5, the sixth activation function layer P6, and the seventh element multiplication layer P7. The spatial attention module is specifically composed of two pooling layers, wherein the pooling layer here is the pooling of the channel dimension to remove redundant information in the channel dimension and retain more accurate spatial information features, one splicing layer, one convolution layer, one activation function layer, and one element multiplication layer. That is, the eighth pooling layer P8, the ninth pooling layer P9, the tenth splicing layer P 10 , the eleventh convolutional layer P 11 , the twelfth activation function layer P 12 , the thirteenth multiplication layer P 13 The activation function layer increases the nonlinear expression of the network model to achieve better feature representation, and the fully connected layer realizes the mapping of features to map the original features to the new feature space, that is, changing the feature dimension to obtain the required feature information.

[0071] S202, multiplying the channel attention weight and the spatial attention weight with the original feature map in sequence to obtain a feature layer after the attention map is extracted.

[0072] Specifically, Taking its network input as an example, the input, output and parameter settings and relationships of each layer are as follows:

[0073] The first pooling layer P1, whose pooling dimension is the spatial dimension, the pooling method is maximum pooling, and outputs a maximum pooling feature map This layer serves as the input of the third fully connected layer P3.

[0074] The second pooling layer P2, whose pooling dimension is the spatial dimension, the pooling method is average pooling, and outputs an average pooling feature map This layer serves as the input of the third fully connected layer P3.

[0075] The third fully connected layer P3, at this time the input vector dimension is 256, the output vector dimension is half of the input vector dimension, that is, 128, the feature map and feature maps They are sent to this layer respectively and will be used to output 128-dimensional column vectors respectively. and This layer serves as the input of the fourth fully connected layer.

[0076] The fourth fully connected layer P4, at this time the input vector dimension is 128, the output vector dimension is 256, the column vector and They are sent to this layer respectively and will be used to output 256-dimensional column vectors respectively. and This layer serves as the input to the fifth element-wise addition layer.

[0077] The fifth element addition layer P5 adds the 256-dimensional column vector and Element-by-element addition to output a 256-dimensional column vector This layer serves as the input of the sixth activation function layer P6.

[0078] The sixth activation function layer P6, whose activation function is Sigmoid function, is used to output a 256-dimensional column vector This layer serves as the input to the seventh element-wise multiplication layer P7.

[0079] The seventh element multiplication layer P7, whose initial features are The weight feature is The weighted method is element-by-element multiplication along the channel to output the feature map

[0080] The eighth pooling layer P8, whose pooling dimension is the channel dimension and the pooling method is maximum pooling, is used to output a maximum pooling feature map This layer is the tenth splicing layer P 10 input.

[0081] The ninth pooling layer P9, whose pooling dimension is the channel dimension and the pooling method is average pooling, is used to output an average pooling feature map This layer and the eighth pooling layer Together as the tenth splicing layer P 10 input.

[0082] Tenth splicing layer P 10 , whose splicing dimension is the channel dimension, used to output feature maps This layer serves as the eleventh convolutional layer P 11 input.

[0083] The eleventh convolutional layer P 11 , the window size of its convolution kernel K is 7×7, and the sliding step size S is 1, which is used to output the feature map This layer serves as the twelfth activation function layer P 12 input.

[0084] The twelfth activation function layer P 12 , whose activation function is Sigmoid function, used to output weight feature map This layer serves as the thirteenth multiplication layer P 13 input.

[0085] The thirteenth multiplication layer P 13 , whose initial characteristics are The weight feature is The weighted method is to multiply the spatial dimension element by element to output the feature map

[0086] The classification and regression prediction module in S102 is used to predict the category and position of the prediction feature layer using the classification prediction unit and the regression prediction unit, and obtain the classification category and bounding box regression position corresponding to the multiple prediction feature layers.

[0087] The classification prediction unit in the classification regression prediction module is specifically used for the following steps S301 to S303.

[0088] S301, performing a convolution operation on the prediction feature layer using four convolutions to obtain a prediction classification feature layer.

[0089] S302, performing a convolution operation on the prediction classification feature layer using convolution to obtain a prediction classification score layer.

[0090] S303, using Sigmoid normalization to perform a normalization operation on the predicted classification score layer to obtain a classification score graph, that is, to obtain the classification category.

[0091] Specifically, the feature layer P1 output by the feature fusion module is first subjected to four convolution operations. The convolution parameters are: the window size of the convolution kernel is 3×3, the step size is 1, and the padding attribute is 1. The obtained feature layer C p , whose size is Then the feature layer C p Perform another convolution operation with the parameters of the convolution kernel window size of 3×3, step size of 1, padding attribute of 1, and a Sigmoid normalization operation, and finally get a The output classification score X of the classification prediction unit of size cls .

[0092] The regression prediction unit in the classification regression prediction module is specifically used for the following steps S401 to S402.

[0093] S401, performing a convolution operation on the predicted feature layer using four convolutions to obtain a feature layer of the predicted bounding box regression position.

[0094] S402, using convolution to perform a convolution operation on the feature layer of the predicted bounding box regression position to obtain the bounding box regression position.

[0095] In a specific embodiment of the present invention, the feature layer P1 output by the feature fusion module is first subjected to four convolution operations to obtain a feature layer R for predicting the bounding box regression position. p ; Then the feature map R p Perform a convolution operation to obtain the bounding box regression position X predicted by the regression branch reg .

[0096] Specifically, the feature layer P1 output by the feature fusion module is first subjected to four convolution operations. The convolution parameters are: the window size of the convolution kernel is 3×3, the step size is 1, and the padding attribute is 1. The resulting feature layer R p , whose size is Then the feature layer R p Perform another convolution operation with the parameters of the convolution kernel window size of 3×3, step size of 1, and padding of 1, and finally get a The bounding box regression position of the regression prediction unit of size X reg .

[0097] S103, inputting the optical remote sensing image slices into the trained target segmentation network to obtain the segmentation results of multiple slices. The target segmentation network includes: an encoding module and a decoding module. The overall structure framework of the target segmentation network is shown in Figure 5 shown.

[0098] The encoding module specifically includes a feature extraction module and a dilated spatial pyramid pooling unit. The feature extraction module is used to extract features from optical remote sensing images to obtain shallow features and deep features. The dilated spatial pyramid pooling unit is used to use dilated convolutions of different sizes to obtain feature maps of different receptive fields for deep features, and to splice feature maps of different receptive fields to obtain deep features.

[0099] For details, see Figure 6 , Figure 6 This is a network framework diagram of the feature extraction module Xception provided by an embodiment of the present invention. The feature extraction module adopts the network structure, parameter settings and corresponding relationships of Xception as follows:

[0100] The Xception network can be divided into three parts in order: Entry, Middle, and Exit. The image size of the input network is H×W×3, and H=W=512.

[0101] (1) The entry part mainly consists of two convolutional layers conv1 and conv2, two batch normalization layers BN1 and BN2, an activation layer ReLU1 and three blocks.

[0102] Specifically, Take the network input size of H×W×3 as an example, where H=W=512:

[0103] Conv1: includes a convolution layer with a window size of 3×3, a step size of 2, and a padding attribute of 1, which is used to output feature maps Its size is Where H and W represent the image slice size of the input target segmentation network, in this case H = W = 512, and the output of conv1 is used as the input of BN1.

[0104] BN1, ReLU1: Each includes a batch normalization layer and an activation function layer, and the normalization method is Where x is the set of all feature pixels of each batch in the feature map, μ and σ are the mean and variance of all feature pixels of each batch in the feature map, respectively. The output of BN1 is used as the input of ReLU1. The activation function used by ReLU1 is ReLU. The ReLU activation function is used to increase the nonlinear expression of the network model to achieve better feature representation. The output feature map is Its size is Where H and W represent the image slice size of the input target segmentation network, in this case H = W = 512, and the output of ReLU1 is used as the input of conv2.

[0105] Conv2: includes a convolutional layer with a window size of 3×3, a step size of 1, and a padding attribute of 1, which is used to output feature maps Its size is Where H and W represent the image slice size of the input target segmentation network, in this case H = W = 512, and the output of conv2 is used as the input of BN2.

[0106] BN2 includes a batch normalization layer, the normalization method is Where x is the set of all feature pixels of each batch in the feature map, μ and σ are the mean and variance of all feature pixels of each batch in the feature map, respectively, used to output the feature map. Its size is Where H and W represent the image slice size of the input target segmentation network. In this case, H = W = 512, and the output of BN2 is used as the input of block1.

[0107] Block1 to block20 each include three depth-wise separable convolutions and a residual structure, where the depth-wise separable convolution is defined as, in sequence, a ReLU activation function, a depth-wise convolution, a batch normalization layer, a ReLU activation function, a point convolution, a batch normalization layer and a ReLU activation function. In the depth-wise separable convolution, the depthwise convolution window size is 3×3, the padding attribute is 1, the step size and the number of output channels are defined by each block, the pointwise convolution window size is 1×1, the step size is 1, the padding attribute is 0, and the number of output channels is defined by each block; compared with ordinary convolution, the depth-wise separable convolution can reduce a large amount of parameters and calculations. The normalization method in the batch normalization layer is Where x is the set of all feature pixels of each batch in the feature map, μ and σ are the mean and variance of all feature pixels of each batch in the feature map respectively; the residual structure consists of a convolution layer and a batch normalization layer, where the convolution layer window size is 1×1, the stride is 2, the padding attribute is 1, and the normalization method in the batch normalization layer is Where x is the set of all feature pixels of each batch in the feature map, μ and σ are the mean and variance of all feature pixels of each batch in the feature map, respectively.

[0108] Block 1: At this time, the three consecutive depth-wise separable convolutions are set as follows: the first depth-wise separable convolution has 64 input channels, 128 output channels, and a stride of 1, where the depth-wise convolution has 64 input channels, 128 output channels, and a stride of 1, and the point-wise convolution has 128 input channels and 128 output channels; the second depth-wise separable convolution has 128 input channels, 128 output channels, and a stride of 1, where the depth-wise convolution has 128 input channels, The number of output channels is 128, the stride is 1, the number of input channels of the point convolution is 128, and the number of output channels is 128; the number of input channels of the third depth-wise separable convolution is 128, the number of output channels is 128, and the stride is 2, among which the number of input channels of the depth-wise convolution is 128, the number of output channels is 128, and the stride is 2, the number of input channels of the point convolution is 128, and the number of output channels is 128; the number of input channels of the convolution layer in the residual structure is 64, and the number of output channels is 128; the input feature map is the output of BN2 The feature map is input into three consecutive depth-separable convolution and residual structures to obtain the feature map and The corresponding positions of the elements are added to obtain the final output feature map Its size is Where H and W represent the image slice size of the input target segmentation network. In this case, H = W = 512, and the output of block 1 serves as the input of block 2.

[0109] Block2: At this time, the three consecutive depth-wise separable convolutions are set as follows: the first depth-wise separable convolution has an input channel number of 128, an output channel number of 256, and a step size of 1, of which the input channel number of the depth convolution is 128, the output channel number of 256, the step size of 1, and the input channel number of the point convolution is 256, and the output channel number of 256; the second depth-wise separable convolution has an input channel number of 256, the output channel number of 256, and a step size of 1, of which the input channel number of the depth convolution is 256, the output channel number of 256, the step size of 1, and the input channel number of the point convolution is 256, and the output channel number of 256. At this time, the output feature map is Size As one of the final outputs of the feature extraction module; the third depth-wise separable convolution has 256 input channels, 256 output channels, and a stride of 2, of which the depth-wise convolution has 256 input channels, 256 output channels, and a stride of 2, and the point-wise convolution has 256 input channels and 256 output channels; the convolution layer in the residual structure has 128 input channels and 256 output channels; the input feature map is the output of block 1 The feature map is input into three consecutive depth-separable convolution and residual structures to obtain the feature map and The corresponding positions of the elements are added to obtain the final output feature map Its size is Where H and W represent the image slice size of the input target segmentation network. In this case, H = W = 512, and the output of block 2 is used as the input of block 3. The output of the second depth-wise separable convolution is the shallow feature L, that is, It is one of the final outputs of the feature extraction module and also the input of the decoding module.

[0110] Block 3: At this time, the three consecutive depth-wise separable convolutions are set as follows: the first depth-wise separable convolution has an input channel number of 256, an output channel number of 728, and a stride of 1, of which the depth-wise convolution has an input channel number of 256, an output channel number of 728, and a stride of 1, and the point-wise convolution has an input channel number of 728, and an output channel number of 728; the second depth-wise separable convolution has an input channel number of 728, an output channel number of 728, and a stride of 1, of which the depth-wise convolution has an input channel number of 728, an output channel number of 728, and a stride of 1, The number of output channels is 728, the stride is 1, the number of input channels of the point convolution is 728, and the number of output channels is 728; the number of input channels of the third depth-wise separable convolution is 728, the number of output channels is 728, and the stride is 2, among which the number of input channels of the depth-wise convolution is 728, the number of output channels is 728, and the stride is 2, the number of input channels of the point convolution is 728, and the number of output channels is 728; the number of input channels of the convolution layer in the residual structure is 256, and the number of output channels is 728; the input feature map is the output of block2 The feature map is input into three consecutive depth-separable convolution and residual structures to obtain the feature map and The corresponding positions of the elements are added to obtain the final output feature map Its size is Where H and W represent the image slice size of the input target segmentation network. In this case, H = W = 512, and the output of block 3 is used as the input of the Middle part.

[0111] (2) Middle part, input is the output of Entry This part mainly consists of 16 identical blocks, block 4 to block 19; the block settings are the same and still consist of three consecutive depth-wise separable convolutions and a residual structure. The specific settings are: the number of input channels of depth-wise convolution and point convolution in the depth-wise separable convolution is 728, the number of output channels is 728, the step size is 1, the void rate is 1, and the residual structure is to directly add the corresponding elements of the input and output of each block, and finally output the feature map Its size is Where H and W represent the image slice size of the input target segmentation network. In this case, H = W = 512. The stacking of the convolutional modules of the network further deepens the network, allowing the extraction of deeper feature maps with more semantic information.

[0112] (3) The input of the Exit part is the output of the Middle part This part mainly consists of a block20 and three convolutional layers conv3, conv4 and conv5.

[0113] Block20: The depth-wise separable convolution inside it is set as follows: the first depth-wise separable convolution has 728 input channels, 728 output channels, and a stride of 1, where the depth-wise convolution has 728 input channels, 728 output channels, and a stride of 1, and the point-wise convolution has 728 input channels, 728 output channels; the second depth-wise separable convolution has 728 input channels, 1024 output channels, and a stride of 1, where the depth-wise convolution has 728 input channels, 1024 output channels, and a stride of 1, 24, stride 1, the number of input channels of the point convolution is 1024, and the number of output channels is 1024; the number of input channels of the third depth-wise separable convolution is 1024, the number of output channels is 1024, and the stride is 2; the number of input channels of the depth-wise convolution is 1024, the number of output channels is 1024, and the stride is 1, the number of input channels of the point convolution is 1024, and the number of output channels is 1024; the number of input channels of the convolution layer in the residual structure is 728, and the number of output channels is 1024; the input feature map is the output of block 19 The feature map is input into three consecutive depth-separable convolution and residual structures to obtain the feature map and The corresponding positions of the elements are added to obtain the final output feature map Its size is Where H and W represent the image slice size of the input target segmentation network. In this case, H = W = 512, and the output of block20 is used as the input of conv3.

[0114] conv3: It uses a depthwise separable convolution module, with the following settings: the number of input channels of depthwise convolution is 1024, the number of output channels is 1536, the step size is 1, the number of input channels of pointwise convolution is 1536, the number of output channels is 1536; the final output feature map Its size is Where H and W represent the image slice size of the input target segmentation network, in this case H = W = 512, and the output of conv3 is used as the input of conv4.

[0115] conv4: It uses a depthwise separable convolution module with the following settings: the number of input channels of depthwise convolution is 1536, the number of output channels is 1536, the step size is 1, the number of input channels of pointwise convolution is 1536, the number of output channels is 1536; the final output feature map Its size is Where H and W represent the image slice size of the input target segmentation network, in this case H = W = 512, and the output of conv4 is used as the input of conv5.

[0116] conv5: It uses a depth-wise separable convolution module with the following settings: the number of input channels of depth convolution is 1536, the number of output channels is 2048, the step size is 1, the number of input channels of point convolution is 2048, the number of output channels is 2048; the final output feature map Its size is Where H and W represent the image slice size of the input target segmentation network. At this time, H = W = 512, and the final feature map output X is obtained, that is,

[0117] At this time, the feature extraction module obtains two parts of output: the shallow feature L and the final output feature X, which are respectively input into the subsequent decoding module and the void space pyramid pooling unit.

[0118] The dilated spatial pyramid pooling unit specifically includes: five parallel branches, a splicing layer, and a convolutional layer. The first four branches each include a convolutional layer, a batch normalization layer, and an activation function, namely the first branch B1, the second branch B2, the third branch B3, and the fourth branch B4. The fifth branch consists of a global pooling layer, a convolutional layer, a batch normalization layer, an activation function layer, and an upsampling layer, namely the fifth branch B5, and the subsequent sixth splicing layer B6 and seventh convolutional layer B7. The dilation rates of the convolutional layers in the first four branches are set to 1, 6, 12, and 18, respectively. For details, please refer to Figure 7 , Figure 7 4 is a network framework diagram of a dilated spatial pyramid pooling unit provided by an embodiment of the present invention.

[0119] Specifically, the feature map As an example of network input, the process is as follows:

[0120] The first branch B1: This branch consists of a convolution layer, a batch normalization layer, and an activation function layer, which are set as follows: the convolution window size is set to 1×1, the number of convolution kernels is 256, the step size is set to 1, the padding attribute is set to 0, and the void rate is set to 1; the batch normalization method is Where x is the set of all feature pixels of each batch in the feature map, μ and σ are the mean and variance of all feature pixels of each batch in the feature map respectively; the activation function is ReLU, and the output feature map is Size Where H and W represent the image slice size of the input target segmentation network. In this case, H = W = 512, which is used as the input of the sixth splicing layer B6.

[0121] The second branch B2: This branch consists of a convolution layer, a batch normalization layer, and an activation function layer, which are set as follows: the convolution window size is set to 3×3, the number of convolution kernels is 256, the step size is set to 1, the padding attribute is set to 6, and the void rate is set to 6; the batch normalization method is Where x is the set of all feature pixels of each batch in the feature map, μ and σ are the mean and variance of all feature pixels of each batch in the feature map, the activation function is ReLU, and the output feature map is Size Where H and W represent the image slice size of the input target segmentation network. In this case, H = W = 512, which is used as the input of the sixth splicing layer B6.

[0122] The third branch B3: This branch consists of a convolution layer, a batch normalization layer, and an activation function layer, which are set as follows: the convolution window size is set to 3×3, the number of convolution kernels is 256, the step size is set to 1, the padding attribute is set to 12, and the void rate is set to 12; the batch normalization method is Where x is the set of all feature pixels of each batch in the feature map, μ and σ are the mean and variance of all feature pixels of each batch in the feature map respectively; the activation function is ReLU, and the output feature map is Size Where H and W represent the image slice size of the input target segmentation network. In this case, H = W = 512, which is used as the input of the sixth splicing layer B6.

[0123] The fourth branch B4: This branch consists of a convolution layer, a batch normalization layer, and an activation function layer, which are set as follows: the convolution window size is set to 3×3, the number of convolution kernels is 256, the step size is set to 1, the padding attribute is set to 18, and the void rate is set to 18; the batch normalization method is Where x is the set of all feature pixels of each batch in the feature map, μ and σ are the mean and variance of all feature pixels of each batch in the feature map respectively; the activation function is ReLU, and the output feature map is Size Where H and W represent the image slice size of the input target segmentation network. In this case, H = W = 512, which is used as the input of the sixth splicing layer B6.

[0124] The fifth branch B5: This branch consists of a global pooling layer, a convolution layer, a batch normalization layer, an activation function layer and an upsampling layer, which are set as follows: the pooling layer is set to global average pooling, the convolution window size is set to 1×1, the number of convolution kernels is 256, the step size is set to 1, the padding attribute is set to 0, and the void rate is set to 0; the batch normalization method is Where x is the set of all feature pixels of each batch in the feature map, μ and σ are the mean and variance of all feature pixels of each batch in the feature map respectively; the activation function is ReLU, the upsampling layer uses bilinear interpolation, the upsampling multiple is 32, and the output feature map is Size Where H and W represent the image slice size of the input target segmentation network. In this case, H = W = 512, which is used as the input of the sixth splicing layer B6.

[0125] The sixth splicing layer B6: the output of the above five branches Splicing along the channel dimension, the output feature map is Size Where H and W represent the image slice size of the input target segmentation network, and H = W = 512, which is used as the input of the seventh convolutional layer.

[0126] The seventh convolutional layer B7; the convolution window size is set to 1×1, the step size is set to 1, the padding attribute is set to 0, the number of convolution kernels is 256, and the output feature map is Size Where H and W represent the image slice size of the input target segmentation network. In this case, H = W = 512, which is used as the input of the decoding module and is named deep feature D.

[0127] The decoding module includes two convolution modules, two upsampling layers, a splicing layer, and a convolution layer. The decoding module is used to splice and fuse shallow layer features with deep convolution layer features to obtain a segmented image, namely the first convolution module B1′, the second upsampling layer B2′, the third splicing layer B3′, the fourth convolution module B4′, the fifth convolution layer B5′, and the sixth upsampling layer B6′. For details, please refer to Figure 8 , Figure 8 This is a network framework diagram of the decoding module provided by an embodiment of the present invention.

[0128] Specifically, taking the shallow feature L and the deep feature D as the network input as an example, the process is as follows:

[0129] The first convolution module B1′, the input of this branch is the shallow feature L output by block2 in the encoding module. This branch consists of a convolution layer, a batch normalization layer, and an activation function layer. The parameters are set as follows: the convolution window size is set to 1×1, the step size is set to 1, the number of convolution kernels is 48, and the batch normalization method is Where x is the set of all feature pixels of each batch in the feature map, μ and σ are the mean and variance of all feature pixels of each batch in the feature map, the activation function is ReLU, and the output feature map is Size Where H and W represent the image slice size of the input target segmentation network, and in this case H = W = 512, which is used as the input of the third splicing layer B3′.

[0130] The second upsampling layer B2′, the input of this layer is the output of the deep feature D of the 7th convolutional layer B7 in the void space pyramid pooling unit. The upsampling layer uses bilinear interpolation, the upsampling multiple is 4, and the output feature map is Size Where H and W represent the image slice size of the input target segmentation network, and in this case H = W = 512, which is used as the input of the third splicing layer B3′.

[0131] The third splicing layer B3′, the feature map and Splicing along the channel dimension, the output feature map is Size Where H and W represent the image slice size of the input target segmentation network, and H = W = 512, which is used as the input of the fourth convolution module B4′.

[0132] The fourth convolution module B4′ consists of a convolution layer, a batch normalization layer, and an activation function layer. The parameters are set as follows: the convolution window size is set to 3×3, the step size is set to 1, and the batch normalization method is Where x is the set of all feature pixels of each batch in the feature map, μ and σ are the mean and variance of all feature pixels of each batch in the feature map, the activation function is ReLU, and the output feature map is Size Where H and W represent the image slice size of the input target segmentation network, and H = W = 512, which is used as the input of the fifth convolutional layer B5′.

[0133] In the fifth convolutional layer B5′, the convolution window size is set to 1×1, the step size is set to 1, and the number of convolution kernels is set to 3, which represents the number of categories trained by the network (background, building, tree). The output feature map is Size Where H and W represent the image slice size of the input target segmentation network, which is used as the input of the sixth upsampling layer B6′.

[0134] The sixth upsampling layer B6′, the input of this branch is the feature map The upsampling layer uses bilinear interpolation, the upsampling factor is 4, and the output feature map is The size is H×W×3, where H and W represent the image slice size of the input target segmentation network, and H=W=512 in this case.

[0135] S104: Map the bounding box onto the final spliced ​​segmented image based on the bounding box regression position, calculate the sum of all non-vehicle target pixels within the bounding box in the mapped image, compare the sum of the non-vehicle target pixels with a threshold, and remove false alarms in the target detection results based on the comparison result, specifically including:

[0136] (1) Obtain the sum of all non-vehicle target pixels within the bounding box in the spliced ​​segmented image.

[0137] (2) Determine the relationship between the sum of the non-vehicle target pixel points and the threshold. If the sum of the non-vehicle target pixel points is greater than or equal to the threshold, the bounding box is a false alarm and the false alarm in the target detection result is removed; if the sum of the non-vehicle target pixel points is less than the threshold, the bounding box is retained in the target detection result.

[0138] In a specific embodiment provided by the present invention, the position of the vehicle target in the SAR image is mapped onto the building and tree scene segmentation map. The sum of the pixels belonging to the building and tree scene in each vehicle bounding box is calculated. If the pixel is greater than a set threshold, the vehicle box is considered a false alarm and needs to be discarded. Vehicle bounding boxes smaller than the threshold are retained. In this case, the threshold is set to 200 and can be manually adjusted according to different images. Finally, the vehicle target detection result after false alarm removal is obtained.

[0139] In the method proposed in the present invention, before mapping the bounding box to the spliced ​​segmented image based on the bounding box regression position, it also includes: using the NMS algorithm to remove overlapping bounding boxes, binarizing the spliced ​​segmented image, and performing morphological corrosion operations. The NMS operation removes overlapping target detection boxes based on the score and intersection-union ratio. Thus, the final target detection result is obtained. At this time, the splicing strategy for splicing multiple segmented images into the final spliced ​​segmented image in scene segmentation is: if there are overlapping parts in different slice images, take the union of non-target pixels in all segmentation results. Thus, a preliminary scene segmentation result is obtained. The preliminary scene segmentation result is then binarized, that is, the part with a pixel value greater than 0 is set to 1, and the part with a pixel value equal to 0 remains unchanged, to obtain a building and tree scene image, that is, the positions with pixel values ​​are all considered to be buildings or trees, and this scene is the same as the target detection SAR image scene. Finally, the building and tree scene image is subjected to morphological processing-corrosion operation to remove some small noise spots to obtain the final scene segmentation result image of scene segmentation. See. Figure 9 False alarm instances in target detection results removed using segmented images.

[0140] Specifically, the specific principles of the corrosion operation in morphological processing are as follows:

[0141] Let X be the image set, B be the structuring element to be eroded, and B define an origin point called the anchor point. The erosion process is as follows: slide B over X. When B's anchor point translates to a pixel (x, y) in image X, if the structuring element B is completely contained within the region overlapping with image X (i.e., all the corresponding image values ​​in X corresponding to the positions of elements in B that are 1 are also 1), then the corresponding pixel (x, y) in the output image is assigned a value of 1; otherwise, it is assigned a value of 0. The erosion of X using B is the set of all anchor points in B that are contained within X. Specifically, when B's anchor point is at the boundary of a region with pixel values ​​within X, further movement of B's ​​anchor point will exceed this boundary and no longer be completely contained within image set X, resulting in an erosion operation, which reduces the image outline. It is the position of the anchor point at this boundary that constitutes the erosion of X using B. This method can eliminate objects or noise in X that are smaller than the structure of B or that do not match B. It can be expressed as follows:

[0142]

[0143] The structuring element used in this invention is a disk with a radius of 15. Its structuring element is symmetrical about the origin, allowing for full-scale erosion. This is suitable for extracting the top, bottom, left, and right edges of the target object, and is effective for processing segmented images with irregular edges. This erosion operation eliminates object boundary points, shrinking the boundary inward. It can also remove objects smaller than the structuring element or separate two objects with small connections, resulting in more accurate scene segmentation results and preventing the removal of vehicles.

[0144] In one embodiment of the present invention, the method further includes training the target detection network and the target segmentation network.

[0145] Step 1: Slice the registered SAR image and optical remote sensing image separately to construct the training set and test set for target detection and target segmentation.

[0146] In step 1, it includes: constructing a new training set and constructing a new test set.

[0147] The construction of new training sets includes: (1) target detection training set. In this example, the training set is constructed by sliding window slicing with 5 SAR images. The original image size is 2510×1680, the slice size is 800×1333, and the sliding window step is set to 45. Finally, 2564 SAR image slices are obtained. (2) target segmentation training set. In this example, the training set is constructed by sliding window slicing with the optical remote sensing images corresponding to the 5 SAR images. The original image size is 2510×1680, the slice size is 512×512, and the sliding window step is set to 128. In order to ensure that there are a certain number of positive samples in the slice image, the slices are screened. The screening principle is: during the sliding window process of the original large image, if the sum of the pixels of the segmented target (i.e., buildings and trees) in the slice is greater than 2500, the slice is retained; otherwise, it is discarded. Finally, 464 optical remote sensing image slices are obtained.

[0148] The construction of a new test set includes: (1) target detection test set. In this example, a test set is constructed by sliding window slicing based on a SAR image. The original image size is 2510×1680, the slice size is 800×1333, and the sliding window step size is set to 150. Finally, 63 SAR image slices are obtained. The position of each test image slice relative to the original SAR image is recorded, namely Loc, which includes the horizontal and vertical coordinates of the upper left corner of each slice, which is used to aggregate the test results of the small image into the large image. (2) target segmentation test set: In this example, a test set is constructed by sliding window slicing based on an optical remote sensing image corresponding to a SAR image. The original image size is 2510×1680, the slice size is 512×512, and the sliding window step size is set to 100. Finally, 273 optical remote sensing image slices are obtained. Similarly, the position of each test image slice relative to the original optical remote sensing image is recorded, namely Loc, which includes the horizontal and vertical coordinates of the upper left corner of each slice, which is used to aggregate the test results of the small image into the large image.

[0149] Step 2: Construct a target detection network and a target segmentation network, and train the target detection network and the target segmentation network according to the target detection training set and the target segmentation training set to obtain the trained target detection network and the target segmentation network.

[0150] The object segmentation model in this invention uses a transfer learning approach, employing a progressive pre-training method. The object segmentation network is first trained on a very large optical dataset, enabling the feature extraction module to extract universal optical features. It is then trained on a larger optical remote sensing building dataset, enabling the feature extraction module to extract universal optical building target features. Finally, a freeze-and-thaw training technique is used to fine-tune the model on a small optical remote sensing dataset, further improving the model's generalization and accuracy. Furthermore, the invention can further optimize the model's performance by adding new training data.

[0151] In order to further verify the effectiveness of the false alarm removal method for assisting vehicle target detection proposed in the present invention, this example also performs detection on MiniSAR data images.

[0152] See Figure 3 , Figure 3It is the MiniSAR test image used in the experiment of the present invention; Table 1 shows the performance of the present invention and the CFAR-Guided-EfficientDet SAR image target detection method with better target detection performance at this stage (referred to as CFAR-Guided-EfficientDet, from the paper "Research on SAR Image Target Detection and Identification Methods", doctoral dissertation of Xidian University, Wang Ning, 2021), the dual-stream SSD SAR image target detection method based on CFAR guidance (referred to as ICSAF-CFAR-SSD, from the paper "Research on SAR Target Detection Combining Constant False Alarm Detection and Deep Network", master's thesis of Xidian University, Tang Tiangu, 2022) and the patent number 202310153318.2 - the anchor-free SAR target detection method based on supervised attention mechanism (referred to as Tri-ICSAF-CSSCAM-Gradient-CFAR-FCOS) The reproduction version (the reproduction device is configured as Windows 10 system, GPU: 3060) Figure 3 Vehicle target detection performance indicators on the MiniSAR data image shown.

[0153] Table 1 Different detection networks Figure 3 Comparison of target detection performance on MiniSAR data images shown

[0154]

[0155] In Table 1, Pre represents precision, i.e., the percentage of true targets in the detected target frame; Rec represents recall, i.e., the percentage of targets correctly detected; and F1-Score represents the harmonic mean, which is a systematic indicator that unifies precision Pre and recall Rec. The target detection partial context information fusion method proposed in the present invention is an improved method that incorporates context information fusion on the basis of the anchor-free SAR target detection method based on the supervised attention mechanism. The final false alarm removal method proposed in the present invention is the final result of the false alarm removal method of the present invention based on the above improved target detection method.

[0156] It can be seen from Table 1 that the method proposed by the present invention Figure 3The precision and recall of SAR image target detection results on the data images shown are improved. The improvement of the feature fusion module in the target detection network using the attention-based contextual feature fusion algorithm can effectively enhance the feature representation ability of the prediction feature layer. The experimental results show that compared with the CFAR-Guided-EfficientDet method and Tri-ICSAF-CSSCAM-Gradient-CFAR-FCOS method in the comparison methods, the recall rate of target detection is improved to a certain extent, and is comparable to the recall rate of ICSAF-CFAR-SSD. The use of the false alarm removal algorithm can effectively remove false alarms in natural and artificial clutter in complex scenes, thereby improving the precision. The final precision index is higher than all the above comparison methods, and the harmonic mean index is also improved to a certain extent. Compared with the comparison methods, the overall effect is improved to a certain extent.

[0157] In general, the present invention is mainly used for false alarm removal. Figure 3 In this complex scenario, which contains a high level of both man-made and natural clutter, the proposed method can more accurately detect targets. Experimental results also validate the accuracy of the proposed method. The proposed method achieves superior SAR target detection performance compared to existing SAR target detection methods, demonstrating its effectiveness and superiority.

[0158] The various embodiments in this specification are described in a progressive manner. References to the same or similar parts between the various embodiments are sufficient. Each embodiment focuses on the differences from other embodiments. All or part of the present invention can be used in a variety of general or specialized computer system environments or configurations. For example, personal computers, server computers, handheld or portable devices, tablet devices, mobile communication terminals, multiprocessor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments that include any of the above systems or devices.

[0159] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it should be understood by those skilled in the art that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the present invention.

Claims

1. A method for removing false alarms for assisting vehicle target detection, characterized in that: include: Acquire mutually registered SAR images and optical remote sensing images; Input the slice of the SAR image into the trained target detection network to obtain the target detection result, which includes: the classification category score and the bounding box regression position corresponding to the predicted feature layer; wherein the target detection network specifically includes: a feature extraction module, a feature fusion module and a classification regression prediction module; the feature extraction module is used to use the ResNet network to extract features from the SAR image to obtain multiple network feature layers; the feature fusion module is used to use multiple identical feature fusion units based on attention context information to perform feature fusion on the multiple network feature layers to obtain fusion features as the predicted feature layer; the classification regression prediction module is used to use the classification prediction unit and the regression prediction unit to score the classification category of the predicted feature layer and predict the bounding box regression position to obtain the initial classification category score and the initial bounding box regression position corresponding to the predicted feature layer, and integrate the initial classification category score and the initial bounding box regression position into the SAR image to obtain the classification category score and the bounding box regression position; Inputting the slices of the optical remote sensing image into a trained target segmentation network to obtain segmented images of multiple slices, and splicing the multiple segmented images to obtain a spliced ​​segmented image; According to the bounding box regression position, the bounding box is mapped to the spliced ​​segmented image, the sum of all pixel points inside the bounding box in the spliced ​​segmented image is calculated, and the sum of the pixel points is compared with a threshold value, and false alarms in the target detection result are removed according to the comparison result.

2. The method according to claim 1, characterized in that The feature fusion unit based on attention context information includes: FFM_AC1 unit, FFM_AC2 unit and FFM_AC3 unit; The multiple network feature layers include: a C1 network feature layer, a C2 network feature layer, and a C3 network feature layer; The FFM_AC1 unit is specifically used to: perform a convolution operation on the C3 network feature layer with a convolution kernel size of 1×1 and a step size of 1 to obtain a size of Feature layer Where H and W represent the SAR image slice size of the input target detection network. At this time, H = 800, W = 1333. The neural network is used to detect the feature layer. Extract the attention map and get the size of Feature layer For the feature layer Upsample to get a size of Feature layer P3; The FFM_AC2 unit is specifically used to: perform a convolution operation on the C2 network feature layer with a convolution kernel size of 1×1 and a step size of 1 to obtain a size of Feature layer Where H and W represent the SAR image slice size of the input target detection network, in this case H = 800, W = 1333; the neural network is used to detect the feature layer Extract the attention map and get the size of Feature layer For the feature layer Upsample to get a size of Feature layer P2; The FFM_AC3 unit is specifically used to: perform a convolution operation on the C1 network feature layer with a convolution kernel size of 1×1 and a step size of 1 to obtain a size of Feature layer Where H and W represent the SAR image slice size of the input target detection network, in this case H = 800, W = 1333; Multiply and add the feature map P2 to obtain the feature layer P1 1 , which is expressed as The feature layer P1 1 Multiply and add the characteristic layer P3 to obtain the characteristic layer P1, which is expressed as P1=P1 1 +P1 1 ×P3.

3. The method according to claim 2, characterized in that The method for extracting the attention map specifically includes: Obtaining the spatial attention weight and the channel attention weight of the input feature map from the spatial dimension and the channel dimension respectively, to obtain the spatial attention weight and the channel attention weight; The channel attention weight and the spatial attention weight are multiplied with the original feature map in sequence to obtain the feature layer after the attention map is extracted.

4. The method according to claim 1, characterized in that: The classification prediction unit is specifically used for: Performing a convolution operation on the prediction feature layer using four convolutions to obtain a prediction classification feature layer; Performing a convolution operation on the prediction classification feature layer using convolution to obtain a prediction classification score layer; The predicted classification score layer is normalized using Sigmoid normalization to obtain a classification feature map, that is, to obtain the classification category.

5. The method according to claim 1, characterized in that The regression prediction unit is specifically used for: Performing a convolution operation on the predicted feature layer using four convolutions to obtain a feature layer of the predicted bounding box regression position; A convolution operation is performed on the feature layer of the predicted bounding box regression position using convolution to obtain the bounding box regression position.

6. The method according to claim 1, characterized in that The target segmentation network includes: an encoding module and a decoding module; The encoding module specifically includes a feature extraction module and a dilated space pyramid pooling unit, wherein the feature extraction module is used to extract features from the optical remote sensing image to obtain shallow features and deep features; the dilated space pyramid pooling unit is used to use dilated convolutions of different sizes to obtain feature maps of different receptive fields for the deep features, and to splice the feature maps of different receptive fields to obtain convolution deep features; The decoding module includes two convolution modules, two upsampling layers, a splicing layer and a convolution layer. The decoding module is used to perform channel splicing and fusion of the shallow layer features and the convolution deep layer features to obtain the slice segmentation result.

7. The method according to claim 6, characterized in that The atrous spatial pyramid pooling unit specifically includes: five parallel branches, a splicing layer and a convolution layer; The first four branches each include a convolution layer, a batch normalization layer and an activation function. The void rates of the convolutions in the convolution layers in the first four branches are set to 1, 6, 12, and 18, respectively. The fifth branch consists of a global pooling layer, a convolution layer, a batch normalization layer, an activation function layer and an upsampling layer.

8. The method according to claim 1, characterized in that The calculating the sum of all pixels in the spliced ​​segmented image that are inside the bounding box, comparing the sum of the pixels with a threshold, and removing false alarms in the target detection result according to the comparison result specifically includes: Obtaining the sum of all pixel points within the bounding box in the spliced ​​segmented image; The relationship between the sum of the pixel points and the threshold is determined. If the sum of the pixel points is greater than or equal to the threshold, the bounding box is a false alarm, and the false alarm in the target detection result is removed; if the sum of the pixel points is less than the threshold, the bounding box is retained in the target detection result.

9. The method according to claim 1, characterized in that: Before mapping the bounding box onto the spliced ​​segmented image according to the bounding box regression position, the method further includes: Using the NMS algorithm to remove the overlapping bounding boxes; The spliced ​​segmented image is subjected to binarization processing and morphological corrosion operation.

10. The method according to claim 1, characterized in that The SAR image and the optical remote sensing image are both images containing vehicles.

Images