A SAR image detection method based on contextual attention mechanism

Abstract

The present invention provides a SAR image detection method based on a contextual attention mechanism, which belongs to the field of image processing technology, and includes: obtaining a processed SAR image, inputting a Transformer backbone network module based on a multi-head self-attention module, and extracting a multi-resolution feature map; performing resolution cross-feature fusion on the multi-resolution feature map through upsampling and the same multi-head self-attention module to obtain multiple fused feature maps; performing target recognition of different sizes on each fused feature map to obtain a SAR image recognition result. The present invention improves the SAR image target detection method from multiple different angles, so that the network training speed, detection speed and accuracy are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a SAR image detection method based on a contextual attention mechanism. Background Art

[0002] Synthetic Aperture Radar (SAR) is an active microwave imaging sensor that can penetrate clouds, fog, smoke, and even leaves. It has all-weather and all-time observation capabilities and is suitable for target detection in complex scenes such as oceans, forests, and cities. SAR image target detection technology has been widely used in military, ocean, forestry, urban planning and other fields. Compared with optical images, the imaging mechanism of SAR images is very different. SAR targets have the characteristics of strong scattering, unclear edge contour information, multi-scale, strong sparse, weak, sidelobe interference, and complex background. Therefore, SAR target detection and recognition tasks face huge challenges. In recent years, the classification detection framework based on Transformer has received widespread attention. Transformer-based network models such as DETR and Swin Transformer have appeared one after another, and corresponding SAR image recognition methods have also appeared. When calculating the attention matrix, the traditional self-attention mechanism is usually based on paired independent queries and keys for each spatial position. It does not make full use of the rich contextual information between adjacent keys, resulting in inaccurate target recognition. Summary of the invention

[0003] In view of the key technical difficulties in the above-mentioned SAR target detection tasks, this paper combines the global context information perception of Transformer and the local information feature extraction capability of CNN for SAR target detection tasks, and innovatively proposes a SAR image ship detection method based on context joint representation learning. The SAR images are analyzed and processed, and the target detection method optimized for SAR images is used to achieve rapid detection of targets such as aircraft and ships in SAR images, providing strong support for related applications in military and civilian fields.

[0004] The present invention provides a SAR image detection method based on a contextual attention mechanism, which is characterized by comprising:

[0005] Obtain the processed SAR image and input it into the Transformer backbone network module based on the multi-head self-attention module to extract multi-resolution feature maps;

[0006] The multi-resolution feature maps are subjected to cross-resolution feature fusion by upsampling and the same multi-head self-attention module to obtain multiple fused feature maps;

[0007] Perform target recognition of different sizes on each fused feature map to obtain the recognition results in the SAR image;

[0008] Wherein, the multi-head self-attention module is used to:

[0009] For each window input , define the feature matrix , , , , , , is a learnable weight matrix;

[0010] For the feature matrix Perform convolution with a convolution kernel of 3×3 to obtain the matrix ;

[0011] Pair Matrix Perform two convolutions with a kernel of 1×1 to obtain the attention matrix :

[0012]

[0013] in, is the convolutional layer weight matrix with ReLU activation function, is the weight matrix of the convolution layer without activation function;

[0014] The feature matrix With the attention matrix Multiply to get the dynamic context representation matrix :

[0015] ;

[0016] The matrix and matrix Add and output the feature map through the ReLU activation function :

[0017] .

[0018] Furthermore, the Transformer backbone network module includes four input-output connected feature extraction blocks in sequence. , k=1,2,3,4, where the feature extraction block Includes 1 encoding unit and 1 attention unit, feature extraction block Output feature map ; The feature map Input feature extraction block , feature extraction block Includes 1 merge unit and 1 attention unit, output feature map ; The feature map Input feature extraction block , feature extraction block Includes 1 merge unit and 3 attention units to output feature maps ; The feature map Input feature extraction block , Feature Extraction Block Includes 1 merge unit and 1 attention unit, output feature map ;

[0019] The encoding unit uses a convolution kernel of size 1×1×32 to expand the feature map to 56×56×32;

[0020] The expanded feature map is divided into 8×8 attention windows of size 7×7×32, and the following formula is used to extract features in each attention window:

[0021] ;

[0022] ;

[0023] in Input feature map to the attention unit, is the output feature map of the attention unit, LN is layer normalization, MLP is a multi-layer fully connected module, MSA is the multi-head self-attention module, The feature map output by the multi-head self-attention module;

[0024] The merging unit is used to: use alternate point sampling to decompose the feature map into 4 sub-feature maps with half the length and width and the same number of channels, and splice them into a feature map with half the length and width and four times the number of channels; then use a 1×1 convolution kernel to perform convolution to halve the number of channels, and finally output a feature map with half the length and width of the input and twice the number of channels.

[0025] Furthermore, the multi-resolution feature maps are subjected to resolution cross-feature fusion through upsampling and the same multi-head self-attention module, including:

[0026] For feature maps Upsample to get feature map , the feature map Input the same attention unit for attention calculation and compare it with the feature map Fusion obtains fusion feature map , where the feature matrix in the multi-head self-attention module is , , , is the weight matrix;

[0027] For feature maps Upsample to get feature map , the feature map Input the same attention unit for attention calculation and compare it with the feature map Fusion obtains fusion feature map , where the feature matrix in the multi-head self-attention module is , , , is the weight matrix;

[0028] Feature Map Directly output as fused feature map .

[0029] Furthermore, the target recognition of different sizes is performed on each fused feature map to obtain a SAR image recognition result, including:

[0030] The fused feature map , , Three detection heads are input respectively, and a fully connected network is used to identify large targets, medium targets, and small targets respectively. The training loss is:

[0031]

[0032] in is the true category of the sample, To predict whether there is a real target in the box, is the probability of a certain type of target existing in the prediction box, is the probability of whether there is a target in the prediction box, is the position parameter of the prediction box, Includes: target center coordinates of the prediction box , , width and height of the annotation box , and rotation angle Five parameters; is the position parameter of the real box, Includes: target center coordinates of the real frame , , width and height of the annotation box , and rotation angle Five parameters; , , is the weight parameter, , The target center coordinates are included , , width and height of the annotation box , and rotation angle Five parameters; is the target confidence loss, is the target category loss.

[0033] Furthermore, the target confidence loss and target category loss Both use binary cross entropy loss:

[0034]

[0035] in, , They are , The intermediate variable after the sigmoid function processing, the target positioning loss Adopt a design similar to CIOU loss:

[0036]

[0037] in,

[0038] in, is the distance between the center of the predicted box and the center of the real box, is the diagonal distance between the predicted box and the minimum external box of the real box, and IOU is the area ratio of the intersection and union of the predicted box and the real box; , are the position parameters of the prediction box respectively Middle frame width , Frame height ; , are the position parameters of the real frame respectively Middle frame width , Frame height ;

[0039] Furthermore, it also includes:

[0040] When performing detection reasoning, the detection head The size of determines whether the detection box retains the prediction box: if If it is greater than the threshold, it is retained. If it is smaller than the threshold, it will not be retained.

[0041] Furthermore, it also includes:

[0042] Improved non-maximum suppression is used for the output of multiple detection heads to solve the problem of multiple detections of the same image, and only the The highest true box or predicted box, the calculation formula of the non-maximum value is:

[0043] .

[0044] The beneficial effects of the present invention are as follows: We propose a SAR image detection method based on a contextual attention mechanism, which can quickly and accurately identify aircraft and ships in SAR images while reducing the time consumption of training models. First, the SAR image target detection backbone network based on the Transformer architecture of the present invention benefits from the superiority of the Transformer. The model can better extract global features, and the feature extraction block enhances the small target detection capability of the model; it is optimized for SAR images to make the model parameters less and the training speed faster; it uses a cross-resolution attention enhancement detection method to make the model have better adaptability to multi-scale targets. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0046] Figure 1 It is a flowchart of a SAR image detection method based on a contextual attention mechanism provided by an embodiment of the present invention;

[0047] Figure 2 is a processing flow chart of various modules in an embodiment of the present invention;

[0048] Figure 3 It is a schematic diagram of a multi-head self-attention calculation method provided by an embodiment of the present invention;

[0049] Figure 4 is a flowchart of a method for image processing and data enhancement provided by an embodiment of the present invention;

[0050] Figure 5 It is a schematic diagram of a Transformer backbone network structure for SAR image feature extraction provided by an embodiment of the invention;

[0051] Figure 6 It is a schematic diagram of an attention unit structure provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the technical solutions in the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0053] A SAR image detection method based on a contextual attention mechanism according to an embodiment of the present invention is described below with reference to the accompanying drawings.

[0054] Figure 1 A flowchart of a SAR image detection method based on a contextual attention mechanism provided by an embodiment of the present invention is provided. The method comprises the following steps:

[0055] S110, obtaining the processed SAR image, inputting it into the Transformer backbone network module based on the multi-head self-attention module, and extracting multi-resolution feature maps.

[0056] S120, performing cross-resolution feature fusion on the multi-resolution feature maps through upsampling and the same multi-head self-attention module to obtain multiple fused feature maps.

[0057] S130, performing target recognition of different sizes on each fused feature map to obtain a SAR image recognition result.

[0058] like Figure 2 As shown, the backbone model corresponding to the detection method provided by the present invention uses a Transformer structure, which is similar to the pyramid hierarchical structure of CNN to enhance the small target detection capability of the model, and redesigns the self-attention module so that it can combine contextual information; cross-resolution feature fusion is performed for small targets in large scenes that are prone to lose information during the downsampling process.

[0059] In this embodiment, in order to better learn context information, the present invention designs a new attention calculation method, such as Figure 3 As shown, the multi-head self-attention module is used to:

[0060] For each window input , define the feature matrix (Query), (Key), (Value), , , , is a learnable weight matrix;

[0061] For the feature matrix Perform convolution with a convolution kernel of 3×3 to obtain the matrix ;

[0062] Pair Matrix Perform two convolutions with a kernel of 1×1 to obtain the attention matrix :

[0063]

[0064] in, is the convolutional layer weight matrix with ReLU activation function, is the weight matrix of the convolution layer without activation function;

[0065] The feature matrix With the attention matrix Multiply to get the dynamic context representation matrix :

[0066] ;

[0067] The matrix and matrix Add and output the feature map through the ReLU activation function :

[0068] .

[0069] Optionally, as an embodiment of the present invention, regarding the image acquisition and data enhancement part of the present invention, the following method is provided, such as Figure 4 As shown:

[0070] The original SAR image is obtained by a SAR device carried by a satellite or aircraft, and is saved and backed up. In this embodiment, the SAR image is divided into a single-channel image with a resolution of 224×224, and contrast enhancement is performed. Each single-channel image is labeled with a target for training and testing. According to the position of the target in the image, the target is translated and rotated at a small angle of no more than 5° without exceeding the boundary of the image, and then saved and added to training and testing. 30% of the images are randomly selected to add Gaussian noise, which is saved and added to training and testing. Through this method, the SAR image is processed into a format that is easy to train, providing training samples for the SAR image detection method based on contextual joint representation learning provided by the present invention, and the amount of training data is expanded through contrast, rotation, and noise processing.

[0071] Optionally, as an embodiment of the present invention, the embodiment of the present invention adopts a Transformer backbone network module combined with context information to extract SAR image features using a relatively small number of parameters while making the features include context information. The model structure of the feature extraction part is as follows: Figure 5 As shown, the Transformer backbone network module includes four input-output connected feature extraction blocks in sequence. , k=1,2,3,4, where the feature extraction block Includes 1 encoding unit and 1 attention unit, feature extraction block Output feature map ; The feature map Input feature extraction block , feature extraction block Includes 1 merge unit and 1 attention unit, output feature map ; The feature map Input feature extraction block , feature extraction block Includes 1 merge unit and 3 attention units to output feature maps ; The feature map Input feature extraction block , Feature Extraction Block Includes 1 merge unit and 1 attention unit, output feature map .

[0072] Specifically, the processed single-channel image of size 224×224 is first combined into a 56×56×16 tensor in units of 4×4 pixel blocks and input into the feature extraction block. , start the operation of the corresponding unit, including:

[0073] In the encoding unit, a convolution kernel of size 1×1×32 is used to expand the feature to 56×56×32;

[0074] Then it is divided into 8×8 attention windows of size 7×7×32, and the attention unit is used to extract its features in each window. The process of attention unit feature extraction is a new attention calculation method designed by the present invention, which can better learn context information. The structure is as follows Figure 6 As shown, including:

[0075] The expanded feature map is divided into 8×8 attention windows of size 7×7×32, and the following formula is used to extract features in each attention window:

[0076] ;

[0077] ;

[0078] in Input feature map to the attention unit, is the output feature map of the attention unit, LN is layer normalization, MLP is a multi-layer fully connected module, MSA is the multi-head self-attention module, It is the feature map output by the multi-head self-attention module.

[0079] The merging unit is used to: use alternate point sampling to decompose the feature map into 4 sub-feature maps with half the length and width and the same number of channels, and splice them into a feature map with half the length and width and four times the number of channels; then use a 1×1 convolution kernel to perform convolution to halve the number of channels, and finally output a feature map with half the length and width of the input and twice the number of channels.

[0080] The final output size is 56×56×32 feature map, denoted as , similarly, The final output size is 28×28×64 feature map , Output feature map of size 14×14×128 , Output feature map of size 7×7×256 .

[0081] Optionally, as an embodiment of the present invention, the present invention performs attention enhancement by upsampling and adding attention connection context information, and fuses shallow and deep feature map information across resolutions.

[0082] Optionally, as an embodiment of the present invention, the step of performing resolution cross-feature fusion on the multi-resolution feature maps through upsampling and the same multi-head self-attention module includes:

[0083] For feature maps Upsample to get feature map , the feature map Input the same attention unit for attention calculation and compare it with the feature map Fusion obtains fusion feature map , where the feature matrix in the multi-head self-attention module is , , , is the weight matrix;

[0084] For feature maps Upsample to get feature map , the feature map Input the same attention unit for attention calculation and compare it with the feature map Fusion obtains fusion feature map , where the feature matrix in the multi-head self-attention module is , , , is the weight matrix;

[0085] Feature Map The direct output is a fused feature map.

[0086] So far, we have obtained the fusion feature map , , .

[0087] The cross-resolution attention enhancement method provided in the embodiment of the present invention adopts cross-resolution cross-fusion features to promote the model to better fuse shallow and deep feature map information and extract the features of small targets, ultimately achieving target positioning, targeting small targets in large scenes, as well as the strong scattering characteristics of SAR imaging and the low distinction between targets and backgrounds.

[0088] Optionally, as an embodiment of the present invention, the step of performing target recognition of different sizes on each fused feature map to obtain a SAR image recognition result includes:

[0089] The fused feature map , , Three detection heads are input respectively, and a fully connected network is used to identify large targets, medium targets, and small targets respectively. The training loss is:

[0090]

[0091] in is the true category of the sample, To predict whether there is a real target in the box, is the probability of a certain type of target existing in the prediction box, is the probability of whether there is a target in the prediction box, is the position parameter of the prediction box, Includes: target center coordinates of the prediction box , , width and height of the annotation box , and rotation angle Five parameters; is the position parameter of the real box, Includes: target center coordinates of the real frame , , width and height of the annotation box , and rotation angle Five parameters; , , is the weight parameter, , The target center coordinates are included , , width and height of the annotation box , and rotation angle Five parameters; is the target confidence loss, is the target category loss.

[0092] The detection result of the embodiment of the present invention is an aircraft and ship target detection result, wherein the detection result includes a category, an object confidence, and a position parameter of a box.

[0093] Optionally, as an embodiment of the present invention, the target confidence loss and target category loss Both use binary cross entropy loss:

[0094]

[0095] in, , They are , The intermediate variable after the sigmoid function processing, the target positioning loss Adopt a design similar to CIOU loss:

[0096]

[0097] in,

[0098] in, is the distance between the center of the predicted box and the center of the real box, is the diagonal distance between the predicted box and the minimum external box of the real box, and IOU is the area ratio of the intersection and union of the predicted box and the real box; , are the position parameters of the prediction box respectively Middle frame width , Frame height ; , are the position parameters of the real frame respectively Middle frame width , Frame height .

[0099] Optionally, as an embodiment of the present invention, it further includes: when performing detection reasoning, the detection head The size of determines whether the detection box retains the prediction box: if If it is greater than the threshold, it is retained. If it is smaller than the threshold, it will not be retained.

[0100] Optionally, as an embodiment of the present invention, it further comprises: using improved non-maximum suppression for the outputs of multiple detection heads to solve the problem of multiple detections of the same image, and only retaining The highest true box or predicted box, the calculation formula of the non-maximum value is: .

[0101] Although the present invention has been described in detail by referring to the accompanying drawings and in combination with the preferred embodiments, the present invention is not limited thereto. Without departing from the spirit and essence of the present invention, a person of ordinary skill in the art may make various equivalent modifications or substitutions to the embodiments of the present invention, and these modifications or substitutions shall be within the scope of the present invention or any person skilled in the art may easily think of changes or substitutions within the technical scope disclosed by the present invention, and shall be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A SAR image detection method based on contextual attention mechanism, characterized in that: include: Obtain the processed SAR image and input it into the Transformer backbone network module based on the multi-head self-attention module to extract multi-resolution feature maps; The multi-resolution feature maps are subjected to cross-resolution feature fusion by upsampling and the same multi-head self-attention module to obtain multiple fused feature maps; Perform target recognition of different sizes on each fused feature map to obtain the recognition results in the SAR image; Wherein, the multi-head self-attention module is used to: For each window input , define the feature matrix , , , , , , is a learnable weight matrix; For the feature matrix Perform convolution with a convolution kernel of 3×3 to obtain the matrix ; Pair Matrix Perform two convolutions with a kernel of 1×1 to obtain the attention matrix : ； in, is the convolutional layer weight matrix with ReLU activation function, is the weight matrix of the convolution layer without activation function; The feature matrix With the attention matrix Multiply to get the dynamic context representation matrix : ； The matrix and matrix Add and output the feature map through the ReLU activation function : ； The Transformer backbone network module includes four input-output connected feature extraction blocks in sequence , k=1,2,3,4, where the feature extraction block Includes 1 encoding unit and 1 attention unit, feature extraction block Output feature map ; The feature map Input feature extraction block , feature extraction block Includes 1 merge unit and 1 attention unit, output feature map ; The feature map Input feature extraction block , feature extraction block Includes 1 merge unit and 3 attention units to output feature maps ; The feature map Input feature extraction block , Feature Extraction Block Includes 1 merge unit and 1 attention unit, output feature map ; The encoding unit uses a convolution kernel of size 1×1×32 to expand the feature map to 56×56×32; The expanded feature map is divided into 8×8 attention windows of size 7×7×32, and the following formula is used to extract features in each attention window: ； ； in Input feature map to the attention unit, is the output feature map of the attention unit, LN is layer normalization, MLP is a multi-layer fully connected module, MSA is the multi-head self-attention module, The feature map output by the multi-head self-attention module; The merging unit is used to: decompose the feature map into four sub-feature maps with half the length and width and the same number of channels by using alternate point sampling, and splice them into a feature map with half the length and width and four times the number of channels; then use a 1×1 convolution kernel to perform convolution to halve the number of channels, and finally output a feature map with half the length and width of the input and twice the number of channels; The multi-resolution feature map is subjected to resolution cross-feature fusion through upsampling and the same multi-head self-attention module, including: For feature maps Upsample to get feature map , the feature map Input the same attention unit for attention calculation and compare it with the feature map Fusion obtains fusion feature map , where the feature matrix in the multi-head self-attention module is , , , is the weight matrix; For feature maps Upsample to get feature map , the feature map Input the same attention unit for attention calculation and compare it with the feature map Fusion obtains fusion feature map , where the feature matrix in the multi-head self-attention module is , , , is the weight matrix; Feature Map Directly output as fused feature map .

2. The method according to claim 1, characterized in that: The target recognition of different sizes is performed on each fusion feature map to obtain the SAR image recognition result, including: The fused feature map , , Three detection heads are input respectively, and a fully connected network is used to identify large targets, medium targets, and small targets respectively. The training loss is: ； in is the true category of the sample, To predict whether there is a real target in the box, is the probability of a certain type of target existing in the prediction box, is the probability of whether there is a target in the prediction box, is the position parameter of the prediction box, Includes: target center coordinates of the prediction box , , width and height of the annotation box , and rotation angle Five parameters; is the position parameter of the real box, Includes: target center coordinates of the real frame , , width and height of the annotation box , and rotation angle Five parameters; , , is the weight parameter, , The target center coordinates are included , , width and height of the annotation box , and rotation angle Five parameters; is the target confidence loss, is the target category loss.

3. The method according to claim 2, characterized in that The target confidence loss and target category loss Both use binary cross entropy loss: ； in, , They are , The intermediate variable after the sigmoid function processing, the target positioning loss Adopt a design similar to CIOU loss: in, ; in, is the distance between the center of the predicted box and the center of the real box, is the diagonal distance between the predicted box and the minimum external box of the real box, and IOU is the area ratio of the intersection and union of the predicted box and the real box; , are the position parameters of the prediction box respectively Middle frame width , Frame height ; , are the position parameters of the real frame respectively Middle frame width , Frame height .

4. The method according to claim 3, characterized in that Also includes: When performing detection reasoning, the detection head The size of determines whether the detection box retains the prediction box: if If it is greater than the threshold, it is retained. If it is smaller than the threshold, it will not be retained.

5. The method according to claim 3, characterized in that: Also includes: Improved non-maximum suppression is used for the output of multiple detection heads to solve the problem of multiple detections of the same image, and only the The highest true box or predicted box, the calculation formula of the non-maximum value is: 。

Images