SAR image ship detection method and device, and training method

Abstract

The invention provides an SAR image ship detection method and device and a training method. The SAR image ship detection method comprises the following steps: performing data preprocessing on an acquired single-channel SAR image of a ship to be detected to obtain a multi-channel SAR image; performing feature extraction on the multi-channel SAR image to obtain a first shallow layer feature map and feature maps of levels outside the shallow layer; coordinate attention operation is carried out on the first superficial layer feature map, a two-dimensional space attention map of the first superficial layer feature map is generated, and a second superficial layer feature map of direction and position perception of the multi-channel SAR image is obtained based on the two-dimensional space attention map; performing fusion processing on the second shallow layer feature map and the feature map of the levels outside the shallow layer to obtain a multi-level fusion feature map; and based on a preset anchor frame, performing prediction and screening processing on the multi-level fusion feature map to obtain a rotating frame detection result of the to-be-detected ship.

Description

Technical Field

[0001] The present invention relates to the technical field of information perception and recognition, and in particular to a SAR image ship detection method and device, and a training method. Background Art

[0002] In coastal production, life and economic activities, achieving all-weather, high-precision and real-time ship target detection is crucial for the rapid deployment of resources, which is not only related to maintaining national marine security, but also plays a key role in promoting civilian production activities. However, restricted by natural factors such as clouds, rain, and light, optical remote sensing images cannot be monitored around the clock. Compared with traditional optical imaging technology, synthetic aperture radar (SAR) microwave imaging technology has attracted much attention due to its all-weather, all-time and anti-interference detection capabilities. With the continuous development of microwave remote sensing technology, many satellites (such as Nuwa, Haisi, and Qilu) are also equipped with SAR sensors, and the resolution of SAR images has been significantly improved. Therefore, how to use SAR images to achieve all-weather, high-precision, and real-time ship target detection has become an urgent problem to be solved. Summary of the invention

[0003] In view of this, the present invention provides a SAR image ship detection method and device, and a training method, which are used to at least partially solve the above technical problems.

[0004] One aspect of the present invention provides a SAR image ship detection method, comprising: performing data preprocessing on an acquired single-channel SAR image of a ship to be detected to obtain a multi-channel SAR image; performing feature extraction on the multi-channel SAR image to obtain a first shallow feature map and a feature map of a layer other than the shallow layer; performing a coordinate attention operation on the first shallow feature map to generate a two-dimensional spatial attention map of the first shallow feature map, and obtaining a second shallow feature map of the multi-channel SAR image for direction and position perception based on the two-dimensional spatial attention map; performing a fusion process on the second shallow feature map and the feature map of the layer other than the shallow layer to obtain a multi-level fused feature map; based on a preset anchor frame, performing a prediction and screening process on the multi-level fused feature map to obtain a rotation frame detection result of the ship to be detected.

[0005] According to an embodiment of the present invention, feature extraction is performed on a multi-channel SAR image to obtain a first shallow feature map and feature maps of layers other than the shallow layer, including: gradually converting original pixel information of the multi-channel SAR image into feature representations of different layers to obtain the first shallow feature map and feature maps of layers other than the shallow layer.

[0006] According to an embodiment of the present invention, feature maps of layers other than the shallow layer include middle-layer feature maps and deep-layer feature maps; the original pixel information of the multi-channel SAR image is gradually converted into feature representations of different layers to obtain a first shallow feature map and feature maps of layers other than the shallow layer, including: performing multiple convolution, pooling and feature aggregation processing on the multi-channel SAR image to obtain a first shallow feature map; performing pooling and feature aggregation processing on the first shallow feature map to obtain a middle-layer feature map; performing pooling and feature aggregation processing on the middle-layer feature map to obtain a deep feature map.

[0007] According to an embodiment of the present invention, a multi-level fusion feature map includes a shallow fusion feature map, a middle fusion feature map and a deep fusion feature map; a second shallow feature map and feature maps of layers other than the shallow layer are fused to obtain a multi-level fusion feature map, including: based on spatial pyramid pooling and a cross-stage partial connection network, the deep feature map is feature enhanced; the middle feature map and the feature-enhanced deep feature map are weightedly fused to the second shallow feature map to obtain a shallow fusion feature map; the second shallow feature map and the feature-enhanced deep feature map are weightedly fused to the middle feature map to obtain a middle fusion feature map; the second shallow feature map and the middle feature map are weightedly fused to the feature-enhanced deep feature map to obtain a deep fusion feature map.

[0008] According to an embodiment of the present invention, a coordinate attention operation is performed on a first shallow feature map to generate a two-dimensional spatial attention map of the first shallow feature map, and a second shallow feature map for direction and position perception of the SAR image is obtained based on the two-dimensional spatial attention map, including: encoding and compressing the first shallow feature map in the horizontal and vertical directions to obtain a two-dimensional spatial attention map; and performing a weighted summation of the two-dimensional spatial attention map and the first shallow feature map to obtain a second shallow feature map for direction and position perception.

[0009] According to an embodiment of the present invention, data preprocessing is performed on the acquired single-channel SAR image of the ship to be detected to obtain a multi-channel SAR image, including: performing band normalization, radiation correction, multi-viewing, filtering, terrain correction and channel expansion processing on the single-channel SAR image in sequence to obtain the multi-channel SAR image.

[0010] According to an embodiment of the present invention, based on a preset anchor frame, a multi-level fusion feature map is predicted and screened to obtain a rotation frame detection result of a ship to be detected, including: based on the preset anchor frame, determining a prediction frame on the fusion feature map of each level in the multi-level fusion feature map; according to the comparison relationship between the preset anchor frame and the prediction frame, by decoding the relative coordinates and applying the category probability threshold, screening out the prediction frame that exceeds a preset confidence; using a non-maximum suppression method to remove overlapping frames in the prediction frame to obtain a rotation frame detection result.

[0011] Another aspect of the present invention provides a SAR image ship detection device, characterized in that it includes: a preprocessing module, which is used to perform data preprocessing on the acquired single-channel SAR image of the ship to be detected to obtain a multi-channel SAR image; an operation model, which is used to extract features from the multi-channel SAR image to obtain a first shallow feature map and feature maps of layers other than the shallow layer, perform coordinate attention operation on the first shallow feature map to generate a two-dimensional spatial attention map of the first shallow feature map, obtain a second shallow feature map for direction and position perception of the SAR image based on the two-dimensional spatial attention map, and fuse the second shallow feature map and the feature maps of layers other than the shallow layer to obtain a multi-level fused feature map; a detection module, which is used to predict and screen the multi-level feature maps based on a preset anchor frame to obtain a rotation frame detection result of the ship to be detected.

[0012] According to an embodiment of the present invention, a computing model is constructed based on a YOLOv7 model, a basic convolution module is configured in the convolution layer of the YOLOv7 model, and the output end of the basic convolution module is connected to a coordinate attention module. The coordinate attention module is used to perform a coordinate attention operation on a first shallow feature map of a multi-channel SAR image, generate a two-dimensional spatial attention map of the first shallow feature map, and obtain a second shallow feature map for direction and position perception of the SAR image based on the two-dimensional spatial attention map.

[0013] Another aspect of the present invention provides a training method for training the computing model of the above-mentioned SAR image ship detection device, including: determining a training data set and a verification data set for the ship to be detected in different scenarios based on a specific data set of the SAR image ship; using a specific loss function and a training data set to perform weight training on the computing model, wherein the specific loss function includes a Kalman filter intersection-over-combination loss function; and verifying the training results of the computing model through a verification data set.

[0014] According to the SAR image ship detection method, device and training method of the embodiments of the present invention, in the process of detecting and identifying SAR image ship targets, by introducing coordinate attention operation into shallow feature processing, a direction-aware and position-sensitive shallow feature map is obtained through a two-dimensional spatial attention map. Since the small target information is mainly in the shallow network and the shallow network is rich in information, the coordinate attention operation is performed on the shallow features and then the feature fusion processing is performed, which can significantly improve the detection precision and accuracy of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0016] Figure 1A flow chart of a SAR image ship detection method according to an embodiment of the present invention is schematically shown;

[0017] Figure 2 A structural block diagram of a SAR image ship detection device according to an embodiment of the present invention is schematically shown;

[0018] Figure 3 A schematic diagram of a structure of a computing model according to an embodiment of the present invention is shown;

[0019] Figure 4 A schematic diagram of a structural block diagram of a backbone network of a computing model according to an embodiment of the present invention is shown;

[0020] Figure 5 A block diagram schematically shows a structure of a head network of a computing model according to an embodiment of the present invention;

[0021] Figure 6 A flowchart of a method for training a computing model according to an embodiment of the present invention is schematically shown;

[0022] Figure 7 Schematically showing the detection results of the computing model in the far-shore area according to the embodiment of the present invention;

[0023] Figure 8 Schematically showing the detection results of the computing model in the nearshore area according to the embodiment of the present invention;

[0024] Fig. 9 A diagram schematically showing a comparison of detection results of a computing model according to an embodiment of the present invention and a comparison model in a far-shore area;

[0025] Fig.10 A diagram schematically showing a comparison of detection results of a computing model according to an embodiment of the present invention and a comparison model in a nearshore area;

[0026] Fig.11 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0027] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0028] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0029] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0030] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0031] In the process of realizing the present invention, the applicant found that the detection methods based on SAR images are often divided into two categories: based on traditional algorithms and based on deep learning algorithms. Traditional algorithms include constant false alarm rate (CFAR) algorithms. Such algorithms have poor detection accuracy and require manual clutter modeling, and have poor generalization ability. Currently, such methods are rarely used for detection. With the rise and development of artificial intelligence, deep learning algorithms have become the mainstream algorithms in target detection tasks. Commonly used target detection algorithms include regional convolutional neural networks (RCNN), You Only Look Once (YOLO), single shot multi-frame detectors (SSD), etc. However, in actual scenarios, they cannot accurately locate and classify ships, and the models are complex and difficult to deploy.

[0032] In view of this, an embodiment of the present invention provides a SAR image ship detection method, which uses a position attention mechanism to efficiently utilize the position information of the image, captures the characteristics of long-range dependencies in one dimension, and retains detailed position information in another dimension, and generates a pair of direction-aware and position-sensitive attention maps. These feature maps can be more effectively applied to input data, thereby improving the characterization ability of the object of interest and identifying and locating the ship category. The following is a detailed description in conjunction with specific embodiments.

[0033] Figure 1 The flowchart of the SAR image ship detection method according to an embodiment of the present invention is schematically shown.

[0034] like Figure 1As shown, in this embodiment, the SAR image ship detection method may include steps S110 to S150.

[0035] In step S110, data preprocessing is performed on the acquired single-channel SAR image of the ship to be detected to obtain a multi-channel SAR image.

[0036] In step S120, feature extraction is performed on the multi-channel SAR image to obtain a first shallow feature map and feature maps of layers other than the shallow layer.

[0037] In step S130, a coordinate attention operation is performed on the first shallow feature map to generate a two-dimensional spatial attention map of the first shallow feature map, and a second shallow feature map of the multi-channel SAR image that is aware of direction and position is obtained based on the two-dimensional spatial attention map.

[0038] This embodiment processes the generated first shallow feature map based on the coordinate attention mechanism, and obtains the second shallow feature map with direction and position perception through the two-dimensional spatial attention map. On the one hand, because the information of small targets is mainly in the shallow network, and on the other hand, because the shallow network is rich in information, the accuracy of the model in detecting small targets is significantly improved after adding the position attention mechanism, which will be explained in detail in experiments later.

[0039] In step S140, the second shallow feature map and the feature maps of the layers other than the shallow layer are fused to obtain a multi-level fused feature map.

[0040] In step S150, based on the preset anchor frame, the multi-level fusion feature map is predicted and screened to obtain the rotation frame detection result of the ship to be detected.

[0041] In step S110, due to the coherence of radar signals, different imaging conditions (such as incident angle, polarization mode, etc.), and other factors, SAR images often contain speckle noise and imaging errors. These noises and errors can be effectively removed by preprocessing. In some embodiments, preprocessing may include but is not limited to band normalization, radiation correction, multi-view, filtering, terrain correction, and channel expansion processing to improve the signal-to-noise ratio and imaging quality of the image. In order to facilitate model detection, a single-channel SAR image can be expanded to three channels (such as HH, HV, VV polarization modes), and detected after slicing. Then, step S120 can be executed.

[0042] In step S120, in the initial stage of feature extraction after preprocessing, for example, the convolutional layer of the convolutional neural network (CNN) can be used to perform preliminary processing on the preprocessed SAR image to extract shallow features of the image, such as edges, textures, etc., to form a first shallow feature map. As the depth of the network increases, the convolutional neural network can gradually capture higher-level semantic information, such as the shape, structure, and position of the target, to form feature maps beyond the shallow layer. In some embodiments, the convolutional neural network (CNN) can be a model built based on YOLOv7. The specific content will be further introduced later.

[0043] In some embodiments, feature maps of layers other than the shallow layer may include mid-level feature maps and deep-level feature maps. The mid-level feature map may contain some features of medium complexity, which can reflect the basic structure of the image and have a certain degree of abstraction, which helps to understand and analyze the image content in subsequent processing. The deep feature map may include the most abstract and advanced semantic information in the image, usually contains enough information to distinguish different image categories or targets, and has strong robustness to interference factors such as noise and illumination changes in the image.

[0044] Please continue reading Figure 1 In some embodiments, step S120 may include: gradually converting the original pixel information of the multi-channel SAR image into feature representations of different levels to obtain a first shallow feature map and feature maps of levels other than the shallow layer.

[0045] Furthermore, when obtaining the first shallow feature map, the middle feature map and the deep feature map, step S120 may also include: performing multiple convolution, pooling and feature aggregation processing on the multi-channel SAR image to obtain the first shallow feature map; performing pooling and feature aggregation processing on the first shallow feature map to obtain the middle feature map; performing pooling and feature aggregation processing on the middle feature map to obtain the deep feature map.

[0046] Convolution operations help extract local features of images, such as edges and corners. The pooling layer reduces the dimension of the feature map through pooling operations (such as maximum pooling, average pooling, etc.), while retaining important features and improving the translation invariance and robustness of the feature map. In the shallow stage, feature aggregation can be the addition or concatenation of feature maps of different layers or branches to integrate information at multiple scales or different perspectives. After obtaining the first shallow feature map, it can be downsampled through the pooling layer to further reduce the size of the feature map and enhance the robustness of the features. Feature aggregation is performed again to combine broader contextual information or features of different scales to construct a richer mid-level feature representation. After obtaining the mid-level feature map, it can be passed through the pooling layer again for further dimensionality reduction and feature abstraction, and then feature aggregation is performed to make full use of multi-scale features to construct a highly abstract deep feature map.

[0047] Through the progressively deepening convolution, pooling and feature aggregation operations, shallower and deeper feature representations are gradually extracted from the input multi-channel SAR image. The shallow feature map mainly focuses on the local details and edge information of the image, the middle feature map integrates more extensive context information, and the deep feature map contains highly abstract and discriminative features. Then, step S130 can be executed.

[0048] In some embodiments, step S130 may include: encoding and compressing the first shallow feature map in the horizontal and vertical directions to obtain a two-dimensional spatial attention map; weighted summing the two-dimensional spatial attention map and the first shallow feature map to obtain a second shallow feature map that is aware of direction and position.

[0049] Encoding can be used to capture feature changes in the horizontal and vertical directions of an image, for example, through convolution operations. After obtaining the encoded features in the horizontal and vertical directions, these features can be compressed into a two-dimensional spatial attention map. The two-dimensional spatial attention map can be regarded as a weight map, in which the weight of each position represents the importance of the corresponding position in the original feature map. This weight map is element-by-element multiplied (i.e., weighted summation) with the first shallow feature map. The feature map after weighted summation, i.e., the second shallow feature map, not only contains the information in the original feature map, but also incorporates the attention information of direction and position. This enables the feature map to focus more on areas that are useful for tasks (such as target detection, classification, etc.) in subsequent processing. Next, step S140 can be executed.

[0050] See also Figure 1 In some embodiments, the multi-level fusion feature map includes a shallow fusion feature map, a middle fusion feature map, and a deep fusion feature map. Step S140 may include: based on spatial pyramid pooling (SPP) and a cross-stage partial connection network, feature enhancement is performed on the deep feature map; weighted fusion of the middle feature map and the feature-enhanced deep feature map to the second shallow feature map to obtain a shallow fusion feature map; weighted fusion of the second shallow feature map and the feature-enhanced deep feature map to the middle feature map to obtain a middle fusion feature map; weighted fusion of the second shallow feature map and the middle feature map to the feature-enhanced deep feature map to obtain a deep fusion feature map.

[0051] In this embodiment, the spatial pyramid pooling and the cross-stage partial connection network are combined together, which can reduce the amount of calculation by half, thereby improving the speed while maintaining high accuracy. Weighted fusion refers to the process of superimposing feature maps of different levels according to certain weight coefficients. These weight coefficients can be fixed or obtained through learning (such as prediction through additional convolutional layers or fully connected layers). Through multiple weighted fusions, the shallow feature map can obtain contextual information from deeper levels while retaining the detail information. At the same time, the representation ability of the middle-level feature map is enhanced so that it contains both shallow detail information and deep semantic information. In addition, the feature information of each level of the network can be integrated to the greatest extent to generate a more comprehensive and powerful deep fusion feature map with feature representation. Then, step S150 can be performed.

[0052] See also Figure 1 In some embodiments, step S150 may include: determining the prediction box on each level of the fused feature map in the multi-level fused feature map based on the preset anchor box; filtering out the prediction box that exceeds the preset confidence by decoding the relative coordinates and applying the category probability threshold according to the comparison relationship between the preset anchor box and the prediction box; removing the overlapping boxes in the prediction box using the non-maximum suppression method (NMS) to obtain the rotation box detection result. In this embodiment, these anchor boxes can be used as the initial position of the prediction to help the model locate the target. Setting a category probability threshold and only retaining those prediction boxes whose category probability exceeds the threshold can reduce false detection and improve detection accuracy. The rotation box detection result may include the location, size and category information of the target. The detection results can be used in practical applications, such as object recognition and tracking.

[0053] Based on the above SAR image ship detection method, the present invention also provides a SAR image ship detection device, which can be used to implement the SAR image ship detection method and has the same technical features and beneficial effects as the SAR image ship detection method. Figure 2 The structural block diagram of the SAR image ship detection device according to an embodiment of the present invention is schematically shown.

[0054] like Figure 2 As shown, the SAR image ship detection device 200 may include a pre-processing module 210 , a calculation model 220 and a detection module 230 .

[0055] The preprocessing module 210 is used to perform data preprocessing on the acquired single-channel SAR image of the ship to be detected to obtain a multi-channel SAR image. In some embodiments, the preprocessing module 210 can be used to perform the above step S110.

[0056] The operation model 220 is used to extract features from the multi-channel SAR image, obtain a first shallow feature map and a feature map of a layer other than the shallow layer, perform a coordinate attention operation on the first shallow feature map, generate a two-dimensional spatial attention map of the first shallow feature map, obtain a second shallow feature map of the SAR image for direction and position perception based on the two-dimensional spatial attention map, and fuse the second shallow feature map and the feature map of the layer other than the shallow layer to obtain a multi-level fused feature map. In some embodiments, the operation model 220 can be used to execute the above steps S120 to S140.

[0057] The detection module 230 is used to predict and filter the multi-level feature graphs based on the preset anchor frame to obtain the rotation frame detection result of the ship to be detected. In some embodiments, the detection module 230 can be used to perform the above step S150.

[0058] The following, combined Figure 3~Figure 5 The processing process of the calculation model 220 is described. It should be noted that, Figure 3 The operation model 220 may be based on the YOLOv7 model. In some other possible embodiments, the operation model may also be constructed based on other convolutional neural network models having similar YOLOv7 functions. Figure 3 The structure block diagram of the operation model according to the embodiment of the present invention is schematically shown. Figure 4 The structure block diagram of the backbone network of the operation model according to the embodiment of the present invention is schematically shown. Figure 5 The structural block diagram of the head network of the computing model according to the embodiment of the present invention is schematically shown.

[0059] like Figure 3 As shown, the operation model 220 may include a head network and a backbone network. The head network is mainly used to extract multi-scale feature maps (such as the shallow feature map, the middle feature map and the deep feature map mentioned above) from the input multi-channel SAR image, and the backbone network is mainly used to fuse the multi-scale feature maps into hierarchical fusion feature maps (such as the shallow fusion feature map, the middle fusion feature map and the deep fusion feature map mentioned above).

[0060] like Figure 3 and Figure 4As shown in the figure, the backbone network consists of four main parts: the basic Convolutional-Batch Normalization-SiLU (CBS) module, the Efficient Layer Aggregation Networks (ELAN), the Coordinate Attention CBS (CA-CBS) module for shallow feature extraction, and the Max Pooling Conv (MP) module. Figure 4 As shown in the figure, the detailed structure of each module is shown. The CBS module can be composed of a convolutional layer (Conv), a batch normalization layer (BN), and a SiLU activation function layer (SiLUActivation Layer). Among them, "k" represents the number of convolution kernels, and "s" represents the stride during the convolution process. The CA module is based on the original CBS module and adds a CA (Coordinate Attention) attention mechanism to the tail end (i.e., the output end). The ELAN module represents an efficient network structure. By adjusting the shortest and longest gradient paths, this module enables the network to learn more features and enhance its robustness. The MP module includes two branches. One branch undergoes maximum pooling followed by a 1x1 convolution, while the other branch undergoes a 3x3 convolution for downsampling after the 1x1 convolution. Finally, the results of the two branches are added.

[0061] like Figure 3 As shown, for the input multi-channel SAR image , R is the real number domain, H is the image height, W is the image width, and C is the number of image channels. First, four convolutional layers are used to extract shallow features, where F i (i is the code of each layer) represents the feature map generated after each convolutional layer operation:

[0062]

[0063]

[0064]

[0065]

[0066] Among them, CBS 11represents a CBS module with a convolution kernel size of 1 and a stride of 1. CBS (·) represents convolution processing. For the extracted features, we can use the ELAN layer to the MP layer and then to the ELAN layer to get F7, which is equivalent to the first shallow feature map in the previous article, and then input it to the CA layer to get F B-1 , which is equivalent to the second shallow feature map in the previous text. The process is expressed as follows:

[0067]

[0068]

[0069] Among them, ELAN (·) indicates efficient convolution aggregation processing, and MP (·) indicates maximum pooling processing. B-1 represents the first-level feature map output by the backbone network. CA-CBS (·) represents the convolution processing for position attention operation. The internal structure of CA-CBS can be shown as follows: Figure 4 As shown, it can include 4 basic blocks, a convolutional layer, a batch normalization module, a SiLU activation function and a CA attention mechanism module. The input of CA-CBS is first extracted by the basic block and then copied. One copy is input to the CA module for processing. The processed features are then output to the subsequent head network for integration, and the other copy continues to be processed downward. The whole process can be expressed as follows:

[0070]

[0071]

[0072] Among them, F B-2 It represents the second-level feature map output by the backbone network, which is equivalent to the middle-level feature map mentioned above and is output to the head network for feature fusion.

[0073] like Figure 3 As shown, the backbone network continues to process the feature map downward:

[0074]

[0075] Among them, F B-3 It represents the third-level feature map output by the backbone network, which is equivalent to the deep feature map mentioned above.

[0076] In this embodiment, the backbone network is responsible for gradually extracting the features of the input image, from low-level edges and textures to high-level semantic features, providing rich feature representations for the subsequent detection head network. In the YOLOv7 network, a total of three scale feature maps F B-1 、F B-2 、F B-3The present invention embeds CA-CBS into shallow features, on the one hand because the information of small targets is mainly in the shallow network, and on the other hand because the shallow network is rich in information. After adding the position attention mechanism, the accuracy of the model in detecting small targets is significantly improved. These generated features are crucial for accurately locating and classifying target objects. They will participate in the subsequent head network processing, and can more comprehensively utilize the rich information of remote sensing images, so that the YOLO v7 network can complete the target detection task in a single forward propagation.

[0077] like Figure 3 and Figure 5 As shown, the head network receives F directly output by the backbone network. B-3 The deep semantic feature map is input into the Spatial Pyramid Pooling Cross Stage PartialConcatenate (SPPCSPC). This embodiment combines the two parts of the spatial pyramid pooling and the cross-stage partial network, which can reduce the amount of calculation by half, thereby improving the speed while maintaining high accuracy. The formula is as follows:

[0078]

[0079] Then the output F 13 It is divided into two parts, one part continues to output downward, and the other part is combined with the next layer of feature map that continues to be input:

[0080]

[0081] After an upsampling layer and the feature map of the backbone network are combined, upsampling is continued:

[0082]

[0083] Among them, Cat represents the combination operation, UP represents the upsampling operation (i.e., UP Conv in the figure), which is implemented by the nearest neighbor difference (resize (nearest)) algorithm. The feature map of the previous backbone network is combined with the feature map after upsampling and input into the next layer of operation:

[0084]

[0085] The result after input is as follows:

[0086]

[0087] Via Two CBS 21 After that, it is divided into two parts, one part is merged with the subsequent output, and the other part is further upsampled. B-1To perform a merge operation:

[0088]

[0089]

[0090] After combining, through ELAN and CBS 21 After processing, it is further divided into two parts. One part is down-sampled and fused with the previous feature map for extraction and output. The other part is directly passed through the REP module. The REP module can be divided into two parts: training and reasoning. In the training module, there are three branches, namely 3x3 convolution for feature extraction, 1x1 convolution for smoothing features, and Identity branch (also called residual connection) for directly transmitting features. Their outputs are added. In the reasoning module, it contains a 3x3 convolution obtained by re-parameterization of the training module. By converting the 1x1 convolution and Identity into 3x3 convolution, a matrix addition fusion process is performed, and finally the weights of the three branches are added to obtain a 3x3 convolution. The three branches in the reasoning module are fused into a line with a single 3x3 convolution structure. The weight of this structure is the superposition result of the three branches. After processing by the REP module, a convolution (Conv) operation is performed to output the detection result at this scale:

[0091]

[0092]

[0093] Among them, out1 represents the first-level fusion feature map of the output, which is equivalent to the shallow fusion feature map mentioned above.

[0094] The other branch continues downsampling, and the same as before F H-2 After the fusion operation, it is divided into two branches after another ELAN and convolution operation. One branch outputs the output result at this scale, and the other branch performs subsequent operations:

[0095]

[0096]

[0097]

[0098] Among them, out2 represents the second-level fusion feature map of the output, which is equivalent to the middle-level fusion feature map mentioned above.

[0099] The last one is the same as the F in the previous part after an ELAN and downsampling operation. H-1 Perform Cat operation, then REP operation and convolution operation to output the final result:

[0100]

[0101]

[0102]

[0103] Among them, out3 represents the third-level fusion feature map of the output, which is equivalent to the deep fusion feature map mentioned above.

[0104] Thus, through the operation model constructed based on YOLOv7, multi-level fusion feature maps out1, out2, and out3 are generated. Then, the output multi-level fusion feature maps out1, out2, and out3 of the three scales can be post-processed to obtain the final detection result. The post-processing process has been described in the aforementioned step S150, so it will not be repeated here.

[0105] Figure 6 The flowchart of the training method of the computing model according to the embodiment of the present invention is schematically shown.

[0106] like Figure 6 As shown, in this embodiment, the training method may include steps S610-630.

[0107] In step S610, a training data set and a verification data set under different scenarios of the ship to be detected are determined according to a specific data set of the ship in the SAR image.

[0108] In step S620, a specific loss function and a training data set are used to perform weight training on the operation model, wherein the specific loss function includes a Kalman filter intersection-over-intersection loss function;

[0109] In step S630, the training result of the computational model is verified by using a verification data set.

[0110] As an example, a specific dataset can be the GF-3 image data of the Rotated Ship Detection Dataset in SAR Images (RSDD). Specifically, the RSDD dataset can be randomly divided to first distinguish between nearshore and farshore scenes. Then in each scene, a part (such as 90%) of the data is used for training, and another part (such as 10%) of the data is used for verification, and the corresponding model is trained for each scene. The training uses the tools provided by YOLOv7, and the loss function uses the Kalman filtering Intersection over Union (KFiou) and the CA attention mechanism. In the detection of rotated objects, the traditional IoU loss function is difficult to handle the rotation box because the calculation of the overlapping area of ​​the rotation box is complicated and unfriendly to the gradient calculation. KFiou models the object as a Gaussian distribution and uses the Kalman filter to simulate the definition mechanism of SkewIoU, so that it is aligned with SkewIoU in terms of trend. KFiou is easier to implement than the exact SkewIoU, is fully differentiable, and can handle non-overlapping situations. Other rotation detectors based on Gaussian modeling such as GWD and KLD require additional hyperparameter tuning.

[0111] In some embodiments, the above step S620 may include: determining the Gaussian distribution of the rotating frame of the target to be detected; using the Kalman filter intersection-over-intersection loss function to calculate the overlapping area of ​​the rotating frame after conversion to the Gaussian distribution; obtaining the Kalman filter intersection-over-intersection loss according to the overlapping area; and updating the weight of the convolutional neural network model according to the Kalman filter intersection-over-intersection loss value. Specifically, it can be expressed as follows:

[0112] For a 2D rotating box in any direction, it is transformed into a Gaussian distribution , the following transformations can be made:

[0113]

[0114] Where R represents the rotation matrix, A diagonal matrix representing eigenvalues. For a two-dimensional target B 2d (x, y, h, w, θ), where h and w are the length and width of the rotation box, x and y are the coordinates, and θ represents the rotation angle, then:

[0115]

[0116]

[0117] Calculate the Gaussian distribution of the corresponding rotation box based on the variance:

[0118]

[0119] For two Gaussian distributions, KFiou can be used to calculate the overlapping area between them:

[0120]

[0121] α and μ can be written as:

[0122]

[0123]

[0124] KFiou loss can be calculated by the above overlapping area :

[0125]

[0126] Figure 7 The detection results of the computing model according to the embodiment of the present invention in the far-shore area are schematically shown. Figure 8 The schematic diagram shows the detection results of the computing model according to an embodiment of the present invention in the nearshore area. The present invention uses position attention in the model and adopts the Kalman intersection loss function for training, and uses the trained model to detect ship targets in nearshore and farshore scenes under SAR images. Figure 7 and Figure 8 As shown in the figure, it is a ship target detection image in a real scene. It can be seen that the model of this experiment can detect the ship target well regardless of the nearshore or farshore scene.

[0127] Fig. 9 A comparison diagram of the detection results of the operation model according to an embodiment of the present invention and the comparison model in the far-shore area is schematically shown. Fig.10 The comparison diagram of the detection results of the operation model according to the embodiment of the present invention and the comparison model in the nearshore area is schematically shown. Furthermore, the proposed detection method is compared with several advanced models. The comparison models include Gliding Vertex, Redet, S2ANet, Rotated FCOS, Rotated RetinaNet, RotatedFaster R-CNN, ROI-Transformer. Fig. 9 As shown in the figure, by comparison, it can be seen that in the far-shore scenario, the present invention is higher than other models in terms of detection accuracy and missed detection rate; the algorithm of the present invention accurately locates the ship, there is no tilt of the detection frame, and no false detection or missed detection occurs. At the same time, the same method is also compared in the near-shore scenario. Fig.10As shown, the method of the present invention has good accuracy in detecting targets, does not detect the shore area as a ship, and has higher positioning accuracy than other models.

[0128] The following table shows the comparison results of the average precision (AP) of the model of the present invention and the comparison model in nearshore and farshore scenarios. It can be clearly seen that the comparison between the target detection model based on the CA attention mechanism in rotating frame SAR images proposed by the present invention and other models shows that it has advantages in both nearshore and farshore detection.

[0129]

[0130] In summary, this paper proposes a high-precision rotating frame ship detection model based on position attention structure based on YOLOV7. It adopts the rotating frame detection mechanism and uses the KFioU loss function to train the model to efficiently learn ship targets in polarimetric SAR images. In shallow areas, position attention is used to focus on key areas to fuse multi-scale features. The KFioU loss function is used during training to avoid the problems of inaccurate positioning and complex hyperparameter adjustment of traditional rotating frame algorithms. Experiments show that these technologies significantly improve the quality of ship detection in SAR images, and can better detect and locate ship targets in both nearshore and farshore scenes.

[0131] Fig.11 A block diagram of an electronic device suitable for implementing the method described above according to an embodiment of the present invention is schematically shown. Fig.11 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0132] like Fig.11 As shown, the electronic device 1100 according to an embodiment of the present invention includes a processor 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1102 or a program loaded from a storage part 1108 to a random access memory (RAM) 1103. The processor 1001 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1001 may also include an onboard memory configured for cache purposes. The processor 1001 may include a single processing unit or multiple processing units configured to perform different actions of the method flow according to an embodiment of the present invention.

[0133] In RAM 1103, various programs and data required for the operation of electronic device 1100 are stored. Processor 1001, ROM 1102 and RAM 1103 are connected to each other via bus 1104. Processor 1001 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 1102 and / or RAM 1103. It should be noted that the program can also be stored in one or more memories other than ROM 1102 and RAM 1103. Processor 1001 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in one or more memories.

[0134] According to an embodiment of the present invention, the electronic device 1100 may further include an input / output (I / O) interface 1105, which is also connected to the bus 1104. The electronic device 1100 may further include one or more of the following components connected to the input / output (I / O) interface 1105: an input portion 1106 including a keyboard, a mouse, etc.; an output portion 1107 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 1108 including a hard disk, etc.; and a communication portion 1109 including a network interface card such as a LAN card, a modem, etc. The communication portion 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the input / output (I / O) interface 1105 as needed. A removable medium 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1110 as needed, so that a computer program read therefrom is installed into the storage portion 1108 as needed.

[0135] According to an embodiment of the present invention, the method flow according to an embodiment of the present invention can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code configured to execute the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1109, and / or installed from the removable medium 1111. When the computer program is executed by the processor 1001, the above-mentioned functions defined in the system of the embodiment of the present invention are executed. According to an embodiment of the present invention, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0136] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0137] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0138] For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 1102 and / or the RAM 1103 described above and / or one or more memories other than the ROM 1102 and the RAM 1103 .

[0139] An embodiment of the present invention also includes a computer program product, which includes a computer program, which contains program code configured to execute the method provided by the embodiment of the present invention. When the computer program product runs on an electronic device, the program code is configured to enable the electronic device to implement the training method and detection method provided by the embodiment of the present invention.

[0140] When the computer program is executed by the processor 1001, the above functions defined in the system / device of the embodiment of the present invention are executed. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0141] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 1109, and / or installed from a removable medium 1111. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0142] According to an embodiment of the present invention, the program code configured to execute the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0143] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present invention. In this regard, each box in the flowchart or block diagram may represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions configured to implement the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. It will be understood by those skilled in the art that the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways, even if such a combination or combination is not explicitly recorded in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features recorded in the various embodiments of the present invention can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present invention.

[0144] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A SAR image ship detection method, characterized in that: include: Performing data preprocessing on the acquired single-channel SAR image of the ship to be detected to obtain a multi-channel SAR image; Performing feature extraction on the multi-channel SAR image to obtain a first shallow feature map and feature maps of layers other than the shallow layer; Performing a coordinate attention operation on the first shallow feature map to generate a two-dimensional spatial attention map of the first shallow feature map, and obtaining a second shallow feature map of the multi-channel SAR image based on the two-dimensional spatial attention map; Fusing the second shallow feature map with the feature maps of the layers other than the shallow layer to obtain a multi-level fused feature map; Based on the preset anchor frame, the multi-level fusion feature map is predicted and screened to obtain the rotation frame detection result of the ship to be detected.

2. The SAR image ship detection method according to claim 1, characterized in that: The step of extracting features from the multi-channel SAR image to obtain a first shallow feature map and feature maps of layers other than the shallow layer includes: The original pixel information of the multi-channel SAR image is gradually converted into feature representations of different levels to obtain the first shallow feature map and feature maps of levels other than the shallow layer.

3. The SAR image ship detection method according to claim 2, characterized in that: The feature maps of the layers other than the shallow layer include a middle layer feature map and a deep layer feature map; The step of gradually converting the original pixel information of the multi-channel SAR image into feature representations of different levels to obtain the first shallow feature map and feature maps of levels other than the shallow layer includes: Performing multiple convolution, pooling and feature aggregation processes on the multi-channel SAR image to obtain the first shallow feature map; Performing pooling and feature aggregation processing on the first shallow feature map to obtain the middle feature map; The middle-layer feature map is pooled and feature aggregated to obtain the deep-layer feature map.

4. The SAR image ship detection method according to claim 3, characterized in that: The multi-level fusion feature map includes a shallow fusion feature map, a middle fusion feature map and a deep fusion feature map; The step of fusing the second shallow feature map with the feature map of a layer other than the shallow layer to obtain a multi-level fused feature map includes: Based on spatial pyramid pooling and a cross-stage partial connection network, feature enhancement is performed on the deep feature map; Weightedly fusing the middle-layer feature map and the feature-enhanced deep-layer feature map to the second shallow-layer feature map to obtain the shallow-layer fused feature map; Weightedly fusing the second shallow feature map and the feature-enhanced deep feature map into the middle-layer feature map to obtain the middle-layer fused feature map; The second shallow feature map and the middle feature map are weightedly fused to the feature-enhanced deep feature map to obtain the deep fused feature map.

5. The SAR image ship detection method according to claim 1, characterized in that: The performing of a coordinate attention operation on the first shallow feature map to generate a two-dimensional spatial attention map of the first shallow feature map, and obtaining a second shallow feature map of the SAR image in terms of direction and position perception based on the two-dimensional spatial attention map, comprises: Encoding and compressing the first shallow feature map in horizontal and vertical directions to obtain the two-dimensional spatial attention map; The two-dimensional spatial attention map and the first shallow feature map are weightedly summed to obtain the second shallow feature map that is aware of direction and position.

6. The SAR image ship detection method according to claim 1, characterized in that: The method of performing data preprocessing on the acquired single-channel SAR image of the ship to be detected to obtain a multi-channel SAR image includes: The single-channel SAR image is subjected to band normalization, radiation correction, multi-viewing, filtering, terrain correction and channel expansion processing in sequence to obtain the multi-channel SAR image.

7. The SAR image ship detection method according to claim 1, characterized in that: The method of predicting and screening the multi-level fusion feature map based on the preset anchor frame to obtain the rotation frame detection result of the ship to be detected includes: Based on the preset anchor frame, determining a prediction frame on each level of the fused feature map in the multi-level fused feature map; According to the comparison relationship between the preset anchor frame and the predicted frame, the predicted frame with a confidence exceeding a preset value is screened out by decoding relative coordinates and applying a category probability threshold; The non-maximum suppression method is used to remove overlapping frames in the predicted frame to obtain the rotation frame detection result.

8. A SAR image ship detection device, characterized in that: include: A preprocessing module is used to perform data preprocessing on the acquired single-channel SAR image of the ship to be detected to obtain a multi-channel SAR image; A computing model is used to extract features from the multi-channel SAR image to obtain a first shallow feature map and a feature map of a layer other than the shallow layer, perform a coordinate attention operation on the first shallow feature map to generate a two-dimensional spatial attention map of the first shallow feature map, obtain a second shallow feature map of the SAR image for direction and position perception based on the two-dimensional spatial attention map, and (fuse the second shallow feature map with the feature map of the layer other than the shallow layer to obtain a multi-level fused feature map; The detection module is used to predict and filter the multi-level feature map based on a preset anchor frame to obtain a rotation frame detection result of the ship to be detected.

9. The SAR image ship detection device according to claim 8, characterized in that: The operation model is constructed based on the YOLOv7 model. A basic convolution module is configured in the convolution layer of the YOLOv7 model. The output end of the basic convolution module is connected to a coordinate attention module. The coordinate attention module is used to perform a coordinate attention operation on a first shallow feature map of the multi-channel SAR image to generate a two-dimensional spatial attention map of the first shallow feature map, and obtain a second shallow feature map of the SAR image for direction and position perception based on the two-dimensional spatial attention map.

10. A training method for training the computing model of the SAR image ship detection device as claimed in claim 8 or 9, characterized in that: include: According to the specific data set of the SAR image ship, determining the training data set and the verification data set of the ship to be detected in different scenes; Using a specific loss function and the training data set to perform weight training on the operation model, wherein the specific loss function includes a Kalman filter intersection-over-combination loss function; The training results of the operation model are verified by using the verification data set.

Images