Ship detection method in multi-scale SAR images based on attention mechanism

Abstract

The invention discloses a multi-scale SAR image ship detection method based on attention mechanism, firstly, a SAR image ship data set is obtained and divided into a training set, a validation set and a test set; then, YOLOv7 is used as a basic model, a backbone network and a feature fusion network are improved, and a coordinate attention module is added to the improved backbone network to construct a multi-scale SAR image ship target detection network SSYOLO; then, the target detection network SSYOLO is trained based on the training set, the validation set and the test set to obtain a SAR image ship target detection model; finally, the image to be detected is input into the SAR image ship target detection model to obtain a detection result. The invention improves the accuracy of SAR image ship detection, can not only accurately locate ship targets under complex ocean backgrounds, but also can reduce a large amount of manual monitoring costs, and is helpful for applications such as marine traffic control, fishery management, environmental protection, disaster relief, etc.

Description

Technical Field

[0001] The present invention relates to the technical field of information perception and recognition, and in particular to a multi-scale SAR image ship detection method based on an attention mechanism. Background Art

[0002] Synthetic Aperture Radar (SAR) is a coherent imaging system that can produce high-resolution remote sensing images all day and all night. It has very good adaptability for monitoring the unpredictable climate of the ocean and plays an important role in military and civilian ship detection, such as marine traffic control, fishery management, environmental protection, disaster relief and other applications. SAR image ship detection can effectively improve ship transportation efficiency and reduce maritime traffic accidents, but due to the uncertainty of sea clutter, the size of ship targets and the interference of land clutter, the detection performance of ship targets will be greatly affected. Therefore, SAR image ship detection is still an important and challenging research task.

[0003] Traditional SAR image ship target detection methods rely on constant false alarm detectors. The main idea is to model the statistical characteristics of seawater and ship targets, but the difficulty of this method also lies in statistical modeling. Different sea conditions and interference from multiple targets in actual scenes will affect the accurate modeling of sea clutter. With the development of deep learning, ship target detection in SAR images has ushered in a new wave of research enthusiasm. Compared with previous methods, it has the advantages of high detection accuracy, fast detection speed, and high robustness, but it still faces major challenges. First, in most of the current multi-scale ship recognition models, in simple scenarios, the interference is small and good detection performance can be achieved, such as the target to be recognized is a ship sailing on a clear sea. However, in addition to this common situation, SAR images show moving ship targets under various driving conditions, including maritime operations, near-shore navigation, and mooring. Since the near-shore area is close to the land, this greatly increases the complexity of the background, especially when the ships are closely clustered together. In this case, most models cannot adequately locate the ship, resulting in more missed detections. Secondly, there is a problem of missed detection of small ship targets. It can be said that small-scale target recognition is the key to enhancing the model's multi-scale ship recognition capability. In multiple downsampling feature extractions, the structure and size limitations of feature pixels make it easy to lose target information that occupies fewer pixel blocks. Summary of the invention

[0004] The technical problem to be solved by the present invention is to address the defects involved in the background technology and provide a multi-scale SAR image ship detection method based on the attention mechanism, which can improve the accuracy of ship detection of different scales in different scenarios, and is helpful for applications such as marine traffic control, fishery management, environmental protection, and disaster relief.

[0005] The present invention adopts the following technical solutions to solve the above technical problems:

[0006] The multi-scale SAR image ship detection method based on attention mechanism includes the following steps:

[0007] Step 1), obtaining a SAR image ship dataset and dividing it into a training set, a validation set, and a test set;

[0008] Step 2), using YOLOv7 as the basic model, improving the backbone network and feature fusion network, and adding a coordinate attention module to the improved backbone network to build a multi-scale SAR image ship target detection network SSYOLO;

[0009] Step 2.1), improve the backbone network;

[0010] Step 2.1.1), build L-ELAN module;

[0011] The L-ELAN module includes the first to third common convolution blocks and the first to fourth depth-separable convolution blocks, wherein the first to third common convolution blocks have the same structure, and all include common convolution, normalized BN layer, and activation function SiLU; the first to fourth depth-separable convolution blocks have the same structure, and all include depth-separable convolution, normalized BN layer, and activation function SiLU;

[0012] The first common convolution block and the first to fourth depth-separable convolution blocks are connected in sequence, the input ends of the first and second common convolution blocks are connected and used as the input end of the L-ELAN module, the output end of the first common convolution block, the output end of the second common convolution block, the output end of the second depth-separable convolution block, and the output end of the fourth depth-separable convolution block are spliced ​​by channel and input to the input end of the third common convolution block, and the output end of the third common convolution block is used as the output end of the L-ELAN module;

[0013] Step 2.1.2), replace the first and fourth ELAN modules in YOLOv7 with L-ELAN modules;

[0014] Step 2.2), improve the feature fusion network:

[0015] Step 2.2.1), replace all nodes in the top-down path of the feature fusion network of YOLOv7 after the improved backbone network with weighted feature fusion nodes. The weighted formula of the weighted feature fusion node is as follows:

[0016]

[0017] Among them, O is the output feature map of the weighted feature fusion node, I i represents the i-th input feature map, w i represents the weight corresponding to the i-th feature map, w j represents the weight corresponding to the jth feature map, ∑ j w j Indicates the sum of all weights; ∈ is any small positive number that is preset to avoid the denominator being 0;

[0018] Step 2.2.2), add skip-layer connections to YOLOv7 after replacing all nodes, and directly connect the 40×40 scale feature map output by its backbone network to the feature map of the same scale in the bottom-up path of its feature fusion network;

[0019] Step 2.3), add the coordinate attention module to the improved backbone network;

[0020] Step 2.3.1), establish LECA module;

[0021] The LECA module includes the fourth to sixth common convolution blocks, the fifth to eighth depth-separable convolution blocks, and the first to second coordinate attention units, wherein the fourth to sixth common convolution blocks have the same structure, and all include common convolution, normalized BN layer, and activation function SiLU; the fifth to eighth depth-separable convolution blocks have the same structure, and all include depth-separable convolution, normalized BN layer, and activation function SiLU; the first to second coordinate attention units have the same structure, and all adopt the coordinate attention mechanism;

[0022] The fourth common convolution block, the fifth to eighth depth-separable convolution blocks, and the first coordinate attention unit are connected in sequence, the input ends of the fourth and fifth common convolution blocks are connected and used as the input end of the LECA module, the output end of the fifth common convolution block and the input end of the second coordinate attention unit are connected, the output end of the fourth common convolution block, the output end of the sixth depth-separable convolution block, the output end of the first coordinate attention unit, and the output end of the second coordinate attention unit are spliced ​​according to channels and input to the input end of the sixth common convolution block, and the output end of the sixth common convolution block is used as the output end of the LECA module;

[0023] Step 2.3.2), replace the second and third ELAN modules in the YOLOv7 backbone network with skip-layer connections with LECA modules;

[0024] Step 3), training the target detection network SSYOLO based on the training set, validation set and test set to obtain a SAR image ship target detection model for identifying ships in different scales and scenes;

[0025] Step 4), input the image to be detected into the SAR image ship target detection model to obtain the detection result.

[0026] As a further optimization scheme of the multi-scale SAR image ship detection method based on the attention mechanism of the present invention, in step 1), the SAR image ship dataset is divided into a training set, a validation set, and a test set according to 7:2:1.

[0027] As a further optimization scheme of the multi-scale SAR image ship detection method based on the attention mechanism of the present invention, when training the target detection network SSYOLO in step 3), the loss function uses EIOU to replace the CIOU function of the original YOLOv7.

[0028] Compared with the prior art, the present invention adopts the above technical solution and has the following technical effects:

[0029] Based on the YOLOv7 model, the present invention proposes a multi-scale SAR image ship detection method based on the attention mechanism. First, the L-ELAN module is constructed using deep separable convolution and replaced with the original YOLOv7 backbone network, which can not only make the model lightweight, but also improve the feature extraction ability of the backbone network; then the feature enhancement LECA module based on the coordinate attention mechanism is constructed and replaced with the original YOLOv7 backbone network to highlight the location information of the target and suppress background interference; then the feature fusion network is improved, weighted nodes and skip layer connections are introduced, and the multi-scale detection ability and detection accuracy of the model are improved; finally, in order to improve the effectiveness of training and the accuracy of reasoning, the EIOU loss function is used to replace the CIOU loss function of the original YOLOv7 in the bounding box regression part. The present invention improves the accuracy and robustness of SAR image ship target detection through the above design, and meets the ship recognition under multi-scale and complex background conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the process of the present invention;

[0031] Figure 2 This is a schematic diagram of the structure of the multi-scale detection network SSYOLO based on the attention mechanism;

[0032] Figure 3 It is a schematic diagram of the structure of the L-ELAN module in the present invention;

[0033] Figure 4It is a schematic diagram of the structure of the LECA module in the present invention;

[0034] Figure 5 It is a structural diagram of the coordinate attention unit CA in the present invention. DETAILED DESCRIPTION

[0035] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings:

[0036] The present invention can be implemented in many different forms and should not be considered to be limited to the embodiments described herein. On the contrary, these embodiments are provided to make this disclosure thorough and complete, and will fully express the scope of the present invention to those skilled in the art. In the accompanying drawings, components are enlarged for clarity.

[0037] In this embodiment, the present invention is based on the YOLOv7 model and proposes a multi-scale SAR image ship detection model SSYOLO based on the attention mechanism. Figure 1 As shown in the flow chart, the method comprises the following steps:

[0038] Step 1), obtain the SAR image ship dataset and divide it into training set, validation set and test set:

[0039] In this embodiment, first, an original SAR image ship data set containing images and corresponding image annotation data is obtained, the image format is generally in jpg format, and the annotation data is generally in VOC format or COCO format; then, the image annotation data format is converted to obtain the txt format data required for model training, and then the data set is divided into a training set, a validation set, and a test set according to a ratio of 7:2:1. It should be noted that the preset ratio of dividing the data set in this embodiment can be adjusted according to the actual application situation, and 7:2:1 is only for example.

[0040] Step 2), using YOLOv7 as the basic model, improving the backbone network and feature fusion network, and adding the coordinate attention module to the improved backbone network, constructing the multi-scale SAR image ship target detection network SSYOLO, as shown in Figure 2 As shown;

[0041] Step 2.1), improve the backbone network;

[0042] Step 2.1.1), establish the L-ELAN module, reduce the number of parameters brought by the ordinary convolution stack, achieve the lightweight effect of the module, and retain more channel information;

[0043] like Figure 3As shown, the L-ELAN module includes the first to third common convolution blocks and the first to fourth depth-separable convolution blocks, wherein the first to third common convolution blocks have the same structure, all including common convolution, normalized BN layer, and activation function SiLU; the first to fourth depth-separable convolution blocks have the same structure, all including depth-separable convolution, normalized BN layer, and activation function SiLU;

[0044] The first common convolution block and the first to fourth depth-separable convolution blocks are connected in sequence, the input ends of the first and second common convolution blocks are connected and used as the input end of the L-ELAN module, the output end of the first common convolution block, the output end of the second common convolution block, the output end of the second depth-separable convolution block, and the output end of the fourth depth-separable convolution block are spliced ​​by channel and input to the input end of the third common convolution block, and the output end of the third common convolution block is used as the output end of the L-ELAN module;

[0045] Step 2.1.2), replace the first and fourth ELAN modules in the original YOLOv7 backbone network with L-ELAN modules;

[0046] After feature extraction by the SSYOLO backbone network, three feature maps of different scales are finally output: C3: 20×20, C4: 40×40, C5: 80×80.

[0047] Step 2.2), improve the feature fusion network:

[0048] Step 2.2.1), such as Figure 2 As shown in the figure, in the top-down path of the feature fusion network of YOLOv7 after the improved backbone network, all nodes are replaced with weighted feature fusion nodes, so that the model can learn the importance of input features at different levels. The weighted formula of the weighted feature fusion node is as follows:

[0049]

[0050] Among them, O is the output feature map of the weighted feature fusion node, I i represents the i-th input feature map, w i represents the weight corresponding to the i-th feature map, w j represents the weight corresponding to the jth feature map, ∑ j w j Indicates the sum of all weights; ∈ is any small positive number that is preset to avoid the denominator being 0;

[0051] In step 2.2.2), a skip connection is added to YOLOv7 after all nodes are replaced, and the C4: 40×40 scale feature map output by its backbone network is directly connected to the feature map of the same scale in the bottom-up path of its feature fusion network; the path between layers of the same size in the network is shortened, and the information flow between different layers is enhanced.

[0052] Step 2.3), add the coordinate attention module to the improved backbone network;

[0053] Step 2.3.1), establish LECA module;

[0054] like Figure 4 As shown, the LECA module includes the fourth to sixth common convolution blocks, the fifth to eighth depth-separable convolution blocks, and the first to second coordinate attention units, wherein the fourth to sixth common convolution blocks have the same structure, all including common convolution, normalized BN layer, and activation function SiLU; the fifth to eighth depth-separable convolution blocks have the same structure, all including depth-separable convolution, normalized BN layer, and activation function SiLU; the first to second coordinate attention units have the same structure, all using the coordinate attention mechanism;

[0055] The fourth common convolution block, the fifth to eighth depth-separable convolution blocks, and the first coordinate attention unit are connected in sequence, the input ends of the fourth and fifth common convolution blocks are connected and used as the input end of the LECA module, the output end of the fifth common convolution block and the input end of the second coordinate attention unit are connected, the output end of the fourth common convolution block, the output end of the sixth depth-separable convolution block, the output end of the first coordinate attention unit, and the output end of the second coordinate attention unit are spliced ​​according to channels and input to the input end of the sixth common convolution block, and the output end of the sixth common convolution block is used as the output end of the LECA module;

[0056] Step 2.3.2), replace the second and third ELAN modules in the YOLOv7 backbone network with skip-layer connections with LECA modules;

[0057] Coordinate attention mechanism such as Figure 5 As shown in the figure, the input features are average pooled along the horizontal and vertical coordinate directions to generate a pair of direction-aware attention features, and then the two feature maps are cascaded, and then the channel dimension reduction operation is performed through convolution. Then the feature map is divided into two parts along the spatial dimension, and it is restored to the same number of channels as the original input through convolution. Finally, sigmoid activation is performed as the final attention weight. The whole process can be expressed as:

[0058] X h =YAvgPool(F in )

[0059] X w =XAvgPool(F in )

[0060] W1=CS(Split h (CBH(Concat(X h , X w ))))

[0061] W2=CS(Split w (CBH(Concat(X h , X w ))))

[0062]

[0063] Among them, YAvgPool(·) represents average pooling according to the vertical coordinate, XAvgPool(·) represents average pooling according to the horizontal coordinate, and F in Represents input features. Concat(·,·) represents a concatenation operation, Split(·) represents a splitting operation, CBH(·) represents first a Conv convolution, then a BN normalization, and then an h_swish activation function, and CS(·) represents first a Conv convolution, and then a sigmoid activation function.

[0064] Step 3), training the target detection network SSYOLO based on the training set, validation set and test set to obtain a SAR image ship target detection model for identifying ships in different scales and scenes;

[0065] Step 3.1), set the training parameters of the SSYOLO network as follows, which can be adjusted according to the actual application:

[0066] Parameter name Parameter Value Parameter Description batch_size 8 Number of training images per batch lr0 0.01 initial learning rate(SGD=1E-2,Adam=1E-3) lrf 0.1 final OneCycleLR learning rate(lr0*lrf) momentum 0.937 SGD momentum / Adam beta1 weight_decay 0.0005 optimizer weight decay 5e-4

[0067] In step 3.2), the EIOU loss function is used to replace the CIOU function of the original YOLOv7 during bounding box regression training.

[0068] The penalty term of EIOU is based on the penalty term of CIOU, which separates the influencing factors of aspect ratio and calculates the length and width of the target box and anchor box respectively. The loss function contains three parts: overlap loss, center distance loss, and width and height loss. The first two parts continue the method in CIOU, but the width and height loss directly minimizes the difference between the width and height of the target box and the anchor box, making the convergence speed faster. The penalty term formula is as follows:

[0069]

[0070] Where IOU is the intersection-over-union ratio of two bounding boxes, ρ 2 (b,bgt ) is the distance between the center of the predicted box and the real box, c is the diagonal distance of the smallest rectangular box containing the two bounding boxes, w and h are the width and height of the predicted box, w gt and h gt is the width and height of the real frame, C w and C h is the width and height of the minimum bounding box covering the two bounding boxes. Considering that there is also an imbalanced training sample problem in the regression of the position box, that is, the number of high-quality anchor boxes with small regression errors in an image is far less than the number of low-quality samples with large errors, and poor-quality samples will produce excessive gradients that affect the training process. Based on EIOU and FocalLoss, from the perspective of gradient, high-quality anchor boxes and low-quality anchor boxes are separated. The penalty term formula is as follows:

[0071] L Focal-EIOU =IOU γ L EIOU

[0072] Among them, γ is a parameter that controls the degree of suppression of outliers. According to the above formula, the higher the IOU value, the greater the loss, which is equivalent to the weighted effect. The better the regression target, the greater the loss, which helps to improve the regression accuracy.

[0073] Step 4), input the image to be detected into the SAR image ship target detection model to obtain the detection result.

[0074] In order to verify the effectiveness of this solution, the following is a further explanation based on experimental data:

[0075] The experimental environment relies on PyCharm IDE. The deep learning framework consists of PyTorch 2.0.1, CUDA 11.8 and CUDNN 8.9. The CPU is 12th Gen Intel(R) Core(TM) i7-12700 2.10GHz, the GPU is NVIDIAGeforce GTX 3060, and the PC operating system is Windows 11. The experimental datasets are: SSDD dataset and HRSID dataset. The experimental evaluation indicators are precision, recall, and average precision (mAP). 50 ), small target accuracy (AP S ) index, mid-target accuracy (AP M ) index, large target accuracy (AP L )index.

[0076] The detection model of this case solution is compared with the original basic YOLOv7 model on the SSDD dataset and the HRSID dataset. The specific comparison results are shown in Table 1 and Table 2. As can be seen from Table 1, compared with YOLOv7, on the SSDD dataset, the accuracy of the model proposed in this case is improved by 1.4%, the recall rate is improved by 7.4%, the average precision is improved by 4%, and the Ap is improved by 1.4%. S Reached 62.3%, AP M Reaching 61.8%, Ap L It reaches 26.8%. As can be seen from Table 2, due to the more complex background and more difficult targets in the HRSID dataset, the overall detection accuracy is lower than that of SSDD, but the performance of this solution is still better than the original YOLOv7, with both precision and recall increased by 5.5%, and the average accuracy increased by 5.9%. In addition, Table 3 lists the different detection results of this solution in nearshore and offshore scenarios. It can be seen that the model performance in complex scenarios is also good.

[0077] Table 1: Performance comparison with YOLOv7 on the SSDD dataset

[0078]

[0079] Table 2: Performance comparison with YOLOv7 on the HRSID dataset

[0080]

[0081] Table 3: Performance of nearshore and offshore scenarios in the SSDD dataset

[0082]

[0083] In summary, the present invention improves YOLOv7 by constructing an L-ELAN module to replace the first and fourth ELAN modules in the original YOLOv7 backbone network; constructing an LECA module to replace the second and third ELAN modules in the original YOLOv7 backbone network; introducing weighted nodes and skip layer connections in the feature fusion network, and using the EIOU bounding box loss function to replace the CIOU loss function in the original YOLOv7 during training. The feature extraction capability of the model is enhanced, which is beneficial to multi-scale feature fusion and improves the overall detection accuracy, especially the small target detection accuracy.

[0084] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with the meanings in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless defined as herein.

[0085] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A multi-scale SAR image ship detection method based on attention mechanism, characterized in that: The following steps are involved: Step 1), obtain the SAR image ship dataset and divide it into training set, validation set and test set; Step 2), based on YOLOv7 as the basic model, the backbone network and feature fusion network are improved, and the coordinate attention module is added to the improved backbone network to build the multi-scale SAR image ship target detection network SSYOLO; Step 2.1), improve the backbone network; Step 2.1.1), build the L-ELAN module; The L-ELAN module includes the first to third common convolution blocks and the first to fourth depth-separable convolution blocks, wherein the first to third common convolution blocks have the same structure, and all include common convolution, normalized BN layer, and activation function SiLU; the first to fourth depth-separable convolution blocks have the same structure, and all include depth-separable convolution, normalized BN layer, and activation function SiLU; The first common convolution block and the first to fourth depth-separable convolution blocks are connected in sequence, the input ends of the first and second common convolution blocks are connected and used as the input end of the L-ELAN module, the output end of the first common convolution block, the output end of the second common convolution block, the output end of the second depth-separable convolution block, and the output end of the fourth depth-separable convolution block are spliced ​​by channel and input to the input end of the third common convolution block, and the output end of the third common convolution block is used as the output end of the L-ELAN module; Step 2.1.2), replace the first and fourth ELAN modules in YOLOv7 with L-ELAN modules; Step 2.2), improve the feature fusion network: Step 2.2.1), in the top-down path of the feature fusion network of YOLOv7 after the improved backbone network, all nodes are replaced with weighted feature fusion nodes. The weighted formula of the weighted feature fusion node is as follows: ； Among them, O is the output feature map of the weighted feature fusion node, Indicates Input feature maps, Indicates The weight corresponding to the feature map is Indicates The weight corresponding to the feature map is It means to sum all weights; It is any small positive number that is preset to avoid the denominator being 0; Step 2.2.2), add skip-layer connections to YOLOv7 after replacing all nodes, and directly connect the 40×40 scale feature map output by its backbone network to the feature map of the same scale in the bottom-up path of its feature fusion network; Step 2.3), add the coordinate attention module to the improved backbone network; Step 2.3.1), establish the LECA module; The LECA module includes the fourth to sixth common convolution blocks, the fifth to eighth depth-separable convolution blocks, and the first to second coordinate attention units, wherein the fourth to sixth common convolution blocks have the same structure, and all include common convolution, normalized BN layer, and activation function SiLU; the fifth to eighth depth-separable convolution blocks have the same structure, and all include depth-separable convolution, normalized BN layer, and activation function SiLU; the first to second coordinate attention units have the same structure, and all adopt the coordinate attention mechanism; The fourth common convolution block, the fifth to eighth depth-separable convolution blocks, and the first coordinate attention unit are connected in sequence, the input ends of the fourth and fifth common convolution blocks are connected and used as the input end of the LECA module, the output end of the fifth common convolution block and the input end of the second coordinate attention unit are connected, the output end of the fourth common convolution block, the output end of the sixth depth-separable convolution block, the output end of the first coordinate attention unit, and the output end of the second coordinate attention unit are spliced ​​according to the channel and input to the input end of the sixth common convolution block, and the output end of the sixth common convolution block is used as the output end of the LECA module; Step 2.3.2), replace the second and third ELAN modules in the YOLOv7 backbone network with skip-layer connections with LECA modules; Step 3), train the target detection network SSYOLO based on the training set, validation set and test set to obtain the SAR image ship target detection model, which is used to identify ships in different scales and scenes; Step 4) Input the image to be detected into the SAR image ship target detection model to obtain the detection result.

2. The multi-scale SAR image ship detection method based on attention mechanism according to claim 1 is characterized in that: In step 1), the SAR image ship dataset is divided into training set, validation set, and test set according to the ratio of 7:2:

1.

3. The multi-scale SAR image ship detection method based on attention mechanism according to claim 1 is characterized in that: In step 3), when training the target detection network SSYOLO, the loss function uses EIOU to replace the CIOU function of the original YOLOv7.

Images