SAR image oil spill detection method
By using a U-Net model with a coordinated attention mechanism for feature fusion, combined with wavelet threshold transformation and multiple feature extraction techniques, the problem of low accuracy in traditional SAR image oil spill detection is solved, achieving a more efficient oil spill detection effect.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- JIANGSU UNIV OF SCI & TECH
- Filing Date
- 2023-01-04
- Publication Date
- 2026-05-12
Smart Images
Figure QLYQS_1 
Figure QLYQS_5 
Figure QLYQS_6
Abstract
Description
Technical Field
[0001] This invention relates to the field of oil spill segmentation in synthetic aperture radar (SAR) images, specifically to a method for detecting oil spills in SAR images, which is based on U-Net feature fusion using a coordinated attention mechanism. Background Technology
[0002] Oil spills cause irreversible damage to marine ecosystems. Oil spill segmentation is a key step in SAR image oil spill detection. Synthetic Aperture Radar (SAR) has become an important technology for monitoring marine oil spills. SAR can provide electromagnetic information for detecting marine oil spills. SAR obtains electromagnetic information on the sea surface through a scattering mechanism. The information obtained differs when the scattering mechanism occurs on an oil-covered surface and on a clean sea surface. For a clean sea surface, strong Bragg scattering occurs, which appears bright in the SAR image. When an oil spill occurs, it weakens Bragg scattering, which appears dark in the SAR image. Traditional methods have unavoidable limitations: (1) the selection of threshold segmentation parameters is greatly affected by subjective factors or experience; (2) single feature information cannot represent global features, which will affect the segmentation effect; (3) traditional methods mostly use low-level information of the image to complete the segmentation task, making it difficult to extract deep semantic information of the image. These limitations lead to low accuracy in oil spill image segmentation based on traditional methods.
[0003] In recent years, some deep learning models have been used for marine oil spill detection in SAR images. Deep learning, as a branch of machine learning, is dedicated to solving machine learning tasks through neural network models. Unlike traditional machine learning algorithms, deep neural networks extract image features layer by layer through their deep network layers. Deep features are usually more abstract and contain deep semantic information. Moreover, the feature extraction process is automatic and does not require manual intervention, greatly improving efficiency. Although deep learning models have achieved good detection results in oil spill detection tasks, there are still some limitations in further improving detection accuracy. Due to the model depth, insufficient feature extraction and a small receptive field lead to the loss of target information and the generation of redundant information in the model. Summary of the Invention
[0004] To address the aforementioned technical problems, this invention provides a SAR image oil spill detection method based on a coordinated attention mechanism-based feature fusion U-Net. This method uses wavelet thresholding to remove image noise and then fuses the features with the original image. By embedding a coordinated attention module into the original network, high-level features in the image are extracted, redundant features are reduced, important features are emphasized, and the segmentation accuracy is improved.
[0005] To achieve the above objectives, the present invention provides the following technical solution:
[0006] A method for detecting oil spills in SAR images includes the following steps:
[0007] Step 1: Select 750 images from the European Space Agency (ESA) oil spill detection dataset as the training set and 250 images as the test set. Then, randomly select 90% of the training set for experimental training and 10% of the training set for experimental verification.
[0008] Step 2: Preprocess the training set selected in Step 1, i.e., randomly crop and stitch the training set: the original SAR image format is 1250×650, and the preprocessed dataset image size is resized to 256×256 with 3 feature channels.
[0009] Step 3: Denoise the training set obtained in Step 2 using Wavelet Threshold Transform (WTT);
[0010] Step 4: Build a feature fusion U-Net model based on the coordinated attention mechanism. Randomly select 50% of the training set images obtained in Step 2 and Step 3 as input to the network model to perform global feature extraction and fusion.
[0011] Step 5: Use the Residual Model module, and add squeezing and excitation (SE) modules after some residual blocks to enable the model to learn the weight coefficients of each channel autonomously;
[0012] Step 6: Embed a Coordinating Attention (CA) module at the jump connection to eliminate redundant information;
[0013] Step 7: A Spatial Pyramid structure with dilated convolutions (ASPP) is used as the bottom layer network to increase the receptive field and extract more extensive feature information.
[0014] Step 8: Input the selected test set into the trained U-Net model based on the coordinated attention mechanism for testing, and obtain the network's segmentation results.
[0015] The present invention is further improved in that the specific data augmentation operation in step 2 includes randomly cropping the image to a size between 0.2 and 0.4 times the original image size, cropping the aspect ratio to 1:2, then stitching the cropped images together in pairs, and reshaping them into a 256×256×3 format.
[0016] The present invention is further improved by introducing variables into the wavelet threshold function proposed in step 3, making the function continuously differentiable as a whole and relatively smooth at the threshold. The wavelet threshold function is as follows:
[0017]
[0018] Here, m and k are variables, m∈[0,1], and k is a positive integer.
[0019] The present invention is further improved, and the specific steps of step 4 are as follows:
[0020] (1) The U-Net model based on coordinated attention mechanism for feature fusion employs 17 convolutional layers, 4 downsampling layers, 4 upsampling layers, and 4 cropping and copying layers. The input image format is 256×256×3. Each convolutional layer forms a residual block in pairs. Each convolutional layer uses a 3×3 kernel, with the activation function being swish and zero-padding. A squeeze activation module is placed after the residual blocks to improve the model's discriminative power for features. The downsampling layers use max pooling with a 2×2 sampling window and a stride of 2. A pyramid structure with dilated convolutions is placed at the bottom of the network to increase the receptive field. The upsampling layers use inverted convolutions with a 3×3 kernel, a stride of 2, and zero-padding. The coordinated attention module is embedded in skip connections to eliminate redundant features. Finally, a 1×1 convolutional layer reduces the dimensionality of the image and outputs the segmentation result.
[0021] (2) Randomly select 50% of the training set images obtained in steps 2 and 3 as the input of the network, that is, fuse the high-dimensional features of the denoised image and the high-dimensional features of the original image.
[0022] The present invention is further improved, and the specific operation of step 5 is as follows:
[0023] (1) Perform convolution operation Ftr: Given an input F∈R H*W*C W and H are the width and height of the input matrix, and C is the number of feature channels of the input matrix. A simple convolution operation is performed, which is the residual network.
[0024] (2) Perform Fsq compression operation, as shown in Formula 2: the feature map is compressed along the spatial dimension, that is, global average pooling is performed to obtain a 1×1×C (C is the number of feature channels) feature matrix.
[0025]
[0026] (3) Perform the Fex activation operation and assign different weights W according to the different importance of the features, as shown in formula (3):
[0027] s = F ex (z,W)=σ(W2δ(W1z)) (3)
[0028] In the formula: z describes the character of the channel, s is the weight used to reflect the feature map in tensor U, δ is the Swish activation function, σ is the Sigmoid activation function, and W1 and W2 are the weights of the fully connected layer used for dimensionality increase and decrease.
[0029] (4) The final weight recalibration operation transforms the two-dimensional matrix u c Multiply each value by s c As shown in formula (4):
[0030]
[0031] In the formula: s c F represents the weight of the c-th feature map in tensor U. scale Indicator quantity s c With feature mapping u c The channel product.
[0032] The present invention is further improved, and the specific operation of step 6 is as follows:
[0033] (1) Perform convolution operation Ftr: Given an input F∈R H*W*C W and H are the width and height of the input matrix, and C is the number of feature channels of the input matrix. A simple convolution operation is performed, which is the residual network processing.
[0034] (2) Perform Fsq compression operation, as shown in Formulas 5 and 6: the feature map is compressed in the horizontal and vertical directions, that is, convolution operation with kernels of 1×W and H×1 is performed to obtain feature matrices of H×1×C and 1×W×C (C is the number of feature channels).
[0035]
[0036]
[0037] (3) Perform the Fex activation operation and assign different weights W according to the different importance of the features, as shown in formula (7):
[0038] S h =F ex (U,W)=σ(g(u,W))=σ(W2δ(W1)) (7)
[0039] S w =F ex (U,W)=σ(g(u,W))=σ(W2δ(W1)) (8)
[0040] In the formula: U describes the character of the channel, S h and S w These are the weights used to reflect the horizontal and vertical feature maps in tensor U, where δ is the Swish activation function, σ is the Sigmoid activation function, and W1 and W2 are the weights used by the fully connected layer for dimensionality increase and decrease.
[0041] (4) The final weight recalibration operation transforms the two-dimensional matrix u c Multiply each value by s c As shown in formula (8):
[0042]
[0043]
[0044] In the formula: s h and s w F represents the weights of the horizontal and vertical feature maps in tensor U. scale Indicator quantity s c With feature mapping u c The channel product.
[0045] The present invention is further improved in that the dilated convolution in the pyramid structure with dilated convolution shown in step 7 is decomposed into two dilated convolutions with different dilation rates, which improves the sensitivity to feature extraction, and each layer is tightly connected to the previous layer for feature sharing.
[0046] The beneficial effects of this invention are as follows: This invention uses data augmentation technology to expand the dataset, fully train the parameters in the model, improve the model's generalization ability and robustness, and prevent overfitting; the feature extraction model replaces conventional convolutional layers with residual modules to fully and comprehensively extract high-level features; squeezing and activation modules are embedded after some residual modules, which only slightly increases the model's complexity and computational load, improves the model's ability to distinguish extracted features, emphasizes important features, and eliminates redundant features; a coordinated attention module is embedded in skip connections to eliminate redundant features in low-level features and improve the network's representation ability; a feature pyramid structure with dilated convolutions is used as the network's bottom layer structure, which increases the receptive field and improves the sensitivity to feature extraction. Attached Figure Description
[0047] Figure 1 This is a flowchart of the present invention.
[0048] Figure 2 This is a diagram of the neural network structure of the present invention.
[0049] Figure 3 This is a structural diagram of the residual module of the present invention.
[0050] Figure 4 This is a structural diagram of the extrusion-excitation attention module of the present invention.
[0051] Figure 5 This is a structural diagram of the coordinated attention module of the present invention.
[0052] Figure 6 This is a structural diagram of the feature pyramid module with dilated convolution of the present invention. Detailed Implementation
[0053] To enhance understanding of the present invention, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. These embodiments are only used to explain the invention and do not limit the scope of protection of the invention.
[0054] Example: Figures 1 to 5 As shown, a method for detecting oil spills in SAR images includes the following steps:
[0055] Step 1: Select 750 images from the European Space Agency (ESA) oil spill detection dataset as the training set and 250 images as the test set. Then, randomly select 90% of the training set for experimental training and 10% of the training set for experimental verification.
[0056] Step 2: Preprocess the training set selected in Step 1, i.e., randomly crop and stitch the training set: the original SAR image format is 1250×650, and the preprocessed dataset image size is resized to 256×256 with 3 feature channels.
[0057] Step 3: Denoise the training set obtained in Step 2 using Wavelet Threshold Transform (WTT);
[0058] Step 4: Build a feature fusion U-Net model based on the coordinated attention mechanism. Randomly select 50% of the training set images obtained in Step 2 and Step 3 as input to the network model to perform global feature extraction and fusion.
[0059] Step 5: Use the Residual Model module, and add squeezing and excitation (SE) modules after some residual blocks to enable the model to learn the weight coefficients of each channel autonomously;
[0060] Step 6: Embed a Coordinating Attention (CA) module at the jump connection to eliminate redundant information;
[0061] Step 7: A Spatial Pyramid structure with dilated convolutions (ASPP) is used as the bottom layer network to increase the receptive field and extract more extensive feature information.
[0062] Step 8: Input the selected test set into the trained U-Net model based on the coordinated attention mechanism for testing, and obtain the network's segmentation results.
[0063] In this embodiment, the specific data augmentation operation in step 2 includes randomly cropping an image whose size is between 0.2 and 0.4 times the original image size, with a cropping aspect ratio of 1:2, then stitching the cropped images together in pairs, and reshaping them into a 256×256×3 format.
[0064] In this embodiment, the wavelet threshold function proposed in step 3 introduces variables to make the function continuously differentiable and relatively smooth at the threshold. The wavelet threshold function is as follows:
[0065]
[0066] Here, m and k are variables, m∈[0,1], and k is a positive integer.
[0067] In this embodiment, as Figure 2 As shown, the specific steps of step 4 are as follows:
[0068] (1) The U-Net model based on coordinated attention mechanism for feature fusion employs 17 convolutional layers, 4 downsampling layers, 4 upsampling layers, and 4 cropping and copying layers. The input image format is 256×256×3. Each convolutional layer forms a residual block in pairs. Each convolutional layer uses a 3×3 kernel, with the activation function being swish and zero-padding. A squeeze activation module is placed after the residual blocks to improve the model's discriminative power for features. The downsampling layers use max pooling with a 2×2 sampling window and a stride of 2. A pyramid structure with dilated convolutions is placed at the bottom of the network to increase the receptive field. The upsampling layers use inverted convolutions with a 3×3 kernel, a stride of 2, and zero-padding. The coordinated attention module is embedded in skip connections to eliminate redundant features. Finally, a 1×1 convolutional layer reduces the dimensionality of the image and outputs the segmentation result.
[0069] (2) Randomly select 50% of the training set images obtained in steps 2 and 3 as the input of the network, that is, fuse the high-dimensional features of the denoised image and the high-dimensional features of the original image.
[0070] In this embodiment, as Figure 3 and 4 As shown, the specific steps for step 5 are as follows:
[0071] (1) Perform convolution operation Ftr: Given an input F∈R H*W*C W and H are the width and height of the input matrix, and C is the number of feature channels of the input matrix. A simple convolution operation is performed, which is the residual network, and the Swish activation function is used instead of the ReLU activation function.
[0072] (2) Perform Fsq compression operation, as shown in Formula 2: the feature map is compressed along the spatial dimension, that is, global average pooling is performed to obtain a 1×1×C (C is the number of feature channels) feature matrix.
[0073]
[0074] (3) Perform the Fex activation operation and assign different weights W according to the different importance of the features, as shown in formula (3):
[0075] s = F ex (z,W)=σ(W2δ(W1z)) (3)
[0076] In the formula: z describes the character of the channel, s is the weight used to reflect the feature map in tensor U, δ is the Swish activation function, σ is the Sigmoid activation function, and W1 and W2 are the weights of the fully connected layer used for dimensionality increase and decrease.
[0077] (4) The final weight recalibration operation transforms the two-dimensional matrix u c Multiply each value by s c As shown in formula (4):
[0078]
[0079] In the formula: s c F represents the weight of the c-th feature map in tensor U. scale Indicator quantity s c With feature mapping u c The channel product.
[0080] In this embodiment, as Figure 5 As shown, the specific steps for step 6 are as follows:
[0081] (1) Perform convolution operation Ftr: Given an input F∈R H*W*C W and H are the width and height of the input matrix, and C is the number of feature channels of the input matrix. A simple convolution operation is performed, which is the residual network processing.
[0082] (2) Perform Fsq compression operation, as shown in Formulas 5 and 6: the feature map is compressed in the horizontal and vertical directions, that is, convolution operation with kernels of 1×W and H×1 is performed to obtain feature matrices of H×1×C and 1×W×C (C is the number of feature channels).
[0083]
[0084]
[0085] (3) Perform F ex The activation operation assigns different weights W according to the different importance of the features, as shown in formula (7):
[0086] S h =F ex (U,W)=σ(g(u,W))=σ(W2δ(W1)) (7)
[0087] Sw =F ex (U,W)=σ(g(u,W))=σ(W2δ(W1)) (8)
[0088] In the formula: U describes the character of the channel, S h and S w These are the weights used to reflect the horizontal and vertical feature maps in tensor U, where δ is the Swish activation function, σ is the Sigmoid activation function, and W1 and W2 are the weights used by the fully connected layer for dimensionality increase and decrease.
[0089] (4) The final weight recalibration operation transforms the two-dimensional matrix u c Multiply each value by s c As shown in formula (8):
[0090]
[0091]
[0092] In the formula: s h and s w F represents the weights of the horizontal and vertical feature maps in tensor U. scale Indicator quantity s c With feature mapping u c The channel product.
[0093] In this embodiment, as Figure 6 As shown in step 7, the dilated convolution in the pyramid structure with dilated convolution is decomposed into two dilated convolutions with different dilation rates, which improves the sensitivity to feature extraction. Furthermore, each layer is tightly connected to the previous layer for feature sharing.
[0094] The effects of the present invention will be further described below with reference to experiments:
[0095] The experiment used the European Space Agency's oil spill detection dataset, and tested 250 randomly selected images with the network model of the training unit, calculated the MIOU and Precision values, and compared them.
[0096] Table 1. MIOU and Precision values for different network structures
[0097]
[0098] As can be seen from Table 1, the addition of different modules still has an impact on the experimental results. The method of combining multiple modules in this paper achieves the best results, with a detection accuracy of 71.7%.
[0099] The above descriptions are merely embodiments of the present invention, and common knowledge regarding specific structures and characteristics of the solutions is not described in detail here. It will be apparent to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the present invention is defined by the appended claims rather than the foregoing description. Therefore, it is intended that all variations falling within the meaning and scope of equivalents of the claims be included within the present invention. No markings in the claims should be construed as limiting the scope of the claims.
Claims
1. A method for detecting oil spills in SAR images, characterized in that, Includes the following steps: Step 1: Select images from the existing oil spill detection database as the training set; Step 2: Preprocess the training set selected in Step 1, that is, randomly crop and splice the training set. Step 3: Denoise the training set obtained in Step 2 using wavelet threshold transformation; Step 4: Build a feature fusion U-Net model based on the coordinated attention mechanism. Randomly select 50% of the training set images obtained in Step 2 and Step 3 as the input of the network model to perform global feature extraction and fusion. Step 5: Use the residual module, and add squeezing and excitation modules after some residual blocks to enable the model to learn the weight coefficients of each channel autonomously; Step 6: Embed a coordinated attention mechanism module at the jump connection to eliminate redundant information; Step 7: A spatial pyramid structure with dilated convolutions is used as the bottom layer network to increase the receptive field and extract a wider range of feature information. Step 8: Input the selected test set into the trained U-Net model based on the coordinated attention mechanism for testing, and obtain the network's segmentation results; The specific process of step 1 is as follows: select 750 images from the European Space Agency oil spill detection dataset as the training set, 250 images as the test set, and then randomly select 90% of the training set for experimental training and 10% of the training set for experimental verification. The wavelet threshold function proposed in step 3 introduces variables to make the function continuously differentiable and relatively smooth at the threshold. The wavelet threshold function is as follows: （1） Here, m and k are variables, m∈[0,1], and k is a positive integer.
2. The SAR image oil spill detection method according to claim 1, characterized in that, The specific process of step 2 is as follows: preprocess the training set selected in step 1, that is, randomly crop and stitch the training set: the original SAR image format is 1250×650, and the size of the preprocessed dataset image is reshaped to 256×256 with 3 feature channels. Specific data augmentation operations include randomly cropping images to a size between 0.2 and 0.4 times the original image size, cropping aspect ratios of 1:2, then stitching the cropped images together in pairs, and reshaping them into a 256×256×3 format.
3. The SAR image oil spill detection method according to claim 2, characterized in that, The specific process of step 4 is as follows: 4.1 The U-Net model based on coordinated attention mechanism for feature fusion employs 17 convolutional layers, 4 downsampling layers, 4 upsampling layers, and 4 cropping and copying layers. The input image format is 256×256×3. Each pair of convolutional layers forms a residual block. Each convolutional layer uses a 3×3 kernel, with the activation function being swish and zero-padding. The squeeze activation module is placed after the residual block to improve the model's discriminative power for features. The downsampling layer uses max pooling with a sampling window size of 2×2 and a stride of 2. A pyramid structure with dilated convolutions is placed at the bottom of the network to increase the receptive field. The upsampling layer uses inverted convolutions with a kernel size of 3×3 and a stride of 2, and zero-padding. The coordinated attention module is embedded in skip connections to eliminate redundant features. Finally, a 1×1 convolutional layer reduces the dimensionality of the image and outputs the segmentation result. 4.2 Randomly select 50% of the training set images obtained in steps 2 and 3 as the input to the network, that is, fuse the high-dimensional features of the denoised image with the high-dimensional features of the original image.
4. The SAR image oil spill detection method according to claim 3, characterized in that, The specific process of step 5 is as follows: 5.1 Performing Convolution Operations Ftr: Given an input , Given the width and height of the input matrix, and C as the number of feature channels in the input matrix, a simple convolution operation is performed, i.e., a residual network. 5.2 Perform Fsq compression, as shown in Equation 2: the feature map is compressed along the spatial dimension, i.e., global average pooling is performed to obtain 1×1× The feature matrix; （2） 5.3 Perform the Fex activation operation and assign different weights W according to the different importance of the features, as shown in formula (3): （3） In the formula: Characters describing the channel, It is used to react to tensors Weights of the feature map in the middle, For Swish activation function, It is the Sigmoid activation function. , These are the weights used in the fully connected layer for dimensionality enhancement and reduction. 5.4 The final weight recalibration operation transforms the two-dimensional matrix... Multiply each value by As shown in formula (4): （4） In the formula: For tensor The weight of the c-th feature map in the middle. Indicator quantity With feature mapping The channel product.
5. The SAR image oil spill detection method according to claim 4, characterized in that, The specific process for step 6 is as follows: 6.1 Performing Convolution Operations Ftr: Given an input , Let C be the width and height of the input matrix, and C be the number of feature channels of the input matrix. Then, a simple convolution operation is performed, which is the residual network processing. 6.2 Perform Fsq compression, as shown in formulas 5 and 6: the feature map is compressed in the horizontal and vertical directions, i.e., by performing convolution operations with kernels of 1×W and H×1 to obtain H×1× and 1×W× The feature matrix; （5） （6） 6.3 Perform the Fex activation operation and assign different weights W according to the different importance of the features, as shown in formulas (7) and (8): （7） （8） In the formula: Characters describing the channel, and It is used to react to tensors Weights of horizontal and vertical feature maps in the middle. For Swish activation function, It is the Sigmoid activation function. , These are the weights used in the fully connected layer for dimensionality enhancement and reduction. 6.4 The final weight recalibration operation.
6. The SAR image oil spill detection method according to claim 5, characterized in that, In the pyramid structure with dilated convolution shown in step 7, the dilated convolution is decomposed into two dilated convolutions with different dilation rates, which improves the sensitivity to feature extraction. Furthermore, each layer is tightly connected to the previous one for feature sharing.