Marine mesoscale vortex segmentation method based on edge attention

Abstract

The invention discloses an ocean mesoscale vortex segmentation method based on edge attention, and relates to the field of semantic segmentation. The problems that an existing model is poor in vortex edge information extraction effect, and the shape and the position of the segmented ocean mesoscale vortex are not accurate enough are solved. The method specifically comprises the following steps of (1) obtaining sea level abnormal data SLA, performing visualization processing on the SLA to generate a sea mesoscale vortex image data set, performing labeling, dividing image data into a training set, a verification set and a test set, and performing preprocessing by using image enhancement operation; (2) creating a network model, inputting the preprocessed ocean mesoscale vortex image data set into an ocean mesoscale vortex segmentation network based on edge attention, namely an EGA-AFFUNet model, for training, and segmenting ocean mesoscale vortexes in the ocean mesoscale vortex image data set; and comparing a test result with the vortex label data set on the test set to obtain an evaluation index.

Description

Technical Field

[0001] The present invention relates to the field of semantic segmentation, and in particular to an ocean mesoscale eddy segmentation method based on edge attention. Background Art

[0002] Marine science is the science that studies the largest and most biodiverse ecosystems on the Earth's surface. Understanding the distribution characteristics of mesoscale eddies in the ocean is of great significance for studying ocean circulation, climate change, and ecosystems. Mesoscale eddies are a complex fluid dynamics phenomenon, and their generation, development, and dissipation are affected by many factors, such as wind fields, ocean currents, and temperature. Therefore, the detection and analysis of mesoscale eddies are challenging.

[0003] Traditional ocean mesoscale eddy detection methods have low detection accuracy or lack of general capabilities. Most traditional ocean mesoscale eddy detection methods are based on the physical or geometric characteristics of ocean mesoscale eddies. These detection methods based on artificially designed ocean mesoscale eddy characteristics are difficult to accurately describe the complex characteristics of ocean mesoscale eddies. In addition, since traditional ocean mesoscale eddy detection methods require experts to set physical parameter thresholds on the premise of fully understanding the mechanism of ocean remote sensing imaging, however, due to the complexity of ocean mesoscale eddies and the limitations of human understanding of them, the physical threshold methods set by experts are often only applicable to the identification of ocean mesoscale eddies in specific sea areas, and it is difficult to meet the requirements for automatic detection of ocean mesoscale eddies in other sea areas, and lack general capabilities.

[0004] The existing deep learning model for ocean mesoscale eddy segmentation using sea level anomaly data does not extract the edge information of ocean mesoscale eddies well. The convolutional neural network model with poor spatial information recovery will destroy its original spatial resolution when automatically segmenting ocean mesoscale eddies, resulting in the shape and position of the segmented ocean mesoscale eddies becoming inaccurate. Summary of the invention

[0005] The purpose of the present invention is to solve the problems mentioned above in the existing methods for detecting ocean mesoscale eddies based on deep learning, such as the failure to extract the boundary information of ocean mesoscale eddies well, the large amount of computation, the influence of physical parameters, and the lack of general capabilities of traditional ocean mesoscale eddy detection methods. The present invention provides an ocean mesoscale eddy method based on edge attention, which uses an encoder-decoder structure based on residual blocks as a model reference network, in conjunction with an edge-guided attention module (EGA) and a feature fusion module (AFF), which not only enhances the segmentation effect of the model on vortex edge information, but also effectively extracts multi-scale vortex information, fuses high-level semantic information with low-level semantic information, and improves the accuracy of vortex segmentation.

[0006] The technical solution of the present invention is as follows: a method for segmenting ocean mesoscale eddies based on edge attention, characterized in that it comprises the following steps:

[0007] Step 1: Obtain sea level anomaly data SLA with a temporal resolution of 1d and a spatial resolution of 0.25°, visualize SLA to generate an ocean mesoscale eddy image dataset and annotate it, divide the image dataset into training set, validation set, and test set according to the ratio of 7:2:1, and use image enhancement operations for preprocessing.

[0008] Step 2: Create a network model, input the preprocessed ocean mesoscale eddy image dataset into the edge attention-based ocean mesoscale eddy segmentation network, namely the EGA-AFFUNet model, for training, segment the ocean mesoscale eddies in the ocean mesoscale eddy image dataset, and select the EGA-AFFUNet model with the best performance;

[0009] Step 3: Perform performance tests on the selected optimal EGA-AFFUNet model on the test set, and compare the test results with the vortex label dataset to obtain evaluation indicators.

[0010] Furthermore, the latitude and longitude range of the sea level anomaly data used in step 1 is 0°N~26°N, 98°E~125°E. The ocean mesoscale eddy image dataset generated by visualization processing is interpolated to generate an image dataset with a pixel size of [330,318].

[0011] Furthermore, the preprocessing of the ocean mesoscale eddy image dataset in step 1 includes:

[0012] In the ocean mesoscale eddy image dataset, experts annotate the mesoscale eddies and generate a label dataset.

[0013] In the ocean mesoscale eddy image dataset, a mask image corresponding to the land range is generated, and the mask image is used to cover the land part in the ocean mesoscale eddy image dataset.

[0014] In the ocean mesoscale eddy image dataset, the dataset is normalized, and data enhancement operations such as horizontal flipping, random radiation transformation, and Gaussian noise are added.

[0015] Furthermore, the EGA-AFFUNet model in step 2 consists of an encoder, a decoder, and a skip connection; the encoder uses ResNet34 as the baseline network; the skip connection consists of an EGA module; the decoder includes an AFF module, linear interpolation, and a convolutional layer:

[0016] Furthermore, the encoder ResNet34 benchmark network described in step 2 specifically includes the following:

[0017] The first network consists of a convolutional layer and a maximum pooling layer. The size of the convolutional layer is 7×7, the input channel is 3, and the output channel is 64. The feature input passes through the convolutional layer and enters the maximum pooling layer with a stride of 2.

[0018] The second segment of the network consists of three residual blocks, each residual block includes two convolutional layers, the size of each convolutional layer is 3×3, the input channel of the second segment of the network is 64, and the output channel is 64.

[0019] The third network segment consists of four residual blocks, each of which includes two convolutional layers. The size of each convolutional layer is 3×3. The input channel of the third network segment is 64 and the output channel is 128.

[0020] The fourth segment of the network consists of six residual blocks, each residual block includes two convolutional layers, the size of each convolutional layer is 3×3, the input channel of the fourth segment of the network is 128, and the output channel is 256.

[0021] The fifth segment of the network consists of three residual blocks, each residual block includes two convolutional layers, the size of each convolutional layer is 3×3, the input channel of the fifth segment of the network is 128, and the output channel is 512.

[0022] Furthermore, the jump-connected EGA module described in step 2 accepts three inputs, namely, a feature map output from the corresponding layer of the encoder, an edge feature from the Laplacian operator, and a predicted feature output from the decoder at a layer higher than the corresponding layer of the encoder; the EGA module includes an edge attention feature map and a reverse attention feature map generated by the predicted features output by the decoder, two 3×3 convolutional layers, and a CBAM module.

[0023] Furthermore, the decoder part of the EGA-AFFUNet model in step 2 consists of multiple decoding modules, each of which includes an AFF module, two convolutional layers of size 3×3, a convolutional layer of size 1×1, and a linear interpolation part. The AFF module consists of two branches, the first branch contains an average pooling layer, two convolutional layers of size 1×1, and the second branch only contains two 1×1 convolutional layers.

[0024] The specific implementation of the system is the same as the method.

[0025] A computer device, characterized in that: the computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

[0026] Beneficial effects: Compared with the prior art, the present invention has the following advantages: an edge attention EGA module is added to the jump connection part between the encoder and the decoder, which enhances the segmentation effect of the model on the vortex edge information; an attention feature fusion AFF module is added to the decoder part, which effectively utilizes multi-scale feature information, fuses high-scale and low-scale semantic information, and improves the accuracy of vortex segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flow chart of the method of the present invention;

[0028] Figure 2 It is a structural diagram of the EGA-AFFUNet model of the present invention;

[0029] Figure 3 This is the encoder picture of the EGA-AFFUNet model of the present invention.

[0030] Figure 4 This is the decoder picture of the EGA-AFFUNet model of the present invention. DETAILED DESCRIPTION

[0031] The present invention will be further described with reference to the accompanying drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art may make various changes or modifications to the present invention, and these equivalent forms also fall within the scope limited by the application.

[0032] like Figure 1 As shown, the present invention implements an ocean mesoscale eddy segmentation method based on edge attention, and the main process includes the following steps:

[0033] (1) Acquire sea level anomaly data SLA with a temporal resolution of 1 d and a spatial resolution of 0.25°. The original SLA data used in the present invention is provided by the Archive Satellite Oceanography Data Center (AVISO) of the French National Center for Space Studies and downloaded by the European Copernicus Marine Environment Monitoring Service (CMEMS). The time period of the selected data starts from January 1, 2000 and ends on December 31, 2023, a total of 23 years. The original data is saved in the NetCDF file format.

[0034] (2) Visualize the acquired original SLA data, with the visualization latitude and longitude range of 0°N~26°N, 98°E~125°E. Interpolate the generated ocean mesoscale eddy image dataset to generate an image dataset with a pixel size of [330,318], and perform expert annotation on the generated ocean mesoscale eddy image dataset to generate a label dataset. Subsequently, add data enhancement operations such as horizontal flipping, random radiation transformation, and Gaussian noise to the generated ocean mesoscale eddy image dataset. Finally, generate a mask image corresponding to the land range, and use the mask image to cover the land part in the ocean mesoscale eddy image dataset.

[0035] (3) Create a network model, input the preprocessed ocean mesoscale eddy image dataset into the edge attention-based ocean mesoscale eddy segmentation network, namely the EGA-AFFUNet model, for training, segment the ocean mesoscale eddies in the ocean mesoscale eddy image dataset, and select the EGA-AFFUNet model with the best performance. Figure 2 As shown in the figure, the details are as follows: the EGA-AFFUNet model consists of an encoder, a decoder, and a jump connection; the encoder uses ResNet34 as the baseline network; the jump connection consists of an EGA module; the decoder includes an AFF module, linear interpolation, and a convolutional layer. Figure 3 As shown in FIG, the network specifically includes five segments. Except for the first segment of the network consisting of a convolutional layer and a maximum pooling layer, the other four segments of the network are composed of several residual blocks, and each residual block includes two convolutional layers.

[0036] (4) The EGA module with jump connection accepts three inputs, namely, the feature map output from the corresponding layer of the encoder, the edge feature from the Laplacian operator, and the predicted feature output from the decoder of the layer higher than the corresponding layer of the encoder; the EGA module includes the edge attention feature map and the reverse attention feature map generated by the predicted feature output by the decoder, two 3×3 convolutional layers, and a CBAM module. The specific formula is as follows:

[0037]

[0038] in, is the edge feature map from the Laplacian operator, is the output from the i-th segment of the encoder network, is the edge feature map calculated from the predicted feature map output from the decoder's i+1th network segment, is the reverse attention feature map calculated from the predicted feature map output from the decoder's i+1th network stage, and σ represents the sigmoid function.

[0039] (5) The decoder part consists of multiple decoding modules, such as Figure 4 As shown: Each decoding module includes an AFF module, two convolutional layers of size 3×3, a convolutional layer of size 1×1, and a linear interpolation part. The AFF module consists of two branches. The first branch contains an average pooling layer, two convolutional layers of size 1×1, and the second branch only contains two 1×1 convolutional layers. The specific formula is as follows:

[0040] L(X)=β(PWConv(δ(β(PWConv(X)))))

[0041] X′=X×M(X)=X×σ(L(X)+g(X))

[0042] Among them, β is the batch normalization layer, σ is the sigmoid function, δ is the ReLU activation function, and M(X) is the attention weight generated by MS-CAM.

[0043] (6) The performance of the best EGA-AFFUNet model is tested on the validation set, and the test set is input into the EGA-AFFUNet with the best performance to obtain the evaluation index. Specifically, the detection evaluation index is established, and the three evaluation indicators of accuracy, mean intersection over union (mIoU), and mean dice coefficient (mDice) are selected. The formulas of each indicator are as follows:

[0044]

[0045] Among them, TP represents the number of vortices correctly predicted by the model; TN represents the number of non-vortices correctly predicted by the model; FP represents the number of false vortices predicted by the model; and FN represents the number of non-vortices incorrectly predicted by the model. C is the total number of categories, IoU c For each category, the IoU value, Dice c Dice value for each category.

[0046] Training details: Our model uses an NVIDIA 4090 GPU during training and testing. Random flipping, random rotation, and translation transformations are used during training; the training optimizer is AdamW, where the parameter beta is set to 0.99-0.999, the learning rate is initialized to 1e-3, the cosine annealing strategy, the minimum learning rate is 1e-8, and the batch size is 8. The total number of training rounds is 100. The main loss function used is a hybrid loss consisting of cross entropy loss and Dice loss.

[0047] Matters not covered by the present invention are known technologies.

[0048] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable people familiar with the technology to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for ocean mesoscale eddy segmentation based on edge attention, characterized in that: The following steps are involved: Step 1: Obtain sea level anomaly data SLA with a temporal resolution of 1d and a spatial resolution of 0.25°, visualize SLA to generate an ocean mesoscale eddy image dataset and annotate it, divide the image dataset into training set, validation set, and test set according to the ratio of 7:2:1, and use image enhancement operations for preprocessing. Step 2: Create a network model and input the preprocessed ocean mesoscale eddy image dataset into the edge attention-based ocean mesoscale eddy segmentation network, namely the EGA-AFFUNet model, for training to segment the ocean mesoscale eddies in the ocean mesoscale eddy image dataset. Step 3: Select the EGA-AFFUNet model with the best performance on the validation set, and compare the test results with the vortex label dataset on the test set to obtain the evaluation indicators.

2. A method for segmenting ocean mesoscale eddies based on edge attention according to claim 1, characterized in that: The latitude and longitude range of the sea level anomaly data used in step 1 is 0°N~26°N, 98°E~125°E. The ocean mesoscale eddy image dataset is generated by visualization processing and interpolation is performed to generate an image dataset with a pixel size of [330, 318].

3. A method for segmenting ocean mesoscale eddies based on edge attention according to claim 1, characterized in that: The preprocessing of the ocean mesoscale eddy image dataset in step 1 includes: In the ocean mesoscale eddy image dataset, experts annotate the mesoscale eddies and generate a label dataset. In the ocean mesoscale eddy image dataset, data enhancement operations such as horizontal flipping, random radial transformation, and Gaussian noise are added. In the ocean mesoscale eddy image dataset, a mask image corresponding to the land range is generated, and the mask image is used to cover the land part in the ocean mesoscale eddy image dataset.

4. A method for segmenting ocean mesoscale eddies based on edge attention according to claim 1, characterized in that: The EGA-AFFUNet model in step 2 is composed of an encoder, a decoder, and a jump connection; the encoder uses ResNet34 as the benchmark network; the jump connection is composed of an EGA module; the decoder includes an AFF module, linear interpolation, and a convolutional layer.

5. A method for segmenting ocean mesoscale eddies based on edge attention according to claim 4, characterized in that: The encoder ResNet34 benchmark network specifically includes the following: The first network consists of a convolutional layer and a maximum pooling layer. The size of the convolutional layer is 7×7, the input channel is 3, and the output channel is 64. The feature input passes through the convolutional layer and enters the maximum pooling layer with a stride of 2. The second segment of the network consists of three residual blocks, each residual block includes two convolutional layers, the size of each convolutional layer is 3×3, the input channel of the second segment of the network is 64, and the output channel is 64. The third network segment consists of four residual blocks, each of which includes two convolutional layers. The size of each convolutional layer is 3×3. The input channel of the third network segment is 64 and the output channel is 128. The fourth segment of the network consists of six residual blocks, each residual block includes two convolutional layers, the size of each convolutional layer is 3×3, the input channel of the fourth segment of the network is 128, and the output channel is 256. The fifth segment of the network consists of three residual blocks, each residual block includes two convolutional layers, the size of each convolutional layer is 3×3, the input channel of the fifth segment of the network is 128, and the output channel is 512.

6. A method for segmenting ocean mesoscale eddies based on edge attention according to claim 4, characterized in that: The jump-connected EGA module accepts three inputs, namely, a feature map output from the corresponding layer of the encoder, an edge feature from the Laplacian operator, and a predicted feature output from the decoder at a layer higher than the corresponding layer of the encoder; the EGA module includes an edge attention feature map and a reverse attention feature map generated by the predicted features output by the decoder, two 3×3 convolutional layers, and a CBAM module.

7. A method for segmenting ocean mesoscale eddies based on edge attention according to claim 4, characterized in that: The decoder part is composed of multiple decoding modules, each decoding module includes an AFF module, two convolutional layers with a size of 3×3, a convolutional layer with a size of 1×1 and a linear interpolation part.

8. A method for segmenting ocean mesoscale eddies based on edge attention according to claim 4, characterized in that: The AFF module consists of two branches, the first branch contains an average pooling layer, two convolutional layers of size 1×1, and the second branch only contains two 1×1 convolutional layers.

9. A computer device, characterized in that: The computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

Images