Remote Sensing Recognition Model and Method for Floating Ulva prolifera Green Tide Integrating Multi-Attention Mechanism Neural Networks

Abstract

This invention belongs to the field of marine remote sensing technology and relates to a remote sensing identification model and method for floating seaweed (Ulva prolifera) green tides that integrates a multi-attention mechanism neural network. The model includes an encoder and a decoder; the encoder consists of five consecutive convolutional blocks, each consisting of two 3×3 convolutional layers, a batch normalization layer, and an activation function, followed by a max-pooling layer; the decoder uses progressive upsampling to restore the spatial resolution of the image; each decoding layer is fused with the corresponding encoding layer via skip connections to process feature maps at different scales, and outputs the final segmentation result after upsampling and convolution operations; a multi-attention weight enhancement module is provided at the back end of each decoding layer to enhance the expressive power of the feature maps. This invention maintains high accuracy in Ulva prolifera detection even in complex backgrounds and performs better in Ulva prolifera feature extraction and segmentation.

Description

Technical Field

[0001] This invention belongs to the field of marine remote sensing technology, and relates to a remote sensing identification model and method for floating seaweed green tides that integrates a multi-attention mechanism neural network. Background Technology

[0002] Ulva prolifera is a large planktonic algae belonging to the family Ulvaceae in the phylum Chlorophyta. Green tides caused by the explosive proliferation of large marine green algae have become a common marine ecological phenomenon. While Ulva prolifera itself is non-toxic, its abnormal proliferation can affect the survival of aquatic organisms, deteriorate the aquatic environment, and severely impact marine fisheries and coastal tourism. Ulva prolifera grows extensively at sea, and its spectral characteristics show absorption valleys in the blue and red light channels, while reflection peaks appear in the green and near-infrared channels. Therefore, extracting information about green tides of Ulva prolifera on the sea surface using the differences in its spectral characteristics is a basis for optical remote sensing image segmentation.

[0003] Traditional remote sensing monitoring of sea surface algae blooms often employs index methods and machine learning to extract information from remote sensing images. Spectral index methods (such as the Normalized Difference Vegetation Index) are simple to calculate, but rely on set thresholds, and the accuracy of the results is easily affected by environmental factors and satellite imaging conditions. This necessitates constructing different index variants for different situations, or combining environmental information to improve the extraction accuracy of algae blooms. Machine learning methods (such as random forests) can integrate and utilize multiple spectral and image features, but feature construction and selection are cumbersome, classification accuracy is easily affected, and satisfactory performance is difficult to achieve when processing complex images.

[0004] Deep learning methods have achieved significant success in extracting features from remote sensing images for image segmentation. By iteratively training and optimizing network parameters, model accuracy is improved, and with sufficient training, deep learning models exhibit good generalization ability. Deep learning models, including Convolutional Neural Networks (CNNs) and U-Net, have been widely applied to semantic segmentation tasks related to *Ulva prolifera* (seaweed) tides, greatly enhancing the processing power and monitoring accuracy of remote sensing images. However, extracting high-precision information on *Ulva prolifera* tides still faces several challenges. The significant differences in morphology and density of *Ulva prolifera* when it aggregates on the sea surface result in distinct spectral reflectance characteristics in the images. Furthermore, the complex and difficult-to-distinguish morphology of *Ulva prolifera* patches contributes to insufficient model generalization ability and low accuracy. Summary of the Invention

[0005] To address the challenge of feature extraction from complex *Ulva prolifera* species, this invention proposes an attention mechanism—Scale-Aware Channel Attention (SACA)—that automatically learns and dynamically adjusts the importance of features at different scales. This mechanism enables adaptive selection of receptive fields at different scales and enhances the model's ability to model multi-scale information. SACA is combined with efficient attention mechanism feature processing modules such as Attention Gating (AG) and Convolutional Block Attention (CBAM) to construct a multi-attention weight enhancement module. This multi-attention weight enhancement module is then built within a symmetrical encoder-decoder structure to construct a novel remote sensing image segmentation model for *Ulva prolifera* green tides, enabling the extraction of *Ulva prolifera* distribution areas.

[0006] The technical solution provided by this invention is: a remote sensing recognition model for floating seaweed green tides that integrates a multi-attention mechanism neural network, including an encoder and a decoder; the encoder consists of five consecutive convolutional blocks, each of which consists of two 3×3 convolutional layers, a batch normalization layer, and an activation function, followed by a max pooling layer; the decoder uses a progressive upsampling operation to restore the spatial resolution of the image; each decoding layer is fused with the corresponding encoding layer through skip connections to process feature maps of different scales, and outputs the final segmentation result after upsampling and convolution operations; the multi-attention weight enhancement module integrates attention gates, convolutional block attention, and scale-aware channel attention mechanisms.

[0007] Preferably, a multi-attention weight enhancement module is provided at the back end of each decoding layer to enhance the expressive power of the feature map. The fused feature map of each decoding layer is sequentially passed through an attention gate, a sigmoid activation function, and a scale-aware channel attention mechanism to calculate the intermediate process feature map. This feature map is then passed through a softmax activation function and a convolutional block attention mechanism to obtain a new feature map. The new feature map is concatenated with the intermediate process feature map in the channel dimension and input into a 1×1 convolutional layer to obtain the multi-attention weight enhancement feature map. After upsampling, the multi-attention weight enhancement feature map is fused with the feature map of the corresponding encoding layer in the next layer and used as the input fused feature of the next decoding layer.

[0008] Preferably, the scale-aware channel attention mechanism specifically includes: input features First, multi-scale features are extracted using parallel 1×1, 3×3, and 5×5 convolutions. , and The formula is:

[0009] ;

[0010] in, , , These represent 1×1, 3×3, and 5×5 convolutions, respectively, to obtain features within different receptive fields. Represents input features;

[0011] Global average pooling is performed on the convolutional outputs at each scale to compress the spatial dimension into channel-dimensional features. , and This enables encoding of the global context at various scales, using the following formula:

[0012] ;

[0013] in, Indicates to Calculate the output features after global pooling. , This represents global pooling; the pooled features from the three scales are stacked along the channel dimension into a multi-scale feature matrix, and then the importance weights of the multi-scale features are calculated using one-dimensional convolution and the softmax activation function. The features at each scale are then weighted and summed according to the weights to obtain the fused multi-scale feature representation. This achieves dynamic weighted fusion of convolutional features at various scales, as shown in the following formula:

[0014] ;

[0015] in, The weights represent the importance of multi-scale features. For activation function, Represents one-dimensional convolution. This indicates stacking along the channel dimension;

[0016] ;

[0017] in, This indicates the fusion of multi-scale features;

[0018] A channel attention weight matrix A is generated by mapping back to the input channel dimension using one-dimensional convolution and a sigmoid activation function.

[0019] Input features Multiply by the channel attention matrix A, using the following formula:

[0020] ;

[0021] in, This represents the channel attention weight matrix, and sigmoid represents the sigmoid activation function.

[0022] ;

[0023] in, This represents the final output feature.

[0024] Preferably, a high-frequency enhancement module is provided at the input end of the third convolutional block of the encoder. This module uses the Laplacian operator to convolve the feature map, extracts high-frequency information, and enhances edge perception by adding it to the original features.

[0025] The present invention also provides a remote sensing identification method for floating green tides that integrates a multi-attention mechanism neural network. This method uses the above-mentioned model to segment the input remote image and realize the distribution identification of floating green tides.

[0026] The floating seaweed green tide remote sensing identification model and method provided by this invention, which integrates a multi-attention mechanism neural network, can maintain high seaweed detection accuracy even in complex backgrounds and performs better in seaweed feature extraction and segmentation. Attached Figure Description

[0027] Figure 1 This is a network structure diagram of the remote sensing identification model for floating seaweed green tides that integrates a multi-attention mechanism neural network in an embodiment of the present invention;

[0028] Figure 2 This is a schematic diagram of the algorithm flow of the multi-attention weight enhancement module in an embodiment of the present invention;

[0029] Figure 3 This is a flowchart of the SACA attention mechanism in an embodiment of the present invention;

[0030] Figure 4 This is a comparison of the results of Ulva prolifera image segmentation using different methods in an embodiment of the present invention. Detailed Implementation

[0031] To facilitate understanding of the present invention, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and specific examples. The following examples or drawings are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0032] Example 1 This example proposes a remote sensing identification model for floating seaweed green tides that integrates a multi-attention mechanism neural network, such as... Figure 1 As shown, the model consists of the following parts:

[0033] 1. The encoder consists of five consecutive convolutional blocks (E1 to E5). Each block comprises two 3×3 convolutional layers, a batch normalization layer, and an activation function (ReLU), followed by a max pooling layer. By progressively extracting deep features from the image and reducing spatial resolution through pooling operations, the output size of each layer gradually decreases while the number of channels gradually increases, thereby capturing multi-layered semantic information.

[0034] To enhance the perception of edges and texture details, this invention adds a high-frequency enhancement module to the input of the E3. This module uses the Laplacian operator to convolve the feature map, extracts high-frequency information, and adds it to the original features to significantly enhance edge perception. The specific formula is as follows:

[0035] (1);

[0036] in, This represents the feature map output after passing through the high-frequency enhancement module. To input the feature map of the high-frequency enhancement module, It is the enhancement coefficient, which controls the intensity of the enhancement effect. Representing the Laplacian transform, it measures the difference between the gray value at a point in an image and the average gray value of its surrounding neighborhood. The core principle behind the high-frequency enhancement method's ability to significantly enhance edge perception lies in the Laplacian operator's extremely high sensitivity to regions in an image where gray values ​​change drastically (i.e., edges). It can accurately locate and highlight these regions, thereby achieving a significant edge enhancement effect.

[0037] The output feature map of the encoder's last layer E5 passes through the Swing Transformer module, which captures global features in the image and enhances long-range dependencies, thereby improving the ability to model the target.

[0038] 2. In the decoder section, the spatial resolution of the image is restored through progressive upsampling. Each decoding layer is fused with its corresponding encoding layer via skip connections (channel concatenation fusion) to obtain a fused feature map. To preserve detailed information, layers D1, D2, D3, and D4 process feature maps of different scales during decoding, and output the final segmentation result after upsampling and convolution operations.

[0039] For each decoding layer backend, a multi-attention weight enhancement module is constructed by integrating AG, SACA, and CBAM attention mechanisms to enhance the expressive power of the feature map.

[0040] The algorithm flow of the multi-attention weight enhancement module is as follows: Figure 2As shown, the fused feature map of each decoding layer First, the feature maps of the intermediate processes are calculated sequentially through the AG attention mechanism, the sigmoid activation function, and the SACA attention mechanism. The feature map is then processed using the softmax activation function and the CBAM attention mechanism to obtain a new feature map. The new feature map is concatenated with the intermediate feature map through channel dimension and input into a 1×1 convolutional layer to obtain a multi-attention weighted feature map. This feature map is then upsampled. This will be input into the next decoder layer.

[0041] The feature processing procedure is as follows:

[0042] (2);

[0043] (3);

[0044] (4);

[0045] in, Using the input features as input, the enhanced feature map is obtained after passing through the multi-attention weight enhancement module. ; This indicates that attention gates are used to extract features. express Activation function; This represents channel attention applied to scale perception. express Activation function; This indicates the application of convolutional block attention; This indicates a splicing operation along the channel dimension; This indicates that a 1×1 kernel is used for convolution operation to fuse feature maps; This indicates an upsampling operation.

[0046] In the multi-attention weight enhancement module, the SACA mechanism proposed in this invention is as follows: Figure 3 As shown, the specific description is as follows: Input features First, multi-scale features are extracted using parallel 1×1, 3×3, and 5×5 convolutions. , and The formula is:

[0047] (5);

[0048] in, , , These represent 1×1, 3×3, and 5×5 convolutions, respectively, to obtain features within different receptive fields. This represents the input features.

[0049] Then, global average pooling is performed on the convolutional outputs at each scale to compress the spatial dimension into channel-dimensional features. , and This enables encoding of the global context at various scales, using the following formula:

[0050] (6);

[0051] in, Indicates to Calculate the output features after global pooling. , This indicates global pooling.

[0052] Next, the pooled features from the three scales are stacked along the channel dimension into a multi-scale feature matrix, and then the importance weights of the multi-scale features are calculated using one-dimensional convolution and softmax activation functions. The features at each scale are then weighted and summed according to this weight to obtain the fused multi-scale feature representation. This enables dynamic weighted fusion of convolutional features at various scales. The formula is as follows:

[0053] (7);

[0054] in, The weights represent the importance of multi-scale features. For activation function, Represents one-dimensional convolution. This indicates stacking along the channel dimension;

[0055] (8);

[0056] in, This represents the fusion of multi-scale features. Finally, a one-dimensional convolution and a sigmoid activation function are used to map back to the input channel dimension, generating a channel attention weight matrix A.

[0057] Finally, input features Multiplying with the channel attention matrix A enables channel-level adaptive recalibration, thereby enhancing the response of important channels and suppressing redundant information. The specific formula is as follows:

[0058] (9);

[0059] in, represents the channel attention weight matrix, and sigmoid represents the Sigmoid activation function.

[0060] (10);

[0061] in, This represents the final output feature.

[0062] In summary, the model proposed in this invention is based on an encoding and decoding structure, integrates multiple attention mechanisms and Swing Transformer global modeling, which makes the model perform well in terms of segmentation accuracy and robustness, and can effectively improve the image segmentation effect of Ulva prolifera in remote sensing images.

[0063] Example 2 To evaluate the performance of the model proposed in this invention, this example compares the model accuracy and performance of different fusion methods of various attention mechanisms. Using the Gaofen-1 satellite data on the green tide of Ulva prolifera, the experimental results under the same training configuration are shown in Table 1.

[0064] Table 1. Comparative Experiments on the Fusion of Different Attention Mechanisms

[0065] .

[0066] The results in Table 1 show that the accuracy, mIou, and F1 score without the attention mechanism were 96.50%, 79.92%, and 88.37%, respectively. Using the CBAM attention mechanism improved the accuracy to 98.40%, increased mIou by 7.16%, and improved the F1 score by 4.19%, indicating that CBAM effectively enhances feature representation capabilities. The model using both AG and CBAM attention mechanisms showed a slight decrease in mIou and F1 score, but reduced computational cost, indicating that AG improved feature utilization in feature selection, but had a certain impact on the final segmentation accuracy. In the CBAM+AG+SACA+U-Net model, three attention mechanisms, AGate, CBAM and SACA, are simultaneously integrated, achieving a mIou of 91.77% and an F1 score of 95.40%, which are 11.85% and 7.03% higher than those without attention mechanisms, respectively. This verifies the effectiveness of the multi-attention weight enhancement module constructed in this invention in capturing information about seaweed in remote sensing images. The proposed SACA module significantly improves mIou (5.48%) after integration.

[0067] Example 3: Application examples of the model and method proposed in this invention

[0068] (1) Study Area: The experimental area for remote sensing image segmentation of Ulva prolifera in this embodiment is located in the southern Yellow Sea of ​​China, near the coastal area of ​​Jiangsu Province, China, between approximately 32°40′~34°10′ north latitude and 120°30′~121°25′ east longitude. The hydrodynamic conditions in the study area are complex, mainly affected by tides, coastal currents and the runoff of the Yangtze River, which easily leads to upwelling and eutrophication, providing a suitable environment for the growth, reproduction and drift of green algae such as Ulva prolifera.

[0069] (2) Remote sensing data: Gaofen-1 (GF-1) is the first satellite of China's "High-Resolution Earth Observation System" national major science and technology project, launched in 2013. Its remote sensing images have high spatial resolution, wide coverage and multispectral capabilities, and are widely used in agriculture, forestry, environmental protection, urban planning and disaster monitoring. The multispectral images (MS) acquired by the PMS sensor on GF-1 include four channels: red, green, blue and near-infrared, with a spatial resolution of 8 meters, a revisit period of 4 days and a swath width of 800 kilometers.

[0070] (3) Data Processing: First, multispectral images of the Jiangsu coastal area on June 3, 2023, were acquired. To ensure data accuracy, the following data preprocessing was performed: First, the original image (L1A) was radiometrically calibrated, and the DN values ​​were converted into apparent reflectance to eliminate sensor response differences. Then, atmospheric correction was performed based on the MODTRAN model to remove atmospheric scattering and absorption effects and obtain the true surface reflectance. After that, geometric correction, image registration, image stitching, and land masking were performed on the atmospherically corrected images. The remote sensing images of each region were cropped to 128×128 pixels to fit the size of the model input. A dataset of 2229 images of *Ulva prolifera* green tide was obtained through screening. In order to train and evaluate the model, the dataset was divided into training set, validation set, and test set in a ratio of 5:3:2, containing 1114, 669, and 446 images, respectively. During training, various data augmentation methods were employed to improve the model's generalization ability, including random horizontal flipping, random vertical flipping, random 90° rotation, random rotation, random brightness and contrast adjustment, and random gamma adjustment.

[0071] (4) Model Training: 1114 128×128 images of *Ulva prolifera* were input into the deep learning model constructed in this invention, which incorporates a multi-attention mechanism. After extensive experiments, the number of training iterations was determined to be 100, and the optimizer was Adam. The initial learning rate was dynamically adjusted using cosine annealing and linear warm-up strategies to ensure stability in the early stages of training and sufficient convergence in the later stages. In addition, a foreground region oversampling strategy was adopted during the training phase to alleviate the underfitting problem of sparse foreground targets. After each round of training, the average intersection-union ratio (mIou), F1 score (F1), and pixel accuracy were calculated using the validation set, and the optimal model was automatically saved based on the mIou accuracy of the validation set.

[0072] (5) Model Application: The model was used to segment images of seaweed on the test sample set and its accuracy was evaluated. The accuracy of this model was 99.08%, mIou=91.77%, and F1=95.4. The model has high test accuracy, excellent generalization ability, and good seaweed segmentation results. Figure 4 As shown, in the segmentation results, red and green rectangles mark typical areas to highlight the differences in segmentation of *Ulva prolifera* patches using different models. From... Figure 4 As can be seen, Model 2 (CBAM+U-Net) has a significant improvement over U-Net in small target extraction; in the results of Model 3 (CBAM+Attention Gate+U-Net), the background false detection is significantly reduced and the boundaries are smoother; the segmentation results of Model 4 (the present invention model integrating CBAM+Attention Gate+SACA) show that it can still maintain high accuracy in detecting seaweed in complex backgrounds, and performs better in terms of seaweed feature extraction and segmentation.

Claims

1. A remote sensing identification model for floating seaweed green tides integrating multi-attention mechanism neural networks, characterized by: The system includes an encoder and a decoder. The encoder consists of five consecutive convolutional blocks, each consisting of two 3×3 convolutional layers, a batch normalization layer, and an activation function, followed by a max-pooling layer. The decoder uses progressive upsampling to restore the spatial resolution of the image. Each decoding layer is fused with the corresponding encoding layer via skip connections to process feature maps at different scales. After upsampling and convolution, the final segmentation result is output. A multi-attention weight enhancement module is provided at the back end of each decoding layer to enhance the expressive power of the feature maps. The multi-attention weight enhancement module integrates attention gates, convolutional block attention, and scale-aware channel attention mechanisms. The scale-aware channel attention mechanism specifically comprises: input features First, multi-scale features are extracted through parallel 1x1, 3x3, 5x5 convolutions 、 and , and the formula is: ； in, , , These represent 1×1, 3×3, and 5×5 convolutions, respectively, to obtain features within different receptive fields. Represents input features; Global average pooling is performed on the convolutional outputs at each scale to compress the spatial dimension into channel-dimensional features. , and This enables encoding of the global context at various scales, using the following formula: ； in, Indicates to Calculate the output features after global pooling. , Indicates global pooling; The pooled features at three scales are stacked along the channel dimension to form a multi-scale feature matrix. Then, the importance weights of the multi-scale features are calculated using one-dimensional convolution and softmax activation functions. The features at each scale are then weighted and summed according to the weights to obtain the fused multi-scale feature representation. This achieves dynamic weighted fusion of convolutional features at various scales, as shown in the following formula: ； in, The weights represent the importance of multi-scale features. For activation function, Represents one-dimensional convolution. This indicates stacking along the channel dimension; ； in, This indicates the fusion of multi-scale features; A channel attention weight matrix A is generated by mapping back to the input channel dimension using one-dimensional convolution and a sigmoid activation function. Input features Multiply by the channel attention matrix A, using the following formula: ； in, This represents the channel attention weight matrix, and sigmoid represents the sigmoid activation function. ； in, Indicates the final output features; Multispectral images of the nearshore area were acquired, and the remote sensing images of each region were cropped into 128×128 images and used as training samples for the *Ulva prolifera* images. The number of training iterations was determined experimentally. During the training phase, the initial learning rate was dynamically adjusted in conjunction with cosine annealing and linear preheating strategies. A foreground region oversampling strategy was adopted. After each round of training, the average intersection-union ratio, F1 score, and pixel accuracy were calculated using the validation set, and the optimal model was automatically saved based on the mIou accuracy of the validation set.

2. The remote sensing identification model for floating seaweed green tides fused with a multi-attention mechanism neural network according to claim 1, characterized in that, The fused feature map of each decoding layer passes through an attention gate, a sigmoid activation function, and a scale-aware channel attention mechanism in sequence to calculate the intermediate feature map. This feature map is then passed through a softmax activation function and a CBAM attention mechanism to obtain a new feature map. The new feature map is then concatenated with the intermediate feature map in the channel dimension and input into a 1×1 convolutional layer to obtain a multi-attention weight-enhanced feature map. The multi-attention weight-enhanced feature map is upsampled and then fused with the feature map of the corresponding coding layer in the next layer, and then used as the input fused feature of the next decoding layer.

3. The remote sensing identification model for floating seaweed green tides fused with a multi-attention mechanism neural network according to claim 1, characterized in that, A high-frequency enhancement module is provided at the input of the third convolutional block of the encoder. This module uses the Laplacian operator to convolve the feature map, extracts high-frequency information, and enhances edge perception by adding it to the original features.

4. A remote sensing method for identifying floating seaweed green tides by integrating a multi-attention mechanism neural network, characterized in that, Using the model described in any one of claims 1-3, the input remote image is segmented to identify the distribution of floating seaweed green tides.

Images