Sea-land island remote sensing image segmentation method based on feature difference enhancement fusion

Abstract

The invention discloses a sea-land island remote sensing image segmentation method based on feature difference enhancement fusion, and the method comprises the steps: collecting sea-land island remote sensing image data, and constructing a sea-land island remote sensing image data set; constructing an initial sea-land island remote sensing image segmentation model; calculating a loss function value of the model, performing back propagation, and training to obtain a sea-land island remote sensing image segmentation model; and outputting a segmentation result graph based on the trained model. According to the method, a coding-decoding architecture is adopted, a difference attention fusion module is introduced in a coding stage, and the two features are fused based on complementarity of global and local features so as to enhance the expression ability of the model; a cross fusion module is introduced in the decoding stage, multi-scale feature fusion is performed on features from different coding stages, and the segmentation precision is improved; and finally, outputting a segmentation prediction result through a full connection layer, and realizing accurate training optimization in combination with a multi-term auxiliary loss function, thereby improving the efficiency and accuracy of sea-land island remote sensing image segmentation.

Description

Technical Field

[0001] The present invention relates to the technical field of smart ocean and computer vision image segmentation, and specifically to a method for segmenting sea, land and island remote sensing images based on feature difference enhancement and fusion. Background Art

[0002] Land, sea, and island segmentation involves pixel-by-pixel classification of images using semantic segmentation methods, labeling each pixel with a category label of ocean, land, or island to separate island targets from ocean and land areas. With the continuous advancement of remote sensing and computer vision technologies, the resolution of land, sea, and island remote sensing images has continued to increase, and the accuracy of computer vision image segmentation techniques on common datasets has also continued to improve, providing data and technical support for land, sea, and island segmentation. Segmenting ocean, land, and island target areas in land, sea, and island remote sensing images plays a vital role in fields such as marine monitoring and port management.

[0003] Remote sensing images of land, sea, and islands typically depict nearshore and ocean scenes. Nearshore areas are rich in features, often including oceans, islands, land, docks, ships, containers, buildings, roads, and other types of features, resulting in a high degree of complexity. Furthermore, these targets, in addition to being located close to the land-sea boundary, also share features such as color and texture that are extremely similar to those of some land targets. In ocean scenes, island targets often appear smaller in remote sensing images of land, sea, and islands, and their pixel proportion in the image is far less than that of oceans and other areas. All of these factors complicate the accurate segmentation of land and island targets. Therefore, in the segmentation of land, sea, and islands in remote sensing images of land, sea, and islands in nearshore and ocean areas, accurately extracting island targets, avoiding confusion with other features, and improving the robustness of the model in complex background areas is a key challenge in current research. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention designs a method for segmenting sea, land and island remote sensing images based on feature difference enhancement fusion, and the technical solutions adopted are as follows: S1) Collect remote sensing image data of land, sea and islands and construct a remote sensing image dataset of land, sea and islands for subsequent model training; S2) Constructing an initial land, sea, and island remote sensing image segmentation model based on the enhanced fusion of local and global features. The model structure consists of four parts: a dual-branch feature extraction module, a differential attention fusion module, a cross-fusion decoding module, and a segmentation head. S3) Inputting the sea, land, and island remote sensing image dataset into the initial sea, land, and island remote sensing image segmentation model for training, calculating the loss value based on the predicted value and the actual value, performing reverse gradient propagation, and optimizing the parameters of the initial sea, land, and island remote sensing image segmentation model using a predetermined optimizer. After multiple rounds of training, the sea, land, and island remote sensing image segmentation model is obtained; S4) Based on the trained sea, land and island remote sensing image segmentation model, the sea, land and island remote sensing image to be segmented is input, processed, and the segmentation result image is output.

[0005] Preferably, in step S2), the dual-branch feature extraction module includes two parts: a local feature extraction module and a global feature extraction module. The feature extraction process is specifically as follows: For the remote sensing images of sea, land and islands input into the dual-branch feature extraction module, they are processed in parallel by the local feature extraction module and the global feature extraction module respectively; The local feature extraction module consists of two parts: the ResNet backbone network and the FCN decoding head. The ResNet backbone network consists of four serially connected ResNet layers, which use the ResNet50 network. The FCN decoding head uses a three-layer MLP network. The global feature extraction module consists of two parts: the Transformer backbone network and the SegFormer decoding head. The Transformer backbone network consists of four serially connected Transformer layers, and the Transformer layer uses the Transformer network. The SegFormer decoding head consists of a three-layer MLP network, bilinear interpolation upsampling operations, tensor splicing operations, and a lightweight convolutional network. In the dual-branch feature extraction module, the local feature extraction module and the global feature extraction module exist in parallel; at the same time, the dual-branch feature extraction module has four different encoding stages, each of which consists of a ResNet layer in the ResNet backbone network, a Transformer layer in the Transformer backbone network, and a differential attention fusion module.

[0006] A. During the processing of the local feature extraction module, the image first passes through the four ResNet layers in the ResNet backbone network for feature extraction. The output of the previous ResNet layer serves as the input of the next ResNet layer. The local features extracted by the ResNet layer in each encoding stage are represented as , H and W represent the width and height of the original input image respectively, express The number of channel dimensions of the ResNet layer feature extraction in the encoding stage; after the ResNet backbone network extracts the features, the local features of the 4th ResNet layer are input into the FCN decoding head for feature refinement and output the local auxiliary prediction results of the local feature extraction module, but at this time it is not used as the final prediction result of the model, but only the loss value of the FCN auxiliary decoding head is calculated based on the cross entropy loss function ; B. In the processing of the global feature extraction module, the image first passes through the four Transformer layers in the Transformer backbone network for feature extraction. The output of the previous Transformer layer serves as the input of the next Transformer layer. The global feature extracted by the Transformer layer at each encoding stage is represented as , , H and W represent the width and height of the original input image respectively, express The number of channel dimensions of the Transformer layer feature extraction in the encoding stage. After the Transformer backbone network extracts the features, the global features of each Transformer layer are input into the SegFormer decoding head for feature refinement and output the global auxiliary prediction results of the global feature extraction module. The same processing process as the local feature extraction module above is used to calculate the loss value of the SegFormer auxiliary decoding head based on this output based on the cross entropy loss function. .

[0007] Preferably, in step S2), the differential attention fusion module is composed of a convolution operation and a compression-excitation network (SE network for short), which is used to effectively fuse the local features of the ResNet layer of the four encoding stages and the global features of the Transformer layer in the dual-branch feature extraction module; the input of the attention fusion module is the local features and global features in the dual-branch feature extraction module, that is, and ,in The first branch in the dual-branch feature extraction module The local features of the ResNet layer in the encoding stage, For the Global features of the Transformer layer in the encoding stage; That is, the local features of the 4 ResNet layers in the local feature extraction module 、 、 、 And the global features of the 4 Transformer layers in the global feature extraction module 、 、 、 Perform fusion processing, and 、 and 、 and 、 and Two by two are used as input, and are input into 4 differential attention fusion modules for processing, and 4 fusion features are obtained, which are 、 、 、 , the specific steps are as follows: Sa1) will and Through two 1×1 convolution operations to align the number of channels, we get and ; Sa2) 、 Input the compression-excitation network respectively, and get and ; Sa3) and The main elements are subtracted and the absolute value is calculated to obtain the difference feature, and then the difference feature is subjected to a 3×3 convolution operation, and then normalized using the Sigmoid activation function to obtain the difference attention weight. ; Sa4) Difference attention weight respectively and and Perform element-by-element multiplication, and then multiply the results with and Add element by element to get and ; Sa5) and The concatenation is performed in the channel dimension, and then a 3×3 convolution operation is performed to extract the features. ; Sa6) The difference attention weight obtained in step Sa3) and Multiply, and then go through a 3×3 convolution operation to get the The output of the differential attention fusion module in the encoding stage , , that is, fusion features.

[0008] Preferably, the cross-fusion decoding module in step S2) is composed of three cross-fusion modules connected in series, each cross-fusion module corresponds to a different decoding stage; wherein, the first input feature of the cross-fusion module in the decoding stage includes the first and The fusion features of the differential attention fusion module in the encoding stage, the second input feature includes The fusion features of the attention fusion module in the first encoding stage and the decoding features of the cross fusion module in the first decoding stage are included in the third input feature. The fusion features of the attention fusion module in the first encoding stage and the decoding features of the cross fusion module in the second encoding stage, where , Indicates the decoding stage of the corresponding cross-fusion module; That is, take the output of the difference attention fusion module 、 、 、 The four fusion features of 、 As the input features of the first cross fusion module, the output is ;Will and As the input features of the second cross fusion module, the output is ; Similarly, and They are respectively used as the input features of the third cross fusion module to obtain the output .

[0009] Preferably, the specific operation steps of the cross-fusion module are as follows: Sb1) First, perform linear projection on the two inputs, first on the first input feature Perform a 2x bilinear interpolation upsampling to make the height and width of the two input features the same, and then input the two input features into the cross fusion module for processing. For the first cross fusion module in the decoding stage, the number of channels of the output features of the difference attention fusion module in the fourth encoding stage is 4; and For example, input Perform a 2x bilinear interpolation upsampling to make it consistent with the feature The height and width are the same, and then the two inputs are linearly projected. After linear transformation, we get the query matrix ,right After linear transformation, the key matrix is ​​obtained Sum Matrix , the formula is as follows:

[0010] in, 、 、 They are the projection matrices of Q, K, and V respectively; Sb2) Transform Q into shape, then perform matrix multiplication with the result of shape transformation and transposition operation of K to obtain the attention score, and then obtain the attention weight matrix after a Softmax normalization operation. , the formula is as follows:

[0011] in, represents the cross attention operation, yes The channel dimension, represents the softmax activation function; Sb3) The attention weight matrix Multiply it by the value matrix V after shape transformation to obtain the initial fusion feature , then further extract and transform features through the feedforward network, and finally perform a 3×3 convolution operation for local information enhancement to obtain the output feature , the formula is as follows:

[0012]

[0013] in, Represents a feedforward network, which consists of two linear connection layers, with a ReLU activation function between the two linear layers; represents a 3×3 convolution operation, is the final output.

[0014] Preferably, the segmentation head in step S2) is composed of a fully connected layer, which combines the decoding features of the third decoding stage in the cross-fusion decoding module , input into a fully connected layer of the segmentation head for classification prediction, and obtain the segmentation prediction results of target areas such as ocean, land, and islands , used to calculate the segmentation loss value of the model .

[0015] Preferably, in step S3) the total loss value of the model is calculated ,include Calculated auxiliary loss value of FCN decoding head , Calculated auxiliary loss value of SegFormer decoding head and Calculated segmentation loss value of the model Three parts, the total loss function of the model The calculation formula is as follows:

[0016] in, is the weight coefficient, which is 0.4; described and The value is calculated based on the cross entropy loss function, and the segmentation loss value The value is composed of the cross entropy loss function and the Focal loss loss function, and the calculation formula is as follows:

[0017] in, and Respectively represent the final prediction results of the model Calculated cross entropy loss value and Focal loss value, is the weight coefficient, which takes a value of 1; The calculation formula of the cross entropy loss function is as follows:

[0018] Among them, N represents the total number of pixels in the sample image, O represents the total number of categories, Indicates the pixels in the category The true label on The predicted result pixels in the category The model predicted probability on ; The calculation formula of the Focal loss function is as follows:

[0019] in, Represents the category weight factor, which is used to solve the category imbalance problem and balance the contribution of minority and majority categories to the loss. Its value is 0.5; Represents the focusing factor, which is used to control the impact of easy-to-classify samples on the loss. When the value is 0, Focal Loss degenerates into the standard cross entropy loss. A value of 2 forces the model to focus more on hard-to-classify areas. is the true label, represents the probability predicted by the model.

[0020] The present invention further proposes a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, it implements the above-mentioned sea, land and island remote sensing image segmentation method based on feature difference enhancement fusion.

[0021] Compared with the existing technology, the present invention addresses the challenges of complex backgrounds in nearshore and ocean areas, overlapping occlusion of land edges, and difficulty in segmenting small target islands in sea, land and island segmentation, and proposes a sea, land and island remote sensing image segmentation method based on feature difference enhancement fusion. This method is an encoding-decoding architecture method. In the encoding stage, two auxiliary encoders are used to encode features and calculate auxiliary losses as loss supervision. Then, in the decoding stage, a decoding structure with multiple feature scales is used to predict the final output. Specifically, a differential attention fusion module is first used in the encoding stage to fuse local features from CNN and global features from Transformer, and the complementarity of the two features is utilized to reduce redundancy between features and enhance the model's understanding of complex scenes. Then, in order to accurately retain detail information, enhance the feature response of small target areas, and improve the model's segmentation effect on small target islands, a cross fusion module is also used in the decoding stage to fuse multi-scale features from the encoder. Finally, the decoded features are passed through a fully connected layer to output the prediction results, and the loss function is calculated for subsequent model optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is a flow chart of the sea, land and island remote sensing image segmentation method based on feature difference enhancement fusion proposed by the present invention; Figure 2 This is a framework diagram of the sea, land and island remote sensing image segmentation model based on enhanced fusion of local and global feature differences proposed in the present invention; Figure 3 This is a structural block diagram of the differential attention fusion module of the present invention; Figure 4 This is a structural block diagram of the cross-fusion module of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of this application to further clearly and completely describe the technical solutions in the embodiments of this application. It should be noted that the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making any creative work shall fall within the scope of protection of this application.

[0024] In order to make the invention objectives, technical solutions and advantages of this application clearer, the embodiments of this application are further described in detail in conjunction with the drawings in the specification: In order to more clearly understand the above-mentioned objectives, features and advantages of the present invention, the advantages of the present invention will be further illustrated by comparing the embodiments in conjunction with the drawings and specific implementation methods.

[0025] The technical solution of the present invention will be described in further detail below with reference to the accompanying drawings and specific implementation examples.

[0026] The sea, land and island remote sensing image segmentation method based on feature difference enhancement fusion proposed in this paper is as follows: Figure 1 As shown, the following steps are included: S1) Collect remote sensing image data of land, sea and islands and construct a remote sensing image dataset of land, sea and islands for subsequent model training; S2) Construct an initial sea, land and island remote sensing image segmentation model based on the difference enhancement fusion of local features and global features. The model structure consists of four parts: a dual-branch feature extraction module, a difference attention fusion module, a cross-fusion decoding module and a segmentation head. Figure 2 As shown; Specifically, the dual-branch feature extraction module consists of two parts: a local feature extraction module and a global feature extraction module that operate in parallel; the differential attention fusion module consists of several convolution operations and two compression-excitation operations; and the cross-fusion decoding module consists of three cross-fusion modules connected in series. Input the remote sensing image of land, sea and islands, and construct the land, sea and island segmentation model based on the enhanced fusion of local and global features. The specific steps are as follows: (1) In the encoding stage, the input image is first processed by the dual-branch feature extraction module; ( Indicates the number of channels of the input image, generally 3 for RGB images. and Representing the width and height of the input image respectively) are copied into two copies and input into the local feature extraction module and the global feature extraction module for parallel processing respectively; A. The local feature extraction module consists of a ResNet backbone network and an FCN decoding head. The ResNet backbone network contains four ResNet layers. The four ResNet layers are operated in sequence, and the output of the previous ResNet layer is used as the input of the next ResNet layer. The local features extracted by the ResNet layer in each encoding stage are expressed as , Indicates the number of feature map channels output after the i-th ResNet layer, and Indicates the feature map size, and then the features output by the 4th ResNet layer Input to the FCN (Fully Convolutional Network) decoding head for processing, and output a local auxiliary prediction result , the loss value for the FCN auxiliary decoding head The loss value can measure the difference between the predicted result and the true label; the ResNet layer uses the ResNet50 network, and the FCN decoding head uses a 3-layer MLP (multi-layer perceptron) network. After performing channel dimension transformation and feature fusion, the local auxiliary prediction results are output; B. The global feature extraction module consists of a Transformer backbone network and a SegFormer decoding head. The Transformer backbone network contains four Transformer layers. The Transformer layer uses a Transformer network. For the input image The four Transformer layers are operated sequentially, and the output of the previous Transformer layer is used as the input of the next Transformer layer. The global feature representation of each Transformer layer is , express The number of channel dimensions of the feature extraction of the Transformer layer in the encoding stage, and then the global features of the four Transformer layers are extracted. 、 、 、 Input to the final SegFormer decoding head, output global auxiliary prediction results of the global feature extraction module branch , loss value for SegFormer auxiliary decoding header The SegFormer decoding head first flattens the global features of each stage, then passes them through a 3-layer MLP network, and then performs bilinear interpolation upsampling on the four features after MLP processing and splicing them in the channel dimension. The spliced ​​features are input into a lightweight convolutional network to generate .

[0027] (2) Based on the features processed by the dual-branch feature extraction module, the differential attention fusion module is used to fuse the dual-branch features; the input of the attention fusion module is the local features and global features in the dual-branch feature extraction module, that is, and ,in The first branch in the dual-branch feature extraction module The local features of the ResNet layer in the encoding stage, For the The global features of the Transformer layer in the encoding stage; that is, the local features of the 4 ResNet layers in the local feature extraction module 、 、 、 And the 4 Transformer layer global features in the global feature extraction module 、 、 、 ,Will and 、 and 、 and 、 and Two by two are used as input, and four differential attention fusion modules are input for fusion processing to obtain four fusion features. 、 、 、 ; Differential attention fusion module such as Figure 3 As shown, and For example, the specific operation steps of each differential attention fusion module are as follows: Sa1) will and Through two 1×1 convolution operations to align the number of channels, we get and ; Sa2) 、 Input the compression-excitation network respectively, and get and ; Sa3) and The main elements are subtracted and the absolute value is calculated to obtain the difference feature, and then the difference feature is subjected to a 3×3 convolution operation, and then normalized using the Sigmoid activation function to obtain the difference attention weight. ; Sa4) Difference attention weight respectively and and Perform element-by-element multiplication, and then multiply the results with and Add element by element to get and ; Sa5) and The concatenation is performed in the channel dimension, and then a 3×3 convolution operation is performed to extract the features. ; Sa6) The difference attention weight obtained in step Sa3) and Multiply, and then go through a 3×3 convolution operation to get the The output of the differential attention fusion module in the encoding stage , , that is, fusion features.

[0028] (3) The decoding stage operates based on the fusion features of the four differential attention fusion modules in step (2); the cross-fusion decoding module is composed of three cross-fusion modules in series, which are used to fuse the multi-scale features from the encoder, and each cross-fusion module corresponds to a different decoding stage; among them, the first input feature of the cross-fusion module in the decoding stage includes the first and The fusion features of the differential attention fusion module in the encoding stage, the second input feature includes The fusion features of the attention fusion module in the first encoding stage and the decoding features of the cross fusion module in the first decoding stage are included in the third input feature. The fusion features of the attention fusion module in the first encoding stage and the decoding features of the cross fusion module in the second encoding stage, where , Indicates the decoding stage of the corresponding cross-fusion module; That is, take the output of the difference attention fusion module 、 、 、 etc., and As the input features of the first cross fusion module, the output is ;Will and As the input features of the second cross fusion module, the output is ; Similarly, and They are respectively used as the input features of the third cross fusion module to obtain the output The three cross-fusion modules all perform the same operations, such as Figure 4 As shown below and Take the cross-fusion module as an example to describe the specific operation steps of the cross-fusion module in detail: Sb1) and For example, input Perform a 2x bilinear interpolation upsampling to make it consistent with the feature The height and width are the same, and then the two inputs are linearly projected. After linear transformation, we get the query matrix ,right After linear transformation, the key matrix is ​​obtained Sum Matrix , the formula is as follows:

[0029] in, 、 、 They are the projection matrices of Q, K, and V respectively; Sb2) Transform Q into shape, then perform matrix multiplication with the result of shape transformation and transposition operation of K to obtain the attention score, and then obtain the attention weight matrix after a Softmax normalization operation. , the formula is as follows:

[0030] in, represents the cross attention operation, yes The channel dimension, represents the softmax activation function; Sb3) The attention weight matrix Multiply it by V after shape transformation to get the initial fusion feature , then further extract and transform features through the feedforward network, and finally perform a 3×3 convolution operation for local information enhancement to obtain the output feature , the formula is as follows:

[0031]

[0032] in, Represents a feedforward network, which consists of two linear connection layers, with a ReLU activation function between the two linear layers; represents a 3×3 convolution operation, is the final output; (4) The segmentation head consists of a fully connected layer that combines the output features of the last cross-fusion module in the decoding stage As the input of the segmentation head, it is input into a fully connected layer of the segmentation head for classification prediction to obtain the segmentation prediction results of target areas such as ocean, land, and islands. , used for the segmentation loss value of the subsequent model .

[0033] S3) Input the sea, land, and island remote sensing image dataset into the initial sea, land, and island remote sensing image segmentation model for training. Calculate the loss value based on the predicted value and the actual value, perform reverse gradient propagation, and optimize the parameters of the initial sea, land, and island remote sensing image segmentation model through a predetermined optimizer. After multiple rounds of training, the sea, land, and island remote sensing image segmentation model is obtained.

[0034] Specifically, in step S3) the total loss value of the model is calculated The specific steps are as follows: (1) The total loss value includes Calculated auxiliary loss value of FCN decoding head , Calculated auxiliary loss value of SegFormer decoding head and Calculated segmentation loss value of the model Three parts, total loss function The calculation formula is as follows:

[0035] in, is the weight coefficient, which is 0.4; (2) Use the cross entropy loss function to calculate the auxiliary loss value of the FCN decoding head and the auxiliary loss value of the SegFormer decoding head ; (3) Calculate the segmentation loss value based on the cross entropy loss function and the focal loss loss function , the calculation formula is as follows:

[0036] in, and Respectively represent the final prediction results of the model Calculated cross entropy loss value and Focal loss value, is the weight coefficient, which takes a value of 1; The formula for the cross entropy loss function is as follows:

[0037] Among them, N represents the total number of pixels in the sample image, O represents the total number of categories, Indicates the pixels in the category The true label on The predicted result pixels in the category The model predicted probability on ; The formula of the Focal loss function is as follows:

[0038] in, Represents the category weight factor, with a value of 0.5; represents the focusing factor, The value is 2; is the true label, represents the probability predicted by the model.

[0039] S4) Based on the trained sea, land, and island remote sensing image segmentation model, the sea, land, and island remote sensing image is input, processed, and the segmentation result map of the target area such as the ocean, land, and islands is output.

[0040] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.

[0041] Finally, it should be noted that the above specific embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection thereof. Although the present invention has been described in detail in conjunction with the preferred embodiments, it should be understood by those skilled in the art that, without departing from the core concept of the present invention, modifications, replacements, or equivalent variations to the technical contents thereof should all be deemed to fall within the scope of protection of the present invention, and the scope of protection of the present invention should be based on the content defined in the claims.

Claims

1. A method for segmenting sea, land and island remote sensing images based on feature difference enhancement fusion, characterized by: The method comprises the following steps: S1) Collect remote sensing image data of land, sea and islands and construct a remote sensing image dataset of land, sea and islands for subsequent model training; S2) Constructing an initial land, sea, and island remote sensing image segmentation model based on the enhanced fusion of local and global features. The model structure consists of four parts: a dual-branch feature extraction module, a differential attention fusion module, a cross-fusion decoding module, and a segmentation head. S3) Input the data into the initial land, sea and island remote sensing image segmentation model for training, calculate the loss value based on the predicted value and the actual value, perform reverse gradient propagation, and optimize the parameters of the initial land, sea and island remote sensing image segmentation model through the established optimizer. After multiple rounds of training, the land, sea and island remote sensing image segmentation model is obtained; S4) Based on the trained sea, land and island remote sensing image segmentation model, the sea, land and island remote sensing image to be segmented is input, processed, and the segmentation result image is output.

2. The method for segmenting sea, land and island remote sensing images based on feature difference enhancement fusion according to claim 1, characterized in that: In step S2), the dual-branch feature extraction module includes two parts: a local feature extraction module and a global feature extraction module that operate in parallel. The feature extraction process is as follows: After the remote sensing image of land, sea and islands is input, the dual-branch feature extraction module will input the image The data is copied into two parts and input into the local feature extraction module and the global feature extraction module respectively for parallel operation and processing; A. Local feature extraction module for input image The operation is carried out in sequence through four ResNet layers. The output of the previous ResNet layer is used as the input of the next ResNet layer. The local features extracted by the ResNet layer in each encoding stage are expressed as , Indicates the number of feature map channels output after the i-th ResNet layer, and Indicates the feature map size, and then the local features output by the 4th ResNet layer Input to the FCN decoding head for processing, and output a local auxiliary prediction result ; B. Global feature extraction module for input image The four Transformer layers are operated sequentially, and the output of the previous Transformer layer is used as the input of the next Transformer layer. The global features extracted by the Transformer layer in each encoding stage are expressed as , express The number of channel dimensions of the Transformer layer feature extraction in the encoding stage, and then the global features output by each Transformer layer 、 、 、 Input to the SegFormer decoding head for processing, and output a global auxiliary prediction result .

3. The method for segmenting sea, land and island remote sensing images based on feature difference enhancement fusion according to claim 1, characterized in that: In step S2), the differential attention fusion module takes local features and global features as input, i.e. and ,in For the The local features of the ResNet layer in the encoding stage, For the Global features of the Transformer layer in the encoding stage; For the local features of the 4 ResNet layers 、 、 、 and the global features of the 4 Transformer layers 、 、 、 Fusion processing: and 、 and 、 and 、 and Two by two are used as input, and are input into 4 differential attention fusion modules for processing to obtain 4 fusion features. 、 、 、 .

4. The method for segmenting sea, land and island remote sensing images based on feature difference enhancement fusion according to claim 3, characterized in that: The specific operation steps of the differential attention fusion module include: Sa1) will and Through two 1×1 convolution operations to align the number of channels, we get and ; Sa2) 、 Input the compression-excitation network respectively, and get and ; Sa3) and The main elements are subtracted and the absolute value is calculated to obtain the difference feature, and then the difference feature is subjected to a 3×3 convolution operation, and then normalized using the Sigmoid activation function to obtain the difference attention weight. ; Sa4) Difference attention weight respectively and and Perform element-by-element multiplication, and then multiply the results with and Add element by element to get and ; Sa5) and The concatenation is performed in the channel dimension, and then a 3×3 convolution operation is performed to extract the features. ; Sa6) The difference attention weight obtained in step Sa3) and Multiply, and then go through a 3×3 convolution operation to get the The output of the differential attention fusion module in the encoding stage , , that is, fusion features.

5. The method for segmenting sea, land and island remote sensing images based on feature difference enhancement fusion according to claim 2, characterized in that: The cross-fusion decoding module in step S2) is composed of three cross-fusion modules in series, taking the output of the difference attention fusion module 、 、 、 The four fusion features of and As the input of the first cross fusion module, the decoding features are obtained ;Will and They are respectively used as the input of the second cross fusion module to obtain the decoding features ; Similarly, and They are respectively used as the input of the third cross-fusion module to obtain the decoding features .

6. The method for segmenting sea, land and island remote sensing images based on feature difference enhancement fusion according to claim 5, characterized in that: The specific processing steps of the cross-fusion module are as follows: Sb1) and For example, input Perform a 2x bilinear interpolation upsampling to make it equal to The height and width are the same, and then the two inputs are linearly projected. After linear transformation, we get the query matrix ,right After linear transformation, the key matrix is ​​obtained Sum Matrix , the formula is as follows: ； in, 、 、 They are the projection matrices of Q, K, and V respectively; Sb2) Transform Q into shape, then perform matrix multiplication with the result of shape transformation and transposition operation of K to obtain the attention score, and then obtain the attention weight matrix after a Softmax normalization operation. , the formula is as follows: ; in, represents the cross attention operation, express The channel dimension, represents the softmax activation function, Represents a transpose operation; Sb3) The attention weight matrix Multiply it by the value matrix V after shape transformation to obtain the initial fusion feature , then further extract and transform features through the feedforward network, and finally perform a 3×3 convolution operation for local information enhancement to obtain the output feature , the formula is as follows: ； ； in, represents a feedforward network, represents a 3×3 convolution operation, is the final output.

7. The method for segmenting sea, land and island remote sensing images based on feature difference enhancement fusion according to claim 2, characterized in that: The segmentation head in step S2) consists of a fully connected layer, taking the decoding features of the last cross-fusion module As input, it is input into a fully connected layer of the segmentation head for classification prediction to obtain the segmentation prediction result .

8. The method for segmenting sea, land and island remote sensing images based on feature difference enhancement fusion according to claim 1, characterized in that: Step S3) Calculate the total loss value of the model ,include Calculated auxiliary loss value of FCN decoding head , Calculated auxiliary loss value of SegFormer decoding head and Calculated segmentation loss value of the model Three parts, total loss function The calculation formula is as follows: ； in, is the weight coefficient, which is 0.4; described and The value is calculated based on the cross entropy loss function, and the segmentation loss value The value is composed of the cross entropy loss function and the Focal loss loss function, and the calculation formula is as follows: ； in, and Respectively represent the final prediction results of the model Calculated cross entropy loss value and Focal loss value, is the weight coefficient, which takes a value of 1; The calculation formula of the cross entropy loss function is as follows: ; Among them, N represents the total number of pixels in the sample image, O represents the total number of categories, Indicates the pixels in the category The true label on The predicted result pixels in the category The model predicted probability on ; The calculation formula of the Focal loss function is as follows: ； in, Represents the category weight factor, with a value of 0.5; Represents the focusing factor. When the value is 0, Focal Loss degenerates into standard cross entropy loss. The value is 2; is the true label, represents the probability predicted by the model.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images