Statistical texture-based method and network for semantic segmentation of ocean remote sensing images

Abstract

The application belongs to the technical field of image processing, and discloses a marine remote sensing image semantic segmentation method and network based on statistical texture, which comprises the following steps: S1, inputting a remote sensing image to be subjected to semantic segmentation into a backbone network for feature extraction to obtain a feature map X; S2, respectively passing the feature map X through a hollow spatial pyramid pooling module and a texture excitation module to respectively obtain a feature map Y and a feature map Z; S3, passing the feature map Y through a multi-granularity attention module to obtain a feature map Y_M; passing the feature map Z through a multi-granularity attention module to obtain a feature map Z_M; and S4, merging the feature map Y_M and the feature map Z_M and performing an upsampling operation to obtain a final semantic segmentation result. Through the application, the influence of interference information can be avoided, and the semantic segmentation prediction accuracy is improved.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to a semantic segmentation method and network for marine remote sensing images based on statistical texture. Background Technology

[0002] As a crucial tool for processing and analyzing marine remote sensing images, semantic segmentation refers to the process of assigning semantic object categories to each pixel. This involves two important issues: first, the multi-scale problem, which refers to problems involving multiple orders of magnitude. For example, semantic segmentation tasks often contain both very small and very large objects, and we aim to correctly segment both large and small objects semantically simultaneously. Second, the context problem. Each pixel in an image cannot be isolated; a pixel must have a certain relationship with its surrounding pixels. The interconnections of a large number of pixels create the various objects in the image. Therefore, contextual features refer to certain relationships between pixels and their surrounding pixels.

[0003] Cutting-edge semantic segmentation methods typically employ an encoder-decoder structure with the addition of a dilated spatial pyramid pooling module. Their advantage lies in the ability to fully fuse low-level and high-level semantic features to achieve high-precision semantic segmentation. However, this method has the following drawbacks: First, it neglects the statistical texture information of marine remote sensing images. Image texture features are divided into structural texture features and statistical texture features. Directly fusing low-level semantic features with upsampled high-level semantic features only focuses on the structural texture features of the image, failing to obtain the statistical texture features, which directly affects the final prediction accuracy. Second, while the dilated spatial pyramid pooling module can obtain a large receptive field, it cannot capture the global dependencies between pixels. Although some techniques use a dual attention mechanism including spatial and channel attention modules to obtain global pixel dependencies, some very similar pixels that do not belong to the same category are considered interference information in this process. As the number of pixels requiring dependency calculation increases, the spatial and channel attention mechanisms are inevitably affected by interference information, which will also affect the final prediction accuracy. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a semantic segmentation method and network for marine remote sensing images based on statistical texture. A texture excitation module is constructed, which extracts statistical texture features from remote sensing images using statistical methods. Simultaneously, structural texture features, statistical texture features, and semantic feature information are fused to ultimately improve prediction accuracy. This invention also constructs a multi-granularity attention module, performing spatial attention and channel attention mechanisms simultaneously at both global and local scales. This reduces the impact of interference information on the attention mechanism, leading to a more comprehensive understanding of the information in the feature maps and ultimately improving prediction accuracy.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] First, this invention provides a semantic segmentation method for marine remote sensing images based on statistical texture, comprising the following steps:

[0007] Step S1: Input the remote sensing image to be semantically segmented into a deep convolutional neural network for feature extraction to obtain feature map X;

[0008] In step S2, feature map X will be processed by the hollow space pyramid pooling module and the texture excitation module to obtain feature map Y and feature map Z respectively;

[0009] The texture excitation module aims to mine hidden texture features in the feature map using statistical methods. The method is as follows: For the input feature map, the pixel values ​​on the feature map are divided into different levels by average pooling and level division operations in sequence. Then, the statistical texture features of the image are extracted from the four directions of horizontal direction, vertical direction, main diagonal direction and anti-diagonal direction respectively. The texture features extracted from the four directions are merged to obtain statistical texture feature information.

[0010] Step S3: Feature map Y is processed by a multi-granularity attention module to obtain feature map Y_M; feature map Z is processed by a multi-granularity attention module to obtain feature map Z_M;

[0011] The multi-granularity attention module explores the deep information of the image from both the spatial domain and the channel domain, assigns different weights to each pixel, and obtains the correlation between each pixel;

[0012] Step S4: Merge feature map Y_M and feature map Z_M, and perform upsampling to obtain the final semantic segmentation result.

[0013] Furthermore, the feature map X obtained in step S1 is used as the input to the texture excitation module, and the input to the texture excitation module is denoted as feature map A. The specific steps of the texture excitation module processing the feature map in step S2 are as follows:

[0014] Step S21, Average Pooling Operation:

[0015] Input feature map First, feature maps are obtained through average pooling operations along the channel dimension. Where C, H, and W represent the number of channels, height, and width of the feature map, respectively.

[0016] Step S22, Level Classification Operation:

[0017] First, the values ​​on feature map B are quantized into n+1 levels P = [P0, P1, ..., P2] by dividing the values ​​between the minimum and maximum values ​​of feature map B into N equal parts. n Specifically, level P n Calculated using the following formula:

[0018]

[0019] Where N = W = H, n ∈ [0, N], max(B) and min(B) represent B respectively. ij The maximum and minimum values ​​in the feature map are determined; next, the levels are divided according to the values ​​on feature map B and F(x) to obtain feature map C, F(x) as shown in formula (2):

[0020] C i,j =m+1 if P m ≤B i,j <P m+1 (2)

[0021] Where m∈[0,N-1], B i,j Represents the values ​​in the i-th row and j-th column of feature map B; C i,j The value of P represents the level to which the pixel in the i-th row and j-th column of the feature map C is classified; m P represents the value of the m-th level. m+1 This represents the value of the (m+1)th level;

[0022] Step S23: Extract statistical texture features from the image in four directions: horizontal, vertical, main diagonal, and anti-diagonal, to obtain feature maps D, E, F, and G respectively. Then, merge and integrate the information of D, E, F, and G to obtain statistical texture feature information H.

[0023] Step S24: Perform a 1×1 convolution on feature map H, which will change the number of channels to C1, resulting in feature map I.

[0024] Furthermore, the operations of extracting statistical texture features of the image in the four directions of step S23 (horizontal, vertical, main diagonal, and anti-diagonal) are denoted as F1(x), F2(x), F3(x), and F4(x), respectively. The resulting feature maps D, E, F, and G are represented as follows:

[0025] D p,q =Count{C id,jd |C id,jd =p,C id,jd+1 =q},

[0026] Where id∈[1,H],jd∈[1,W-1],p∈[1,N],q∈[1,N] (3)

[0027] E p,q =Count{C ie,je |C ie,je =p,C ie+1,je =q},

[0028] Where ie∈[1,H-1], je∈[1,W], p∈[1,N], q∈[1,N] (4)

[0029] F p,q =Count{C if,jf |C if,jf =p,C if+1,jf+1 =q},

[0030] Where if∈[1,H-1],jf∈[1,W-1],p∈[1,N],q∈[1,N] (5)

[0031] G p,q =Count{C ig,jg |C ig,jg =p,C ig-1,jg-1 =q},

[0032] Where ig∈[2,H],jg∈[2,W],p∈[1,N],q∈[1,N] (6) Where p and q represent the assigned level, C id,jd The value C represents the level to which the pixel in row id and column jd is classified. id,jd+1 The value C represents the level to which the pixel in row id and column jd+1 is classified. ie,je The value of C indicates the level to which the pixel in row ie and column je is classified. ie+1,je The value of C indicates the level to which the pixel in row (ie+1) and column (je) is classified. if,jf The value of C indicates the level to which the pixel in the jf-th column of the if-th row is classified. if+1,jf+1The value C indicates the level to which the pixel in row (if+1) and column (jf+1) is classified. ig,jg The value C represents the level to which the pixel in row ig and column jg is classified. ig-1,jg-1 The value indicates the level to which the pixel in row (ig-1) and column (jg-1) is classified; D p,q This represents the value in the p-th row and q-th column of feature map D. Count represents the summation function, and the formula within the summation function is called the discriminant. p,q The value of E is determined by the number of times the discriminant is true; p,q This represents the value in the p-th row and q-th column of feature map E. p,q The value of F is determined by the number of times the discriminant is true; p,q This represents the value in the p-th row and q-th column of the feature map F. p,q The value of G is determined by the number of times the discriminant is true; p,q G represents the value in the p-th row and q-th column of the feature map G. p,q The value of is determined by the number of times the discriminant is true.

[0033] Furthermore, in step S3, the multi-granularity attention module is designed based on the basic spatial attention module and the basic channel attention module. The multi-granularity attention module includes a multi-granularity spatial attention module and a multi-granularity channel attention module. The multi-granularity spatial attention module adopts a three-level architecture pattern of coarse-grained, medium-grained, and fine-grained. Each level is designed based on the basic spatial attention module. The feature maps output by the three levels of coarse-grained, medium-grained, and fine-grained are merged to obtain the final feature map E_S.

[0034] The multi-granularity channel attention module still adopts a three-level architecture pattern of coarse-grained, medium-grained, and fine-grained. Each level is designed based on the basic channel attention module, and the feature maps output by the three levels of coarse-grained, medium-grained, and fine-grained are merged to obtain the final feature map E_C.

[0035] Furthermore, the input to the multi-granularity spatial attention module is denoted as the feature map. The multi-granularity spatial attention module adopts a three-level architecture. In the first level, the feature map A_S is input into the basic spatial attention module to obtain the feature map B_S. In the second level, the feature map A_S is divided into four parts A_S1, A_S2, A_S3, and A_S4 on an average basis along the H and W dimensions. These four parts are then input into the basic spatial attention module to obtain the feature map B_S. Finally, these four parts are combined to obtain the final feature map. In the third level, feature map A_S is divided into 16 equally sized parts along the H and W dimensions. After processing by the basic spatial attention module, feature map D_S is obtained. Finally, the three-level feature maps B_S, C_S, and D_S are merged to obtain the final feature map E_S.

[0036] E_S = CAT(B_S, C_S, D_S) (7)

[0037] CAT stands for merge operation.

[0038] Furthermore, the input to the multi-granularity channel attention module is denoted as the feature map. The multi-granularity channel attention module still adopts a three-level architecture. In the first level, the feature map... The input is fed into the basic channel attention module to obtain feature map B_C; in the second stage, feature map A_C is divided into four parts along dimension C: A_C1, A_C2, A_C3, and A_C4. These four parts are then fed into the basic channel attention module to obtain... Finally, these four parts are combined to obtain the final feature map. The third level uses the same idea. Feature map A_C is divided into 16 parts of the same size along the C dimension. After being processed by the basic spatial attention module, feature map D_C is obtained. Finally, the three-level feature maps B_C, C_C, and D_C are merged to obtain the final feature map E_C.

[0039] Then, this invention provides a semantic segmentation network for marine remote sensing images based on statistical texture, employing an encoder-decoder structure to implement the previously described semantic segmentation method for marine remote sensing images based on statistical texture. The network includes a backbone network, a hollow spatial pyramid pooling module, a texture excitation module, a multi-granularity attention module, and an output module.

[0040] The backbone network is used to extract feature maps X from remote sensing images;

[0041] The void space pyramid pooling module is used to extract deep feature information to obtain feature map Y;

[0042] The texture excitation module is used to extract hidden statistical texture features from the feature map to obtain feature map Z;

[0043] The multi-granularity attention module includes a multi-granularity spatial attention module and a multi-granularity channel attention module. The output of the hollow spatial pyramid pooling module is used as the input of the multi-granularity spatial attention module, and the output of the texture excitation module is used as the input of the multi-granularity channel attention module. The multi-granularity attention module extracts image features from both the spatial domain and the channel domain at three levels: coarse-grained, medium-grained, and fine-grained, to obtain the correlation between each pixel.

[0044] The output module is used to merge the outputs of the multi-granularity spatial attention module and the multi-granularity channel attention module, and after upsampling, output the semantic segmentation result.

[0045] Compared with the prior art, the advantages of this invention are:

[0046] (1) A texture excitation module was constructed to solve the problem that traditional neural network models cannot extract statistical textures. Based on the information of remote sensing images, statistical texture features of remote sensing images are extracted using statistical methods, enabling the model to simultaneously integrate structural texture features, statistical texture features, and semantic feature information, ultimately improving prediction accuracy.

[0047] (2) A multi-granularity attention module was constructed to address the problem that traditional attention mechanisms are easily affected by interference information. Based on a dual attention mechanism comprising spatial and channel attention modules, the multi-granularity attention module segments the feature map into different sizes, applies the dual attention mechanism to these segmented feature maps, assigns different weights to feature maps of different sizes, and finally fuses this information. Because the feature map is segmented into smaller feature maps of different sizes, it plays a crucial role in reducing the amount of interference information. Simultaneously, the multi-granularity attention mechanism observes the feature map from different perspectives, thus providing a more comprehensive understanding of the information on the feature map, ultimately improving prediction accuracy. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 This is a network structure diagram of the present invention;

[0050] Figure 2 This is a structural diagram of the texture excitation module of the present invention;

[0051] Figure 3 This is a structural diagram of the basic spatial attention module of the present invention;

[0052] Figure 4 This is a structural diagram of the basic channel attention module of the present invention;

[0053] Figure 5 This is a structural diagram of the multi-granularity spatial attention module of the present invention;

[0054] Figure 6 This is a structural diagram of the multi-granularity channel attention module of the present invention. Detailed Implementation

[0055] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0056] Example 1

[0057] Combination Figure 1 The aforementioned network structure diagram, in this embodiment, provides a semantic segmentation method for marine remote sensing images based on statistical texture, including the following steps:

[0058] Step S1: Input the remote sensing image to be semantically segmented into a deep convolutional neural network for feature extraction to obtain a feature map X. In this embodiment, a ResNet network can be used for feature extraction, which uses convolution operations to extract features.

[0059] In step S2, feature map X will be processed by the void space pyramid pooling module and the texture excitation module to obtain feature map Y and feature map Z respectively.

[0060] The dilated spatial pyramid pooling module employs existing technology and comprises five parts: a 1×1 convolution, a 3×3 convolution with a dilation rate of 6, a 3×3 convolution with a dilation rate of 12, a 3×3 convolution with a dilation rate of 18, and a global average pooling operation. After passing through the dilated spatial pyramid pooling module, the feature map will yield five new feature maps. These five new feature maps are then merged and passed through a 1×1 convolution to obtain a new feature map.

[0061] The texture excitation module aims to mine hidden texture features in the feature map using statistical methods. The method is as follows: For the input feature map, the pixel values ​​on the feature map are divided into different levels by average pooling and level division operations in sequence. Then, the statistical texture features of the image are extracted from the four directions of horizontal, vertical, main diagonal and anti-diagonal, and the texture features extracted from the four directions are merged to obtain statistical texture feature information.

[0062] The feature map X obtained in step S1 is used as the input to the texture excitation module, and the input to the texture excitation module is denoted as feature map A. As a preferred implementation, combined with... Figure 2 When the texture excitation module processes the feature map, the specific steps are as follows:

[0063] Step S21, Average Pooling Operation:

[0064] Input feature map First, feature maps are obtained through average pooling operations along the channel dimension. Where C, H, and W represent the number of channels, height, and width of the feature map, respectively.

[0065] Step S22, Level Classification Operation:

[0066] First, the values ​​on feature map B are quantized into n+1 levels P = [P0, P1, ..., P2] by dividing the values ​​between the minimum and maximum values ​​of feature map B into N equal parts. n Specifically, level P n Calculated using the following formula:

[0067]

[0068] Where N = W = H, n ∈ [0, N], and max(B) and min(B) represent B respectively. i,j The maximum and minimum values ​​in the feature map are determined; next, the levels are divided according to the values ​​on feature map B and F(x) to obtain feature map C, F(x) as shown in formula (2):

[0069] C i,j =m+1ifP m ≤B i,j <P m+1 (2)

[0070] Where m∈[0,N-1], B i,j Represents the values ​​in the i-th row and j-th column of feature map B; C i,j The value of P represents the level to which the pixel in the i-th row and j-th column of the feature map C is classified; m P represents the value of the m-th level. m+1 This represents the value of the (m+1)th level.

[0071] Step S23: Extract statistical texture features from the image in four directions: horizontal, vertical, main diagonal, and anti-diagonal, to obtain feature maps D, E, F, and G respectively. Then, merge and integrate the information of D, E, F, and G to obtain statistical texture feature information H.

[0072] In detail, the operations for extracting statistical texture features from the image in the four directions—horizontal, vertical, main diagonal, and anti-diagonal—are denoted as F1(x), F2(x), F3(x), and F4(x), respectively. The resulting feature maps D, E, F, and G are represented as follows:

[0073] D p,q =Count{C id,jd |C id,jd =p,C id,jd+1 =q},

[0074] Where id∈[1,H],jd∈[1,W-1],p∈[1,N],q∈[1,N](3)

[0075] E p,q =Count{C ie,je |C ie,je=p,C ie+1,je =q},

[0076] Where ie∈[1,H-1], je∈[1,W], p∈[1,N], q∈[1,N](4)

[0077] F p,q =Count{C if,jf |C if,jf =p,C if+1,jf+1 =q},

[0078] Where if∈[1,H-1],jf∈[1,W-1],p∈[1,N],q∈[1,N](5)

[0079] G p,q =Count{C ig,jg |C ig,jg =p,C ig-1,jg-1 =q},

[0080] Where ig∈[2,H],jg∈[2,W],p∈[1,N],q∈[1,N] (6) where p and q represent the assigned level, C id,jd The value C represents the level to which the pixel in row id and column jd is classified. id,jd+1 The value C represents the level to which the pixel in row id and column jd+1 is classified. ie,je The value of C indicates the level to which the pixel in row ie and column je is classified. ie+1,je The value of C indicates the level to which the pixel in row (ie+1) and column (je) is classified. if,jf The value of C indicates the level to which the pixel in the jf-th column of the if-th row is classified. if+1,jf+1 The value C indicates the level to which the pixel in row (if+1) and column (jf+1) is classified. ig,jg The value C represents the level to which the pixel in row ig and column jg is classified. ig-1,jg-1 The value indicates the level to which the pixel in row (ig-1) and column (jg-1) is classified; D p,q This represents the value in the p-th row and q-th column of feature map D. Count represents the summation function, and the formula within the summation function is called the discriminant. p,q The value of E is determined by the number of times the discriminant is true; p,q This represents the value in the p-th row and q-th column of feature map E. p,q The value of F is determined by the number of times the discriminant is true; p,q This represents the value in the p-th row and q-th column of the feature map F. p,q The value of G is determined by the number of times the discriminant is true; p,q G represents the value in the p-th row and q-th column of the feature map G. p,q The value of is determined by the number of times the discriminant is true.

[0081] It's important to note that p and q in all formulas have the same meaning and the same range of values. They represent the specific level. For example, C1,1 might be assigned to level 5, with a value of 5; C1,2 might be assigned to level 8, with a value of 8. In this case, the value of D5,8 will be incremented by one. Similarly, C3,2 might be assigned to level 5, with a value of 5; C3,3 might be assigned to level 8, and the value of D5,8 will also be incremented by one.

[0082] Integrating information from D, E, F, and G yields more comprehensive statistical texture feature information H:

[0083] H = CAT(D, E, F, G) (7)

[0084] CAT stands for merge operation.

[0085] The four channels in feature map H contain statistical texture features extracted along the horizontal, vertical, main diagonal, and sub-diagonal directions, respectively.

[0086] Step S24: Perform a 1×1 convolution on feature map H, which will change the number of channels to C1, resulting in feature map I. In this process, the deep neural network will analyze the information extracted along the four directions to obtain the statistical texture feature information that can be extracted by observing along the C1 directions.

[0087] Step S3: Feature map Y is processed by a multi-granularity attention module to obtain feature map Y_M; feature map Z is processed by a multi-granularity attention module to obtain feature map Z_M; wherein the multi-granularity attention module explores the deep information of the image from both the spatial domain and the channel domain, assigns different weights to each pixel, and obtains the correlation between each pixel.

[0088] Research by scientists in the field of human vision has shown that different parts of the human retina have varying degrees of information processing capabilities. To make efficient use of limited visual information processing resources, humans need to select specific parts of the visual region and focus their attention on them. Influenced by this idea, researchers in the field of computer vision have proposed attention mechanisms. In semantic segmentation, attention mechanisms primarily explore the deeper information of images from both the spatial and channel domains, assigning different weights to each pixel to ultimately improve prediction accuracy.

[0089] Attention mechanisms can be used to obtain dependencies between pixels globally. However, similar pixels that do not belong to the same category are considered noise. As the number of pixels requiring correlation calculation increases, the attention mechanism becomes more susceptible to noise, which in turn affects the final prediction accuracy. To address this issue, this invention proposes a multi-granularity attention mechanism.

[0090] The multi-granularity attention module of the present invention is designed based on the basic spatial attention module and the basic channel attention module. The multi-granularity attention module includes a multi-granularity spatial attention module and a multi-granularity channel attention module.

[0091] The basic spatial attention module will be introduced below. Its detailed structure is as follows: Figure 3 As shown,

[0092] Given feature map Feature map S_1 will first undergo a 1×1 convolution operation to generate the feature map. With feature map Next, the dimensions of feature maps S_2 and S_3 are changed to obtain the final feature maps. With feature map Where M = H × W. Next, the transpose of feature map S_2 is multiplied by feature map S_3, and then passed through a normalized exponential function to finally obtain the spatial attention map.

[0093]

[0094] Among them, S_5 ji This represents the degree of influence of position i on position j. The more similar the feature representations of two positions, the greater their correlation. Meanwhile, feature map S_1 will undergo convolution operations to generate a new feature map. And change its dimensions to make it a feature map. Next, matrix multiplication is performed on feature map S_4 and spatial attention map S_5, and the resulting dimension is changed. Then multiply by the hyperparameter α, and add corresponding elements to the feature map S_1 to obtain the final feature map.

[0095]

[0096] The detailed structure of the basic channel attention module is as follows: Figure 4 As shown. A given feature map. Feature map V_1 will first undergo a 1×1 convolution operation to generate the feature map. With feature map Next, the dimensions of feature map V_2 and feature map V_3 are changed to obtain the final feature map. With feature map Where M = H × W. Next, matrix multiplication is performed on the transposes of feature maps V_2 and V_3, and then the result is normalized through an exponential function to obtain the channel attention map.

[0097]

[0098] Among them, S_5 ji This represents the degree of influence of the i-th channel on the j-th channel. The more similar the feature representations at two locations, the greater their correlation. Meanwhile, feature map V_1 will undergo a 1×1 convolution operation to generate a new feature map. And change its dimensions to make it a feature map. Next, matrix multiplication is performed on the spatial attention map V_5 and the feature map V_4, and the resulting dimension is changed. Then multiply by the hyperparameter β, and add corresponding elements to the feature map V_1 to obtain the final feature map.

[0099]

[0100] Combination Figure 5 In the figure, "spatial attention" represents the basic spatial attention module. The multi-granularity spatial attention module adopts a three-level architecture pattern of coarse-grained, medium-grained, and fine-grained. Each level is designed based on the basic spatial attention module. The feature maps output by the three levels of coarse-grained, medium-grained, and fine-grained are merged to obtain the final feature map E_S.

[0101] Specifically, the input to the multi-granularity spatial attention module is denoted as the feature map. Traditional spatial attention mechanisms only input the entire feature map into the spatial attention module. Unlike traditional spatial attention mechanisms, the multi-granularity spatial attention module adopts a three-level architecture. In the first level, feature map A_S is input into the basic spatial attention module to obtain feature map B_S; in the second level, ... Figure 5 As shown in the medium-granularity section, the feature map A_S is divided into four parts A_S1, A_S2, A_S3, and A_S4 on an average basis along the H and W dimensions. These four parts will be input into the basic spatial attention module to obtain... Finally, these four parts are combined to obtain the final feature map. In the third level, feature map A_S is divided into 16 equally sized parts along the H and W dimensions. After processing by the basic spatial attention module, feature map D_S is obtained. Finally, the three-level feature maps B_S, C_S, and D_S are merged to obtain the final feature map E_S.

[0102] E_S = CAT(B_S, C_S, D_S) (7)

[0103] CAT stands for merge operation.

[0104] Combination Figure 6 In the figure, “channel attention” represents the basic channel attention module. The multi-granularity channel attention module still adopts a three-level architecture pattern of coarse-grained, medium-grained and fine-grained. Each level is designed based on the basic channel attention module. The feature maps output by the three levels of coarse-grained, medium-grained and fine-grained are merged to obtain the final feature map E_C.

[0105] Specifically, the input to the multi-granularity channel attention module is denoted as the feature map. The multi-granularity channel attention module still adopts a three-level architecture. In the first level, the feature map... It is input into the basic channel attention module to obtain feature map B_C; in the second level, such as Figure 6 As shown in the medium-granularity section, the feature map A_C is divided into four parts along the C dimension: A_C1, A_C2, A_C3, and A_C4. These four parts will be input into the basic channel attention module to obtain... Finally, these four parts are combined to obtain the final feature map. The third level uses the same idea. Feature map A_C is divided into 16 parts of the same size along the C dimension. After being processed by the basic spatial attention module, feature map D_C is obtained. Finally, the three-level feature maps B_C, C_C, and D_C are merged to obtain the final feature map E_C.

[0106] Step S4: Merge feature map Y_M and feature map Z_M, and perform upsampling to obtain the final semantic segmentation result.

[0107] Example 2

[0108] This embodiment provides a semantic segmentation network for marine remote sensing images based on statistical texture. It employs an encoder-decoder structure to implement a semantic segmentation method for marine remote sensing images based on statistical texture. The steps of the method are described in Embodiment 1 and will not be repeated here. The semantic segmentation network includes a backbone network, a hollow spatial pyramid pooling module, a texture excitation module, a multi-granularity attention module, and an output module. The backbone network is used to extract the feature map X of the remote sensing image.

[0109] The void space pyramid pooling module is used to extract deep feature information to obtain feature map Y;

[0110] The texture excitation module is used to extract hidden statistical texture features from the feature map to obtain feature map Z;

[0111] The multi-granularity attention module includes a multi-granularity spatial attention module and a multi-granularity channel attention module. The output of the hollow spatial pyramid pooling module is used as the input of the multi-granularity spatial attention module, and the output of the texture excitation module is used as the input of the multi-granularity channel attention module. The multi-granularity attention module extracts image features from both the spatial domain and the channel domain at three levels: coarse-grained, medium-grained, and fine-grained, to obtain the correlation between each pixel.

[0112] The output module is used to merge the outputs of the multi-granularity spatial attention module and the multi-granularity channel attention module, and after upsampling, output the semantic segmentation result.

[0113] In summary, this invention (1) constructs a texture excitation module to extract statistical texture features. Based on the information of the remote sensing image itself, it uses statistical methods to extract the statistical texture features of the remote sensing image, enabling the model to simultaneously fuse structural texture features, statistical texture features, and semantic feature information, ultimately improving prediction accuracy. (2) It constructs a multi-granularity attention module. The multi-granularity attention module is based on a dual attention mechanism that includes a spatial attention module and a channel attention module. The multi-granularity attention module divides the feature map into different sizes, performs a dual attention mechanism on these divided feature maps, and assigns different weights to feature maps of different sizes, ultimately fusing these information. Since the feature map is divided into small feature maps of different sizes, it plays a key role in reducing the amount of interference information. At the same time, the multi-granularity attention mechanism observes the feature map from different perspectives, thereby gaining a more comprehensive understanding of the information on the feature map, ultimately improving prediction accuracy.

[0114] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should be protected by the present invention.

Claims

1. A semantic segmentation method for marine remote sensing images based on statistical texture, characterized in that, Includes the following steps: Step S1: Input the remote sensing image to be semantically segmented into a deep convolutional neural network for feature extraction to obtain feature map X; In step S2, feature map X will be processed by the hollow space pyramid pooling module and the texture excitation module to obtain feature map Y and feature map Z respectively; The texture excitation module aims to mine hidden texture features in the feature map using statistical methods. The method is as follows: For the input feature map, the pixel values ​​on the feature map are divided into different levels by average pooling and level division operations in sequence. Then, the statistical texture features of the image are extracted from the four directions of horizontal direction, vertical direction, main diagonal direction and anti-diagonal direction respectively. The texture features extracted from the four directions are merged to obtain statistical texture feature information. Step S3: Feature map Y is processed by a multi-granularity attention module to obtain feature map Y_M; feature map Z is processed by a multi-granularity attention module to obtain feature map Z_M; The multi-granularity attention module explores the deep information of the image from both the spatial domain and the channel domain, assigns different weights to each pixel, and obtains the correlation between each pixel; Step S4: Merge feature map Y_M and feature map Z_M, and perform upsampling to obtain the final semantic segmentation result.

2. The semantic segmentation method for marine remote sensing images based on statistical texture according to claim 1, characterized in that, The feature map X obtained in step S1 is used as the input of the texture excitation module, and the input of the texture excitation module is denoted as feature map A. The specific steps of the texture excitation module processing the feature map in step S2 are as follows: Step S21, Average Pooling Operation: Input feature map First, feature maps are obtained through average pooling operations along the channel dimension. Where C, H, and W represent the number of channels, height, and width of the feature map, respectively. Step S22, Level Classification Operation: First, the values ​​on feature map B are quantized into n+1 levels P = [P0, P1, ..., P2] by dividing the values ​​between the minimum and maximum values ​​of feature map B into N equal parts. n Specifically, level P n Calculated using the following formula: Where N = W = H, n ∈ [0, N], and max(B) and min(B) represent B respectively. i,j The maximum and minimum values ​​in; Next, the levels are determined based on the values ​​on feature map B and F(x) to obtain feature map C, where F(x) is as shown in formula (2): C i,j =m+1 if P m ≤B i,j <P m+1 (2) Where m∈[0,N-1], B i,j Represents the values ​​in the i-th row and j-th column of feature map B; C i,j The value of P represents the level to which the pixel in the i-th row and j-th column of the feature map C is classified; m P represents the value of the m-th level. m+1 This represents the value of the (m+1)th level; Step S23: Extract statistical texture features from the image in four directions: horizontal, vertical, main diagonal, and anti-diagonal, to obtain feature maps D, E, F, and G respectively. Then, merge and integrate the information of D, E, F, and G to obtain statistical texture feature information H. Step S24: Perform a 1×1 convolution on feature map H, which will change the number of channels to C1, resulting in feature map I.

3. The semantic segmentation method for marine remote sensing images based on statistical texture according to claim 2, characterized in that, Step S23: The operations of extracting statistical texture features from the image in the four directions of horizontal, vertical, main diagonal, and anti-diagonal are denoted as F1(x), F2(x), F3(x), and F4(x), respectively. The resulting feature maps D, E, F, and G are represented as follows: D p,q =Count{C id,jd ∣C id,jd =p,C id,jd+1 =q}, Where id∈[1,H],jd∈[1,W-1],p∈[1,N],q∈[1,N](3) E p,q =Count{C ie,je ∣C ie,je =p,C ie+1,je =q}, Where ie∈[1,H-1], je∈[1,W], p∈[1,N], q∈[1,N](4) F p,q =Count{C if,jf ∣C if,jf =p,C if+1,jf+1 =q}, Where if∈[1,H-1],jf∈[1,W-1],p∈[1,N],q∈[1,N](5) G p,q =Count{C ig,jg ∣C ig,jg =p,C ig-1,jg-1 =q}, Where ig∈[2,H],jg∈[2,W],p∈[1,N],q∈[1,N] (6) where p and q represent the assigned level, C id,jd The value C represents the level to which the pixel in row id and column jd is classified. id,jd+1 The value C represents the level to which the pixel in row id and column jd+1 is classified. ie,je The value of C indicates the level to which the pixel in row ie and column je is classified. ie+1,je The value of C indicates the level to which the pixel in row (ie+1) and column (je) is classified. if,jf The value of C indicates the level to which the pixel in the jf-th column of the if-th row is classified. if+1,jf+1 The value C indicates the level to which the pixel in row (if+1) and column (jf+1) is classified. ig,jg The value C represents the level to which the pixel in row ig and column jg is classified. ig-1,jg-1 The value indicates the level to which the pixel in row (ig-1) and column (jg-1) is classified; D p,q This represents the value in the p-th row and q-th column of feature map D. Count represents the summation function, and the formula within the summation function is called the discriminant. p,q The value of E is determined by the number of times the discriminant is true; p,q This represents the value in the p-th row and q-th column of feature map E. p,q The value of F is determined by the number of times the discriminant is true; p,q This represents the value in the p-th row and q-th column of the feature map F. p,q The value of G is determined by the number of times the discriminant is true; p,q G represents the value in the p-th row and q-th column of the feature map G. p,q The value of is determined by the number of times the discriminant is true.

4. The semantic segmentation method for marine remote sensing images based on statistical texture according to claim 1, characterized in that, In step S3, the multi-granularity attention module is designed based on the basic spatial attention module and the basic channel attention module. The multi-granularity attention module includes a multi-granularity spatial attention module and a multi-granularity channel attention module. The multi-granularity spatial attention module adopts a three-level architecture pattern of coarse-grained, medium-grained, and fine-grained. Each level is designed based on the basic spatial attention module. The feature maps output by the three levels of coarse-grained, medium-grained, and fine-grained are merged to obtain the final feature map E_S. The multi-granularity channel attention module still adopts a three-level architecture pattern of coarse-grained, medium-grained, and fine-grained. Each level is designed based on the basic channel attention module, and the feature maps output by the three levels of coarse-grained, medium-grained, and fine-grained are merged to obtain the final feature map E_C.

5. The semantic segmentation method for marine remote sensing images based on statistical texture according to claim 4, characterized in that, The input to the multi-granularity spatial attention module is denoted as the feature map. The multi-granularity spatial attention module adopts a three-level architecture. In the first level, the feature map A_S is input into the basic spatial attention module to obtain the feature map B_S. In the second level, the feature map A_S is divided into four parts A_S1, A_S2, A_S3, and A_S4 on an average basis along the H and W dimensions. These four parts are then input into the basic spatial attention module respectively to obtain the feature map B_S. Finally, these four parts are combined to obtain the final feature map. In the third level, feature map A_S is divided into 16 equally sized parts along the H and W dimensions, and after processing by the basic spatial attention module, feature map D_S is obtained. Finally, the three-level feature maps B_S, C_S, and D_S are merged to obtain the final feature map E_S. E_S = CAT(B_S, C_S, D_S) (7) CAT stands for merge operation.

6. The semantic segmentation method for marine remote sensing images based on statistical texture according to claim 4, characterized in that, The input to the multi-granularity channel attention module is denoted as the feature map. The multi-granularity channel attention module still adopts a three-level architecture. In the first level, the feature map... The input is fed into the basic channel attention module to obtain feature map B_C; in the second stage, feature map A_C is divided into four parts along dimension C: A_C1, A_C2, A_C3, and A_C4. These four parts are then fed into the basic channel attention module to obtain... Finally, these four parts are combined to obtain the final feature map. The third level uses the same idea. Feature map A_C is divided into 16 parts of the same size along the C dimension. After being processed by the basic spatial attention module, feature map D_C is obtained. Finally, the three-level feature maps B_C, C_C, and D_C are merged to obtain the final feature map E_C.

7. A semantic segmentation system for marine remote sensing images based on statistical texture, employing an encoder-decoder structure, characterized in that... The method for semantic segmentation of marine remote sensing images based on statistical texture as described in any one of claims 1-6 includes a backbone network, a hollow spatial pyramid pooling module, a texture excitation module, a multi-granularity attention module, and an output module. The backbone network is used to extract feature maps X from remote sensing images; The void space pyramid pooling module is used to extract deep feature information to obtain feature map Y; The texture excitation module is used to extract hidden statistical texture features from the feature map to obtain feature map Z; The multi-granularity attention module includes a multi-granularity spatial attention module and a multi-granularity channel attention module. The output of the hollow spatial pyramid pooling module is used as the input of the multi-granularity spatial attention module, and the output of the texture excitation module is used as the input of the multi-granularity channel attention module. The multi-granularity attention module extracts image features from both the spatial domain and the channel domain at three levels: coarse-grained, medium-grained, and fine-grained, to obtain the correlation between each pixel. The output module is used to merge the outputs of the multi-granularity spatial attention module and the multi-granularity channel attention module, and after upsampling, output the semantic segmentation result.

Images