Spartina alterniflora remote sensing image segmentation method in extreme cloudy and foggy weather

Abstract

The invention provides a spartina alterniflora remote sensing image segmentation method in extreme cloudy and foggy weather. The spartina alterniflora remote sensing image segmentation method in the extreme cloudy and foggy weather comprises the following steps: S1, filtering a spartina alterniflora remote sensing image; s2, establishing a spartina alterniflora detail information feature extraction network; s3, establishing a spartina alterniflora semantic information feature extraction network; and S4, fusing the detail information and the semantic information by using a bilateral aggregation guide layer. The spartina alterniflora remote sensing image segmentation method in the extreme cloudy and foggy weather provided by the invention has the advantages that the segmentation precision of the spartina alterniflora can be improved in the extreme cloudy and foggy weather, so that the remote sensing data quality is improved.

Description

Technical Field

[0001] The invention relates to the technical field of remote sensing image segmentation, and in particular to a method for segmenting a Spartina alterniflora remote sensing image under extreme foggy weather. Background Art

[0002] Spartina alterniflora is a perennial herb native to South America. It is commonly found in water environments such as rivers, lakes and rice fields. It is an invasive species in my country. Spartina alterniflora has the ability to grow and reproduce rapidly, and can quickly occupy waters and wetlands, posing a threat to my country's ecosystem. At present, the segmentation technology of Spartina alterniflora usually extracts features related to Spartina alterniflora from remote sensing images, including vegetation index, texture features, shape features, etc. After feature extraction, different objects in the image are classified by classification algorithms, such as support vector machines, maximum likelihood classification, random forests, etc. The algorithm divides the image pixels into different categories according to the extracted features, and finally segments the Spartina alterniflora. However, for the Spartina alterniflora data in remote sensing images, the traditional machine learning model cannot fully capture the deep features in the data using the extracted features, thus affecting the classification accuracy of Spartina alterniflora.

[0003] With the development of computer vision and artificial intelligence, deep learning models have been applied to the segmentation of Spartina alterniflora. However, there are few technologies for segmenting Spartina alterniflora in extreme foggy weather. Spartina alterniflora is mainly distributed in the Yangtze River Basin and the Pearl River Basin, which often have more foggy weather in summer and the rainy season. In foggy weather, remote sensing images will be affected by changes in lighting conditions and scattering, resulting in more noise in the images, which greatly reduces the segmentation accuracy of Spartina alterniflora. This requires the fusion of information of different scales in remote sensing images to compensate for the impact of fog on the quality of remote sensing image data.

[0004] The invention discloses a method for extracting Spartina alterniflora remote sensing images of a coastal salt marsh wetland based on deep learning, which belongs to the field of geological survey and research of coastal salt marsh wetlands. The method comprises the following steps: constructing a Spartina alterniflora extraction model and training it with a remote sensing image data set; the Spartina alterniflora extraction model takes D-LinkNet34 as a basic model, adopts an encoder and decoder architecture, and stacks the void convolution in a cascade and parallel mode in a jump connection part, and embeds a convolution attention module in the jump connection part, so that the Spartina alterniflora features extracted by the encoder at each stage are first passed through the convolution attention module and then connected to the decoder part; finally, based on the trained Spartina alterniflora extraction model, the extraction of Spartina alterniflora images of a coastal salt marsh wetland is realized.

[0005] This method does not consider the identification of Spartina alterniflora in extreme foggy weather. In extreme foggy weather, remote sensing images will be affected by changes in lighting conditions and scattering, resulting in more noise in the image, which greatly reduces the recognition accuracy of Spartina alterniflora. At the same time, this method does not fully integrate information of different scales in the image, ignoring the characteristics of Spartina alterniflora in image detail information and semantic information.

[0006] Therefore, it is necessary to provide a new remote sensing image segmentation method for Spartina alterniflora under extreme foggy weather to solve the above technical problems. Summary of the invention

[0007] The technical problem solved by the invention is to provide a method for segmenting a remote sensing image of Spartina alterniflora in extreme foggy weather, which can improve the segmentation accuracy of Spartina alterniflora in extreme foggy weather, thereby improving the quality of remote sensing data.

[0008] In order to solve the above technical problems, the present invention provides a method for segmenting a remote sensing image of Spartina alterniflora under extreme foggy weather, comprising the following steps:

[0009] S1: Spartina alterniflora remote sensing image filtering;

[0010] S2: Establish a feature extraction network for detail information of Spartina alterniflora;

[0011] S3: Establish a semantic information feature extraction network for Spartina alterniflora;

[0012] S4: Using bilateral aggregation guided layer to fuse detail information and semantic information.

[0013] Preferably, in S1, the image is processed by median filtering, a filter window n is set, the median of the pixel points in each window is calculated, and the median is assigned to the point at the center of the window, and the process is repeated until the entire Spartina alterniflora remote sensing image is covered, and the calculation formula is as follows:

[0014] Among them, is the pixel value of the position in the output image, is the pixel value of the position in the input image, is the function of finding the mean, and n is the window size.

[0015] Preferably, in S2, the Spartina alterniflora detail information feature extraction network extracts local features through three convolution operations to capture edge information:

[0016] The input image is convolved through the first convolutional layer Conv1, and the number of output channels is half of the number of input channels, and the input image is downsampled; after passing through the Conv1 convolutional layer, the obtained feature map is input into the next convolutional layer Conv2; the number of output channels of the Conv2 convolutional layer is half of the number of output channels of Conv1, further reducing the dimension of the feature map; the number of output channels of the Conv3 convolutional layer is further reduced to half of the number of output channels of Conv2, and finally the calculation results of the Spartina alterniflora detail information feature extraction network are obtained.

[0017] Preferably, in S3, the semantic information feature extraction network is responsible for capturing the contextual information of the global Spartina alterniflora image, extracting higher-level abstract features, and distinguishing Spartina alterniflora from other types of objects from the overall pattern and large-scale object categories in the segmented image. Through a deeper convolutional network and downsampling operation, the semantic branch can extract a wide range of contextual information from the remote sensing image, solving the scale change and complex background problems in the remote sensing image. The specific steps are as follows:

[0018] S31: The semantic information feature extraction network processes the Spartina alterniflora image through the backbone module and increases the number of channels while reducing the spatial dimension, thereby capturing the distribution pattern of Spartina alterniflora at different scales;

[0019] S32: The semantic information feature extraction network captures the contextual information of the Spartina alterniflora feature image through the context embedding module to obtain a richer feature representation.

[0020] Preferably, in said S31, the specific steps are as follows:

[0021] S31A: Perform a 3×3 convolution operation on the input image to obtain a feature map after the operation;

[0022] S31B: perform batch normalization on , and use the ReLU activation function to calculate to obtain a feature map;

[0023] S31C: Perform 1×1 convolution operation on to obtain a feature map;

[0024] S31D: perform batch normalization on , and use the ReLU activation function to calculate to obtain the feature map;

[0025] S31E: Perform a 3×3 convolution operation on the image to obtain a feature map;

[0026] S31F: perform batch normalization on , and use the ReLU activation function to calculate to obtain the feature map;

[0027] S31G: Perform a 3×3 maximum pooling operation to obtain a feature map;

[0028] S31H: Connect with the output of the maximum pooling layer to obtain a feature map;

[0029] S31I: Perform 3×3 convolution operation on to obtain feature map;

[0030] S31J: Perform batch normalization on , and use the ReLU activation function to calculate, and get the final Spartina alterniflora feature map output.

[0031] Preferably, in said S32, the specific steps are as follows:

[0032] S32A: Perform global average pooling calculation on the input Spartina alterniflora feature map:

[0033] Among them, is the value of the c-th channel of the input feature map, and is the value of the c-th channel after pooling;

[0034] S32B: perform batch normalization on the pooled data to obtain a feature map;

[0035] S32C: Perform 1×1 convolution operation on to obtain the feature map;

[0036] S32D: perform batch normalization on the data and use the ReLU activation function to calculate and obtain the feature map;

[0037] S32E: Add the broadcasted and input feature maps to obtain a feature map;

[0038] S32F: Perform a 3×3 convolution operation to obtain a feature map.

[0039] Preferably, in S4, after the detail information feature extraction network and the semantic information feature extraction network have completed their respective Spartina alterniflora image information extraction tasks, the bilateral aggregation guidance layer fuses the features of the two, and the specific calculation steps are as follows:

[0040] S41: performing a convolution operation on the feature map calculated by the detail information feature extraction network, and performing an element-by-element product with the up-sampling result of the semantic information feature extraction network;

[0041] S41: Perform convolution operation on the feature map of the semantic branch, process the convolution result through the Sigmoid function, and perform element-by-element product with the average pooling result of the detail branch;

[0042] S41: Add the operation results of the detail branch and the semantic branch pixel by pixel, and perform a convolution operation on the addition result to obtain the final output result.

[0043] Compared with the related art, the remote sensing image segmentation method of Spartina alterniflora under extreme foggy weather provided by the present invention has the following beneficial effects:

[0044] The invention provides a method for segmenting a Spartina alterniflora remote sensing image under extreme foggy weather. The method comprises the following steps: extracting detail information in a Spartina alterniflora remote sensing image through a Spartina alterniflora detail feature extraction network, reducing the influence of noise on detail features such as edge and texture of a Spartina alterniflora object under foggy scenes; extracting semantic information in a Spartina alterniflora remote sensing image through a Spartina alterniflora semantic feature extraction network, understanding the semantic meanings of different regions in the remote sensing image, and improving the accuracy of Spartina alterniflora segmentation; using a bilateral guided aggregation layer to fuse detail information and semantic information in a Spartina alterniflora remote sensing image, using context information obtained by a semantic information extraction module to guide the output of a detail information feature map, and improving the accuracy of Spartina alterniflora segmentation under the condition of reducing the influence of foggy weather on the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A structural block diagram of the main modules of the method for segmenting the remote sensing image of Spartina alterniflora under extreme foggy weather provided by the present invention;

[0046] Figure 2 A structural block diagram of a context embedding module of a method for segmenting a Spartina alterniflora remote sensing image under extreme foggy weather provided by the present invention;

[0047] Figure 3 This is a structural block diagram of the bilateral aggregation guidance layer of the Spartina alterniflora remote sensing image segmentation method under extreme foggy weather provided by the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described below in conjunction with the accompanying drawings and implementation modes.

[0049] Please refer to Figure 1 , Figure 2 and Figure 3 ,in, Figure 1 A structural block diagram of the main modules of the method for segmenting the remote sensing image of Spartina alterniflora under extreme foggy weather provided by the present invention; Figure 2 A structural block diagram of a context embedding module of a method for segmenting a Spartina alterniflora remote sensing image under extreme foggy weather provided by the present invention; Figure 3The present invention provides a bilateral aggregation guidance layer structure block diagram of the method for segmenting the remote sensing image of Spartina alterniflora under extreme foggy weather. A method for segmenting the remote sensing image of Spartina alterniflora under extreme foggy weather based on information fusion. First, the detail information in the remote sensing image of Spartina alterniflora is extracted by a network detail branch to reduce the influence of noise on detail features such as the edge and texture of the Spartina alterniflora object in foggy scenes. Then, the semantic information in the remote sensing image of Spartina alterniflora is extracted by a network semantic branch to understand the semantic meaning of different regions in the remote sensing image, thereby improving the accuracy of the image segmentation of Spartina alterniflora. Finally, the detail information and semantic information in the remote sensing image of Spartina alterniflora are fused using a bilateral guidance aggregation layer, and the context information obtained by the semantic information extraction module is used to guide the output of the detail branch feature map, and the final segmentation result is generated by the segmentation head. To realize the above technical solution, the present invention comprises the following steps:

[0050] S1: Spartina alterniflora remote sensing image filtering

[0051] The quality of remote sensing images collected in foggy weather will be seriously affected. The images are processed by median filtering. Set the filter window n, calculate the median of the pixels in each window, and assign the median to the point in the center of the window. Repeat this process until the entire Spartina alterniflora remote sensing image is covered. The calculation formula is as follows:

[0052] Output(x,y)=median{Input(x+i,y+j)|i=-n,...,n,j=-n,...,n}

[0053] Among them, Output(x,y) is the pixel value at position (x,y) in the output image, Input(x+i,y+j) is the pixel value at position (x+i,y+j) in the input image, median is the function for finding the mean, and n is the window size. In this way, median filtering can eliminate the impact of fog on remote sensing images to a certain extent.

[0054] S2: Establishing a feature extraction network for Spartina alterniflora detail information

[0055] The spatial distribution of Spartina alterniflora involves the precise extraction of small areas and edge contours. The detail information feature extraction network is used to capture subtle features in Spartina alterniflora images, such as the boundaries and shapes of objects such as leaves and stems of Spartina alterniflora, thereby reducing the impact of noise on detail features such as the edges and textures of Spartina alterniflora objects in foggy scenes.

[0056] The feature extraction network for the detail information of Spartina alterniflora extracts local features and captures edge information through three convolution operations. The input image is convolved through the first convolution layer Conv1, and the number of output channels is half of the number of input channels, and the input image is downsampled; after the Conv1 convolution layer, the feature map obtained is input into the next convolution layer Conv2; the number of output channels of the Conv2 convolution layer is half of the number of output channels of Conv1, further reducing the dimension of the feature map; the number of output channels of the Conv3 convolution layer is further reduced to half of the number of output channels of Conv2, and finally the calculation result X1 of the feature extraction network for the detail information of Spartina alterniflora is obtained. The basic information of the convolution layer of the network is shown in Table 1. After three layers of convolution, the detail branch outputs a high-resolution feature map for subsequent fusion with the semantic branch.

[0057] Table 1 Basic structure of detail branch

[0058]

[0059]

[0060] S3: Establishing a semantic information feature extraction network for Spartina alterniflora

[0061] The semantic information feature extraction network is responsible for capturing the contextual information of the global Spartina alterniflora image, extracting higher-level abstract features, and distinguishing Spartina alterniflora from other vegetation, water bodies, or bare land from the overall pattern and large-scale object categories in the segmented image. Through deeper convolutional networks and downsampling operations, the semantic branch can extract extensive contextual information from remote sensing images and solve the scale changes and complex background problems in remote sensing images. The specific steps are as follows:

[0062] S31: The semantic information feature extraction network processes the Spartina alterniflora image through the backbone module, and increases the number of channels while reducing the spatial dimension, so as to capture the distribution pattern of Spartina alterniflora at different scales, especially the global features such as the overall vegetation coverage and the spatial relationship between objects. The backbone module structure is shown in the attached figure. Figure 1 As shown, the specific steps are as follows:

[0063] S31A: Perform a 3×3 convolution operation on the input image to obtain a feature map M1 after the operation;

[0064] S31B: perform batch normalization on M1 and use the ReLU activation function to calculate to obtain the feature map M2;

[0065] S31C: Perform a 1×1 convolution operation on M2 to obtain feature map M3;

[0066] S31D: perform batch normalization on M3 and use the ReLU activation function to calculate to obtain the feature map M4;

[0067] S31E: Perform a 3×3 convolution operation on M4 to obtain a feature map M5;

[0068] S31F: perform batch normalization on M5 and use the ReLU activation function to calculate to obtain the feature map M6;

[0069] S31G: Perform a 3×3 MaxPooling operation on M6 to obtain feature map M7;

[0070] S31H: Connect M6 with the output M7 of the maximum pooling layer to obtain feature map M8;

[0071] S31I: Perform a 3×3 convolution operation on M8 to obtain feature map M9;

[0072] S31J: Perform batch normalization on M9 and use the ReLU activation function to calculate and obtain the final Spartina alterniflora feature map output.

[0073] S32: The semantic information feature extraction network captures the context information of the Spartina alterniflora feature image through the context embedding module to obtain a richer feature representation. The structure diagram is shown in the attached figure. Figure 2 As shown, the specific steps are as follows:

[0074] S32A: Perform global average pooling calculation on the input Spartina alterniflora feature map:

[0075]

[0076] Among them, f(i,j,c) is the value of the cth channel of the input feature map at position (i,j), f gap (c) is the value of the cth channel after pooling.

[0077] S32B: Perform batch normalization on the pooled data to obtain the feature map M 10 ;

[0078] S32C: For M 10 Perform a 1×1 convolution operation to obtain the feature map M 11 ;

[0079] S32D: For M 11 Perform batch normalization and use the ReLU activation function to calculate and obtain the feature map M 12 :

[0080] S32E: M after broadcast 12 Add it to the input feature map f(i,j,c) to get the feature map M13 :

[0081] S32F: For M 13 Perform a 3×3 convolution operation to obtain the feature map M 14 :

[0082] S4: Using bilateral aggregation guidance layer to fuse detail information and semantic information

[0083] After the detail information feature extraction network and the semantic information feature extraction network have completed their respective Spartina alterniflora image information extraction tasks, the bilateral aggregation guidance layer fuses the features of the two. The structure is shown in the attached figure. Figure 3 As shown, the specific calculation steps are as follows:

[0084] S41: performing a convolution operation on the feature map calculated by the detail information feature extraction network, and performing an element-by-element product with the up-sampling result of the semantic information feature extraction network;

[0085] S42: Perform convolution operation on the feature map of the semantic branch, process the convolution result through the Sigmoid function, and perform element-by-element product with the average pooling result of the detail branch;

[0086] S43: Add the operation results of the detail branch and the semantic branch pixel by pixel, and perform a convolution operation on the addition result to obtain the final output result.

[0087] The working principle of the Spartina alterniflora remote sensing image segmentation method under extreme foggy weather provided by the present invention is as follows:

[0088] first.

[0089] Compared with the related art, the remote sensing image segmentation method of Spartina alterniflora under extreme foggy weather provided by the present invention has the following beneficial effects:

[0090] The invention provides a method for segmenting a Spartina alterniflora remote sensing image under extreme foggy weather. The method comprises the following steps: extracting detail information in a Spartina alterniflora remote sensing image through a Spartina alterniflora detail feature extraction network, reducing the influence of noise on detail features such as edge and texture of a Spartina alterniflora object under foggy scenes; extracting semantic information in a Spartina alterniflora remote sensing image through a Spartina alterniflora semantic feature extraction network, understanding the semantic meanings of different regions in the remote sensing image, and improving the accuracy of Spartina alterniflora segmentation; using a bilateral guided aggregation layer to fuse detail information and semantic information in a Spartina alterniflora remote sensing image, using context information obtained by a semantic information extraction module to guide the output of a detail information feature map, and improving the accuracy of Spartina alterniflora segmentation under the condition of reducing the influence of foggy weather on the image.

[0091] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for segmenting a remote sensing image of Spartina alterniflora under extreme foggy weather, characterized in that: The following steps are involved: S1: Spartina alterniflora remote sensing image filtering; S2: Establish a feature extraction network for detail information of Spartina alterniflora; S3: Establish a semantic information feature extraction network for Spartina alterniflora; S4: Using bilateral aggregation guided layer to fuse detail information and semantic information.

2. The method for segmenting the remote sensing image of Spartina alterniflora under extreme foggy weather according to claim 1, characterized in that: In S1, the image is processed by median filtering, a filter window n is set, the median of the pixel points in each window is calculated, and the median is assigned to the point at the center of the window, and the process is repeated until the entire Spartina alterniflora remote sensing image is covered. The calculation formula is as follows: Output(x,y)=median{Input(x+i,y+j)|i=-n,...,n,j=-n,...,n} Among them, Output(x,y) is the pixel value at position (x,y) in the output image, Input(x+i,y+j) is the pixel value at position (x+i,y+j) in the input image, median is the function for finding the mean, and n is the window size.

3. The method for segmenting Spartina alterniflora remote sensing images under extreme foggy weather according to claim 1, characterized in that: In S2, the Spartina alterniflora detail information feature extraction network extracts local features through three convolution operations to capture edge information: The input image is convolved through the first convolutional layer Conv1, and the number of output channels is half of the number of input channels, and the input image is downsampled; After passing through the Conv1 convolution layer, the obtained feature map is input into the next convolution layer Conv2; the number of output channels of the Conv2 convolution layer is half of the number of output channels of Conv1, further reducing the dimension of the feature map; the number of output channels of the Conv3 convolution layer is further reduced to half of the number of output channels of Conv2, and finally the calculation results of the Spartina alterniflora detail information feature extraction network are obtained.

4. The method for segmenting Spartina alterniflora remote sensing images under extreme foggy weather according to claim 1, characterized in that: In S3, the semantic information feature extraction network is responsible for capturing the context information of the global Spartina alterniflora image, extracting higher-level abstract features, and distinguishing Spartina alterniflora from other types of objects from the overall pattern and large-scale object categories in the segmented image. Through deeper convolutional networks and downsampling operations, the semantic branch can extract extensive context information from remote sensing images and solve the scale changes and complex background problems in remote sensing images. The specific steps are as follows: S31: The semantic information feature extraction network processes the Spartina alterniflora image through the backbone module and increases the number of channels while reducing the spatial dimension, thereby capturing the distribution pattern of Spartina alterniflora at different scales; S32: The semantic information feature extraction network captures the contextual information of the Spartina alterniflora feature image through the context embedding module to obtain a richer feature representation.

5. The method for segmenting Spartina alterniflora remote sensing images under extreme foggy weather according to claim 4, characterized in that: In the S31, the specific steps are as follows: S31A: Perform a 3×3 convolution operation on the input image to obtain a feature map after the operation; S31B: perform batch normalization on , and use the ReLU activation function to calculate to obtain a feature map; S31C: Perform 1×1 convolution operation on to obtain a feature map; S31D: perform batch normalization on , and use the ReLU activation function to calculate to obtain the feature map; S31E: Perform a 3×3 convolution operation on the image to obtain a feature map; S31F: perform batch normalization on , and use the ReLU activation function to calculate to obtain the feature map; S31G: Perform 3×3 MaxPooling operation to obtain the feature map; S31H: Connect with the output of the maximum pooling layer to obtain a feature map; S31I: Perform 3×3 convolution operation on to obtain feature map; S31J: Perform batch normalization on , and use the ReLU activation function to calculate, and get the final Spartina alterniflora feature map output.

6. The method for segmenting Spartina alterniflora remote sensing images under extreme foggy weather according to claim 4, characterized in that: In the S32, the specific steps are as follows: S32A: Perform global average pooling calculation on the input Spartina alterniflora feature map: Among them, f(i,j,c) is the value of the cth channel of the input feature map at position (i,j), f gap (c) is the value of the cth channel after pooling; S32B: Perform batch normalization on the pooled data to obtain the feature map M 10 ; S32C: For M 10 Perform a 1×1 convolution operation to obtain the feature map M 11 ; S32D: For M 11 Perform batch normalization and use the ReLU activation function to calculate and obtain the feature map M 12 ; S32E: M after broadcast 12 Add it to the input feature map f(i,j,c) to get the feature map M 13 ; S32F: For M 13 Perform a 3×3 convolution operation to obtain the feature map M 14 .

7. The method for segmenting Spartina alterniflora remote sensing images under extreme foggy weather according to claim 1, characterized in that: In S4, after the detail information feature extraction network and the semantic information feature extraction network have completed their respective Spartina alterniflora image information extraction tasks, the bilateral aggregation guidance layer fuses the features of the two. The specific calculation steps are as follows: S41: performing a convolution operation on the feature map calculated by the detail information feature extraction network, and performing an element-by-element product with the up-sampling result of the semantic information feature extraction network; S41: Perform convolution operation on the feature map of the semantic branch, process the convolution result through the Sigmoid function, and perform element-by-element product with the average pooling result of the detail branch; S41: Add the operation results of the detail branch and the semantic branch pixel by pixel, and perform a convolution operation on the addition result to obtain the final output result.

Images