Boundary auxiliary based remote sensing image detection method for enteromorpha

Abstract

This invention relates to the field of remote sensing image target recognition and information extraction technology, specifically to a boundary-assisted remote sensing image Ulva prolifera detection method, comprising the following steps: Step 1: Obtaining initial Ulva prolifera detection and Ulva prolifera boundary detection results from the remote sensing image using a convolutional neural network semantic segmentation architecture based on Ulva prolifera characteristics; Step 2: Optimizing the Ulva prolifera detection results from the remote sensing image based on Ulva prolifera boundary assistance. Addressing the problem that existing convolutional neural network-based semantic segmentation architectures are difficult to effectively apply to the detection of Ulva prolifera in high-resolution remote sensing images with different distribution shapes, blurred boundaries, and diffuse distributions, this invention proposes a boundary-assisted dual-path convolutional neural network method for high-resolution remote sensing image Ulva prolifera detection, which can be effectively applied to the accurate detection of Ulva prolifera in high-resolution remote sensing images with different distribution shapes, blurred boundaries, and diffuse distributions.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image target recognition and information extraction technology, specifically to a method for detecting seaweed in remote sensing images based on boundary assistance. Background Technology

[0002] As a recurring marine ecological disaster, *Ulva prolifera* causes significant losses to my country's marine industry every year. Therefore, the ability to quickly and accurately detect *Ulva prolifera* and monitor its condition is crucial for timely control, reducing its harm, and promoting the healthy development of my country's marine industry. Remote sensing imagery, with its large observation range, short revisit period, and multi-angle and multi-frequency observation capabilities, has become an important technical means for detecting *Ulva prolifera* disasters using remote sensing imagery. Extensive research has been conducted by scholars both domestically and internationally on *Ulva prolifera* detection using remote sensing imagery. Based on the method of generating *Ulva prolifera* features during the detection process, *Ulva prolifera* detection methods are divided into traditional methods and convolutional neural network methods. Traditional methods often extract spectral, textural, and chlorophyll content features of *Ulva prolifera*, and then classify the image based on these features to detect *Ulva prolifera*. This type of method can achieve good results in detecting *Ulva prolifera* in remote sensing imagery under specific conditions. However, due to the complex and varied shooting conditions of remote sensing satellites and the large amount of remote sensing images generated every day, this type of method is difficult to adapt to the detection of seaweed in large amounts of remote sensing images under different environments, and the robustness and universality of the algorithm are weak.

[0003] Because convolutional neural networks (CNNs) can automatically learn and generate highly complex nonlinear features based on their unique network structure, they overcome the limitations of manual design. When training data is sufficient, the target features automatically learned and extracted by CNNs exhibit good robustness and universality under various complex conditions. Therefore, scholars both domestically and internationally have applied semantic segmentation architectures based on CNNs to the detection and extraction of *Ulva prolifera* from remote sensing images. However, when remote sensing satellites instantaneously image the Earth from a top-down perspective, the *Ulva prolifera* regions in the acquired images exhibit blurred boundaries and diffuse distribution due to the influence of ocean waves, tides, and ocean currents. Existing semantic segmentation architectures based on CNNs are difficult to effectively apply to the detection of *Ulva prolifera* in high-resolution remote sensing images with different distribution shapes, blurred boundaries, and diffuse distributions.

[0004] To address the above problems, this invention proposes a high-resolution remote sensing image seaweed detection method based on boundary-assisted dual-path convolutional neural network. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a method for detecting seaweed in remote sensing images based on boundary assistance.

[0006] The technical solution adopted by this invention to solve its technical problem is: a method for detecting *Ulva prolifera* in remote sensing images based on boundary assistance, comprising the following steps:

[0007] Step 1: Based on the characteristics of Ulva prolifera, a convolutional neural network semantic segmentation architecture is used to obtain the initial Ulva prolifera detection and Ulva prolifera boundary detection results of the remote sensing image;

[0008] Step 2: Optimization and processing of Ulva prolifera detection results from remote sensing images based on Ulva prolifera boundary assistance.

[0009] Step one borrows the idea of ​​multi-attribute target feature fusion detection to extract multi-attribute features for the detection of seaweed regions and boundaries in the image. Based on the fusion processing of multi-attribute features of seaweed regions and boundaries, various types of seaweed regions in the image are detected.

[0010] Step one includes a dual-path convolutional neural network semantic segmentation architecture, one path for extracting the seaweed region in the image and the other path for extracting the seaweed region boundary in the image.

[0011] In the dual-path convolutional neural network architecture for semantic segmentation of seaweed in remote sensing images, the network parameters are optimized by integrating residual modules, combining low-level feature location information in the encoding stage with high-level feature semantic information in the decoding stage, and ensuring that the size scale of the input image and the output target attribute features is symmetrical.

[0012] In the dual-path convolutional neural network semantic segmentation architecture, each path of the convolutional neural network semantic segmentation network includes two stages: network encoding and decoding.

[0013] The network coding stage uses five levels of features to extract image features. At each level of features, there are two feature maps for extracting image features and a residual module to optimize the training parameters of the feature maps.

[0014] The decoding stage uses six levels of features for image feature extraction. The first five levels of features each have two feature maps, and the positional information of the corresponding lower level in the network coding stage is used for image feature extraction. The sixth level of features has one feature map, which is used for the fusion of extracted image features and pixel category classification.

[0015] In the decoding stage, a logistic layer is connected after the sixth-level feature map to map the output values ​​of the fifth-level features to the range of 0 to 1.

[0016] In convolutional neural networks, stochastic gradient descent is used to train the network parameters. The training loss of the semantic segmentation architecture is calculated based on the normalized ground truth labels and the output value of the logistic layer. The training loss of the dual-path convolutional neural network semantic segmentation architecture is calculated as follows:

[0017]

[0018]

[0019]

[0020] In the formula, L({EP) kijr},{EPB kijr}) and , respectively, are the training loss functions for the dual-path convolutional neural network semantic segmentation architecture, the seaweed detection network architecture, and the seaweed boundary detection network architecture; λ is the training loss balance coefficient for the seaweed detection network architecture and the seaweed boundary detection network architecture, with λ=0.4; N is the number of images in the min-batch; m and n are the height and width of the input image, respectively; EP kijr This represents the output value of the logistic layer of the seaweed detection network at position (r,j,i,k). This is the normalized ground truth label for the logistic layer of the seaweed detection network at position (r,j,i,k); EPB kijr This represents the output value of the logistic layer of the seaweed boundary detection network at position (r,j,i,k); This is the normalized ground truth label for the logistic layer of the seaweed boundary detection network at position (r,j,i,k).

[0021] Step one also includes testing the semantic segmentation architecture of high-resolution remote sensing image Ulva prolifera using a dual-path convolutional neural network. The output values ​​of the two logistic layers of the dual-path convolutional neural network are multiplied by 255 and mapped to the range of 0 to 255. The values ​​mapped by the two logistic layers are calculated using the following formula to generate two single-band images.

[0022]

[0023] In the formula, is the pixel value at position (i,j) in a single-band image; and These are the output values ​​of the logistic layer at position (i,j);

[0024] The pixel value of a single-band image is compared with a set threshold. If it is greater than or equal to the threshold, the pixel at the corresponding position in the prediction result is a seaweed or seaweed boundary. Otherwise, it is a background pixel, thus obtaining the initial seaweed detection result and seaweed boundary detection result of the image.

[0025] In step two, the results of the *Ulva prolifera* boundary detection are superimposed on the initial *Ulva prolifera* detection results for optimization. The optimization process follows these rules:

[0026] Rule 1: In the overlay result, region growing is performed based on whether the image pixels are classified as seaweed or not. If the number of pixels in a non-seaweed region is greater than or equal to 300, then the non-seaweed region is filled with seaweed.

[0027] Rule 2: In the region growth results, if the number of pixels in the seaweed region is less than 100, then the seaweed region is a noise region, and the seaweed category label of the region is changed to non-seaweed.

[0028] Compared with the prior art, the present invention has the following beneficial effects:

[0029] This application addresses the problem that existing semantic segmentation architectures based on convolutional neural networks are difficult to effectively apply to the detection of Ulva prolifera in high-resolution remote sensing images with different distribution shapes, blurred boundaries, and diffuse distributions. It proposes a boundary-assisted dual-path convolutional neural network-based method for Ulva prolifera detection in high-resolution remote sensing images, which can be effectively applied to the accurate detection of Ulva prolifera in high-resolution remote sensing images with different distribution shapes, blurred boundaries, and diffuse distributions. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the seaweed detection method in remote sensing images based on boundary-assisted dual-path convolutional neural network according to the present invention.

[0031] Figure 2 This is a schematic diagram of the semantic segmentation architecture for Ulva prolifera in remote sensing images using a dual-path convolutional neural network, as described in this invention.

[0032] Figure 3 This is a schematic diagram illustrating the optimized processing of Ulva prolifera detection results from remote sensing images based on Ulva prolifera boundary assistance according to the present invention. Detailed Implementation

[0033] The embodiments of the present invention will be further described below with reference to the accompanying drawings:

[0034] Example

[0035] like Figures 1 to 3 As shown, the present invention addresses the characteristics of Ulva prolifera in high-resolution remote sensing images, such as different distribution shapes, blurred boundaries, and diffuse distribution. The proposed method for detecting Ulva prolifera is mainly based on boundary-assisted dual-path convolutional neural network high-resolution remote sensing images.

[0036] Specifically, the boundary-assisted remote sensing image Ulva detection method includes the following steps:

[0037] Step 1: Using a convolutional neural network semantic segmentation architecture based on the characteristics of *Ulva prolifera*, initial *Ulva prolifera* detection and *Ulva prolifera* boundary detection results are obtained from the remote sensing image. Step 1 mainly uses a convolutional neural network semantic segmentation architecture based on the characteristics of *Ulva prolifera* to detect and extract attribute features such as *Ulva prolifera* regions and boundaries in the image.

[0038] In step one, the idea of ​​multi-attribute target feature fusion detection is adopted to extract multi-attribute features such as Ulva prolifera region detection and boundary detection in the image. Based on the fusion processing of multi-attribute features such as Ulva prolifera region and boundary, the accurate detection of various types of Ulva prolifera regions in the image is achieved.

[0039] Reference Figure 1 Step one includes a dual-path convolutional neural network semantic segmentation architecture, where one path is used to extract the *Ulva prolifera* region in the image, and the other path is used to extract the boundary of the *Ulva prolifera* region in the image. Leveraging the powerful adaptive target feature extraction and learning capabilities of the convolutional neural network, high-confidence initial *Ulva prolifera* detection results and *Ulva prolifera* boundary detection results with good boundary adhesion are obtained.

[0040] In the dual-path convolutional neural network architecture for semantic segmentation of seaweed in remote sensing images, the advantages of integrating residual modules to optimize training network parameters, combining low-level feature location information in the encoding stage with high-level feature semantic information in the decoding stage, and ensuring scale symmetry between input image and output target attribute feature size are utilized.

[0041] The residual module is used to eliminate the problems of "gradient vanishing" and "gradient explosion" during network training;

[0042] Combining low-level feature location information with high-level feature semantic information can more accurately obtain the spatial location of the target in the prediction result;

[0043] The input image and the output target attribute feature size are symmetrical, which can achieve a one-to-one correspondence between pixels in the input image and the prediction result. The detailed design of the dual-path convolutional neural network remote sensing image seaweed semantic segmentation architecture is as follows: Figure 2 As shown.

[0044] In the dual-path convolutional neural network semantic segmentation architecture, each path of the convolutional neural network semantic segmentation network includes two stages: network encoding and decoding.

[0045] The network coding stage uses five levels of features to extract image features. At each level of features, there are two feature maps for extracting image features and a residual module to optimize the training parameters of the feature maps.

[0046] The decoding stage uses six levels of features for image feature extraction. The first five levels of features each have two feature maps, and the positional information of the corresponding lower level in the network coding stage is used for image feature extraction. The sixth level of features has only one feature map, which is used for the fusion of extracted image features and pixel category classification.

[0047] In the decoding stage, a logistic layer is connected after the sixth-level feature map to map the output values ​​of the fifth-level features to the range of 0 to 1.

[0048] The formula for the logistic function s(x) is as follows:

[0049] Where x is the output value of the sixth-level feature layer.

[0050] The output value of the Logistic layer is used to calculate the network training loss and predict the network classification result. The input image size of the convolutional neural network is 240×240 pixels, and the kernel size, stride, and padding of the convolutional layer are 3, 1, and 1, respectively; the kernel size of the max pooling layer in the network encoding stage is 2×2. In the convolutional neural network, stochastic gradient descent is used to train the network parameters. In the seaweed and boundary labels of this application, the target region and the background region are represented by (255, 255, 255) and (0, 0, 0), respectively. When the network is training its parameters, the pixel values ​​in the seaweed and boundary labels are divided by 255 to map the pixel values ​​to 0~1, which is used to calculate the network loss function. In this application, the training loss of the semantic segmentation architecture is based on the normalized ground truth labels and the output value of the logistic layer. The training loss of the dual-path convolutional neural network semantic segmentation architecture is calculated as follows:

[0051]

[0052]

[0053]

[0054] In the formula, L({EP) kijr},{EPB kijr}) and , respectively, are the training loss functions for the dual-path convolutional neural network semantic segmentation architecture, the seaweed detection network architecture, and the seaweed boundary detection network architecture; λ is the training loss balance coefficient for the seaweed detection network architecture and the seaweed boundary detection network architecture, with λ=0.4; N is the number of images in the min-batch; m and n are the height and width of the input image, respectively; EP kijr This represents the output value of the logistic layer of the seaweed detection network at position (r,j,i,k). This is the normalized ground truth label for the logistic layer of the seaweed detection network at position (r,j,i,k); EPB kijr This represents the output value of the logistic layer of the seaweed boundary detection network at position (r,j,i,k); The normalized ground truth label is provided for the logistic layer of the seaweed boundary detection network at position (r,j,i,k). When training the seaweed boundary detection network, since there is a large difference between the number of target pixels and the number of background pixels, the balance coefficient α is used to balance the difference between the number of target pixels and the number of background pixels. In this application, the balance coefficient α is set to 1 / 50.

[0055] Step one also includes testing the semantic segmentation architecture of high-resolution remote sensing image Ulva prolifera using a dual-path convolutional neural network. The output values ​​of the two logistic layers of the dual-path convolutional neural network are multiplied by 255 and mapped to the range of 0 to 255. The values ​​mapped by the two logistic layers are calculated using the following formula to generate two single-band images.

[0056]

[0057] In the formula, is the pixel value at position (i,j) in a single-band image; and These are the output values ​​of the logistic layer at position (i,j);

[0058] The pixel value of a single-band image is compared with a set threshold. If it is greater than or equal to the threshold, the pixel at the corresponding position in the prediction result is a seaweed or seaweed boundary. Otherwise, it is a background pixel, thus obtaining the initial seaweed detection result and seaweed boundary detection result of the image.

[0059] Step 2: Optimization of Ulva prolifera detection results from remote sensing images based on Ulva prolifera boundary assistance. The initial Ulva prolifera detection results are optimized based on high-precision land cover boundary extraction results, achieving accurate detection of Ulva prolifera areas with different distribution patterns in high-resolution remote sensing images.

[0060] Due to the blurred boundaries and diffuse distribution of *Ulva prolifera* regions in high-resolution remote sensing images, it is difficult to accurately determine the boundaries of the *Ulva prolifera* regions from the initial detection results. However, boundary detection results can accurately determine the boundaries of the *Ulva prolifera* regions. Therefore, this invention proposes a remote sensing image *Ulva prolifera* detection result optimization method based on *Ulva prolifera* boundary assistance to optimize the initial *Ulva prolifera* detection results and obtain accurate remote sensing image *Ulva prolifera* region detection results. This method superimposes the *Ulva prolifera* boundary detection results onto the initial *Ulva prolifera* detection results. Some "gaps" and *Ulva prolifera* noise may exist in the superimposed results. This method fills the "gaps" and eliminates the *Ulva prolifera* noise using the following two rules, thereby obtaining accurate image *Ulva prolifera* region detection results with good boundary adhesion. The optimization process of the initial *Ulva prolifera* detection results based on *Ulva prolifera* boundary assistance is as follows: Figure 3As shown. In step two, the *Ulva prolifera* boundary detection results are superimposed on the initial *Ulva prolifera* detection results for optimization. The optimization process follows these rules:

[0061] Rule 1: In the overlay result, region growing is performed based on whether the image pixels are classified as seaweed or not. If the number of pixels in a non-seaweed region is greater than or equal to 300, then the non-seaweed region is filled with seaweed.

[0062] Rule 2: In the region growth results, if the number of pixels in the seaweed region is less than 100, then the seaweed region is a noise region, and the seaweed category label of the region is changed to non-seaweed.

[0063] After sequentially determining and processing the category attributes of all regions, accurate and well-defined high-resolution remote sensing imagery of Ulva prolifera regions is obtained.

Claims

1. A method for detecting Enteromorpha based on boundary auxiliary in remote sensing image, characterized in that, The method comprises the following steps: Step one: obtaining initial Enteromorpha detection and Enteromorpha boundary detection results of remote sensing images based on a convolutional neural network semantic segmentation architecture with Enteromorpha characteristics; the step one comprises a dual-path convolutional neural network semantic segmentation architecture, one path of which is used for extracting Enteromorpha regions in images, and the other path is used for extracting Enteromorpha region boundaries in images; In the dual-path convolutional neural network semantic segmentation architecture, each path of the convolutional neural network semantic segmentation network comprises two stages of network encoding and decoding; The network encoding stage adopts five levels of features to extract image features, and there are two feature maps on each level of features for extracting image features and a residual module for optimizing feature map training parameters; The decoding stage adopts six levels of features for image feature extraction, wherein there are two feature maps on each level of features in the first five levels of features, and the position information of the corresponding low-level in the network encoding stage is used for image feature extraction; there is one feature map on the sixth level of features for image feature fusion and pixel category classification; In the decoding stage, a logistic layer is connected behind the sixth level of feature maps, which is used for mapping the output value of the fifth level of features to the range of 0-1; In the convolutional neural network, the network parameters are trained using the stochastic gradient descent method, and the training loss of the semantic segmentation architecture is calculated based on the normalized true value label and the output value of the logistic layer. The training loss of the dual-path convolutional neural network semantic segmentation architecture is calculated as follows: ； ； ； wherein, , and are the training loss function of the dual-path convolutional neural network semantic segmentation architecture, the Enteromorpha detection network architecture and the Enteromorpha boundary detection network architecture, respectively; is the training loss balancing coefficient of the Enteromorpha detection network architecture and the Enteromorpha boundary detection network architecture, is 0.4; is the number of images contained in the min-batch; and are the height and width of the input image, respectively; is the output value of the Enteromorpha detection network logistic layer at the position; is the normalized ground truth label corresponding to the Enteromorpha detection network logistic layer at the position; is the output value of the Enteromorpha boundary detection network logistic layer at the position; is the normalized ground truth label corresponding to the Enteromorpha boundary detection network logistic layer at the position. The step one also comprises a dual-path convolutional neural network high-resolution remote sensing image Enteromorpha semantic segmentation architecture test. The output values of the two logistic layers of the dual-path convolutional neural network are multiplied by 255 to map them to the range of 0-255. The following formula is used to calculate the mapped values of the two logistic layers to generate two single-band images; ； wherein, is the pixel value at position in the single-band image; , and are the output values of the logistic layers at position , respectively. The pixel value of the single-band image is compared with the set threshold value. If it is greater than or equal to the threshold value, the pixel at the corresponding position in the prediction result of the single-band image is Enteromorpha or Enteromorpha boundary, otherwise it is background pixel, thereby obtaining the initial Enteromorpha detection result and the Enteromorpha boundary detection result of the image; Step two: optimization processing of remote sensing image Enteromorpha detection results based on Enteromorpha boundary assistance; In the step two, the Enteromorpha boundary detection result is superimposed on the initial Enteromorpha detection result for optimization processing. The following rules are followed during optimization processing: Rule one: in the superimposed result, region growing is performed based on image pixels of Enteromorpha and non-Enteromorpha categories. If the number of pixels of a non-Enteromorpha region is greater than or equal to 300, the non-Enteromorpha region is filled with Enteromorpha; Rule two: in the region growing result, if the number of pixels of an Enteromorpha region is less than 100, the Enteromorpha region is a noise region, and the Enteromorpha category label of the region is changed to non-Enteromorpha.

2. The boundary-assisted remote sensing image green tide detection method according to claim 1, characterized in that, In the step one, the multi-attribute target feature fusion detection idea is used to realize the extraction of multi-attribute features of Enteromorpha region detection and boundary detection in images, and the detection of various types of Enteromorpha regions in images is realized based on multi-attribute feature fusion processing of Enteromorpha regions and boundaries.

3. The boundary-assisted based remote sensing image green tide detection method according to claim 1, characterized in that, In the dual-path convolutional neural network remote sensing image macroalgal semantic segmentation architecture, the residual module is fused to optimize the training network parameters, the position information of the bottom layer features in the encoding stage is combined with the semantic information of the high layer features in the decoding stage, and the input image and the output target attribute feature size scale are symmetrical.

Images