Remote sensing image ground object segmentation method based on hierarchical feature extraction and decoding

Abstract

The invention discloses a remote sensing image ground object segmentation method based on hierarchical feature extraction and decoding, which relates to the technical field of remote sensing image processing, and comprises the following steps: step 1, collecting and preprocessing a remote sensing image, and constructing a training sample set; 2, constructing an image segmentation network, wherein the image segmentation network comprises a hierarchical encoder, a receptive field module unit and a network decoder which are connected in sequence; 3, training and optimizing the image segmentation network by adopting the training sample set to obtain a remote sensing image ground object segmentation model; and 4, collecting a remote sensing image to be segmented, and inputting the remote sensing image to the remote sensing image ground object segmentation model to obtain a ground object segmentation result. According to the method, the advantages of the advanced visual basic model and the U-shaped network architecture are combined, so that rapid and accurate segmentation of the ground target is realized.

Description

Technical Field

[0001] The invention relates to the technical field of remote sensing image processing, and more particularly to a remote sensing image object segmentation method based on hierarchical feature extraction and decoding. Background Art

[0002] In today's information age, remote sensing technology has become an important means of obtaining surface information. Remote sensing images, with their wide coverage, short update cycle, and relatively low cost, play an irreplaceable role in military reconnaissance, land and resources surveys, environmental monitoring and other fields. As the core link of remote sensing image processing, object segmentation aims to identify and distinguish various objects from images, such as roads, buildings, water bodies, etc., to provide basic data for subsequent analysis and application.

[0003] However, the segmentation of objects in remote sensing images faces a series of technical challenges. First, remote sensing images usually contain a variety of object types, which vary significantly in shape, size, texture, etc., which increases the difficulty of segmentation. Second, the boundaries between objects are often blurred, especially for complex scenes such as cities, where mutual occlusion and penetration between different objects make boundary identification more difficult. In addition, the imaging conditions of remote sensing images are changeable, such as the influence of factors such as lighting, weather, and season, which leads to fluctuations in image quality, further increasing the complexity of segmentation. Traditional remote sensing image segmentation methods, such as pixel-based threshold segmentation and region-based merging and growing algorithms, often rely on low-level features of images, such as color and texture, and lack the grasp of deep semantic information of objects. With the rise of deep learning technology, methods based on convolutional neural networks (CNNs) have made breakthrough progress in the field of image segmentation due to their powerful feature extraction capabilities and ability to recognize complex patterns. In particular, structures such as fully convolutional networks (FCNs) and U-shaped networks (UNets) can automatically learn the spatial hierarchy and contextual information in images through end-to-end training, significantly improving the accuracy and efficiency of segmentation.

[0004] Although deep learning-based segmentation methods have achieved certain success, they still have limitations when processing remote sensing images. On the one hand, the resolution of remote sensing images is high, and the computational cost of segmentation directly at the original resolution is high, and it is difficult to process large-scale objects. On the other hand, most existing methods focus on feature extraction at a single scale, which makes it difficult to capture multi-scale object information, limiting the generalization ability of the model. In addition, the class imbalance problem in remote sensing images also poses a challenge to the performance of the model, requiring the model to be able to effectively segment objects of different sizes and shapes.

[0005] Therefore, how to improve the accuracy of ground object segmentation in remote sensing images is a problem that technical personnel in this field need to solve urgently. Summary of the invention

[0006] In view of this, the present invention provides a remote sensing image ground object segmentation method based on hierarchical feature extraction and decoding, aiming to improve the segmentation accuracy and efficiency of ground targets in remote sensing images. By combining the advantages of advanced visual basic model and U-shaped network architecture, fast and accurate segmentation of ground targets is achieved, which is particularly suitable for processing large-scale remote sensing image data.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A remote sensing image object segmentation method based on hierarchical feature extraction and decoding comprises the following steps:

[0009] Step 1: Collect remote sensing images and preprocess them to construct a training sample set;

[0010] Step 2: Construct an image segmentation network, which includes a hierarchical encoder, a receptive field module unit, and a network decoder connected in sequence;

[0011] Step 3: Use the training sample set to train and optimize the image segmentation network to obtain the remote sensing image object segmentation model;

[0012] Step 4: Collect the remote sensing image to be segmented, input it into the remote sensing image ground object segmentation model, and obtain the ground target segmentation result.

[0013] Preferably, the preprocessing in step 1 includes operations such as denoising and contrast enhancement, aiming to improve image quality so as to improve the accuracy of subsequent feature extraction.

[0014] Preferably, the hierarchical encoder adopts a pre-trained Hiera architecture, which can effectively capture multi-scale features and provide rich semantic information for subsequent segmentation tasks; the hierarchical encoder can be represented as a sequence model, including a plurality of encoding units connected in sequence, and the output of the previous encoding unit is used as the input of the next encoding unit; each encoding unit includes a normalization layer, an attention mechanism layer and a multi-layer perceptron connected in sequence; each encoding unit outputs a hierarchical feature map to form a multi-scale feature; the feature map X i =Encoder(X i-1 ), i=2, ..., N, N represents the number of coding units, N=4, C i represents the number of feature channels; X1 = Encoder (I), I represents the input image, Where H represents the image height and W represents the image width.

[0015] Preferably, the receptive field module unit includes 4 receptive field modules, each of which is used to reduce the number of feature channels and enhance the feature expression capability, including a multi-branch convolution layer, an expansion pooling or convolution layer, a merging layer, a skip connection layer, and an activation function layer connected in sequence, wherein:

[0016] The multi-branch convolutional layer adopts the Inception network (such as the structure in InceptionV4 and Inception-ResNet V2), including a bottleneck structure and stacked convolutional layers connected in sequence. The bottleneck structure includes three 1×1 convolutional layers to reduce the number of feature channels. The stacked convolutional layers include a 1×1 convolutional layer, a 3×3 convolutional layer, a 1×3 convolutional layer, and a 3×1 convolutional layer. The three 1×1 convolutional layers of the bottleneck structure are respectively connected to the 1×3 convolutional layer, the 3×1 convolutional layer, and the 3×3 convolutional layer of the stacked convolutional layer; multi-scale features are input to the bottleneck structure of the multi-branch convolutional layer and the 1×1 convolutional layer of the stacked convolutional layer;

[0017] The dilated pooling or convolution layer includes four 3×3 dilated convolution layers with different dilation rates, which are 1, 3, and 5 respectively, including two 3×3 dilated convolution layers with a dilation rate of 3 connected to the 1×3 convolution layer and the 3×1 convolution layer of the stacked convolution layer, a 3×3 dilated convolution layer with a dilation rate of 1 connected to the 1×1 convolution layer of the stacked convolution layer, and a 3×3 dilated convolution layer with a dilation rate of 5 connected to the 3×3 convolution layer of the stacked convolution layer; the dilated convolution layer is used to capture a wider range of context information and enhance the feature expression capability; in the bottleneck structure, a 1×1 convolution layer is sequentially connected to a convolution layer in the stacked convolution layer and a dilated convolution layer in the dilated pooling or convolution layer to form a convolution branch, and output a branch feature map;

[0018] The merging layer merges all branch feature maps to form a spatial pooling or convolution array, expressed as:

[0019] F multi-branch =concat(f1(X i ), f2(X i ), ..., f n (X i ))

[0020] Among them, F multi-branch represents a spatial pooling or convolution array; f1, f2, ..., f n Respectively represent the convolution operations of different convolution branches; X i Represents the input feature map; concat() represents the concatenation operation;

[0021] The skip connection layer inputs the multi-scale features and spatial pooling or convolution array output by the hierarchical encoder into the activation function layer;

[0022] The activation function layer obtains different enhanced features X based on multi-scale features and spatial pooling or convolution arrays. output , forming multi-scale enhanced features, enhanced features X output It is expressed as:

[0023] X output =ReLU(F multi-branch +X shortcut )

[0024] Among them, X shortcut ReLU is the activation function. ReLU is used to increase the nonlinear expression ability of the model and enhance the feature X. output It can effectively combine information from different receptive fields and enhance the expression of multi-scale features while maintaining the input feature information.

[0025] Preferably, the network decoder comprises three decoding blocks connected in sequence, each decoding block comprises two decoding groups, a 1×1 convolution layer and an upsampling layer, and each decoding group comprises a 3×3 convolution layer, a batch normalization layer and an activation function layer connected in sequence;

[0026] The multi-scale enhanced features output by the fourth receptive field module and the third receptive field module are input into the first decoding block, where one scale of the enhanced features X output The 3×3 convolution layer of the first decoding group is input and then passes through the 3×3 convolution layer, batch normalization layer and activation function layer in sequence. The enhanced features X of the remaining scales output The second decoding group is input for residual connection, and the features output by the first decoding group and the features output by the second decoding group are input into the 1×1 convolution layer to generate a segmentation head. The segmentation head is passed through the upsampling layer to obtain the first segmentation output that matches the size of the remote sensing image; the first segmentation output and the multi-scale enhanced features output by the second receptive field module are input into the second decoding block to obtain the second segmentation output, and the processing process is the same as that of the first decoding block. The second segmentation output and the multi-scale enhanced features output by the first receptive field module are input into the third decoding block to obtain the third segmentation output, and the third segmentation output of the third decoding block is selected as the ground target segmentation result.

[0027] Preferably, the remote sensing images in the training sample set are annotated with a true mask G, and the true mask G is used as a supervisory signal for image segmentation network training in step 3. The true mask G represents the true label value of the category corresponding to each pixel in the remote sensing image.

[0028] Preferably, multiple loss functions are used to optimize the image segmentation network during the training process, wherein the cross entropy loss, Dice loss and boundary smoothing loss are calculated respectively according to the true mask G and all segmentation outputs of the network decoder, and the weighted sum of the cross entropy loss, Dice loss and boundary smoothing loss is used to obtain multiple loss values. The image segmentation network is optimized according to the multiple loss values ​​to obtain a remote sensing image ground object segmentation model.

[0029] Preferably, the cross entropy loss It is expressed as:

[0030]

[0031] Where Ω represents the set of all pixels in the image, G(j) represents the true label value (0 or 1) at pixel j in the true mask G, and S i (j) is the predicted segmentation output S i The probability of pixel j in ;

[0032] Dice loss It is expressed as:

[0033]

[0034] Among them, ∈ is a small constant used to avoid the denominator being zero, the numerator represents the intersection of the predicted mask and the true mask, and the denominator represents the sum of the predicted mask and the true mask;

[0035] Boundary smoothing loss, defined using edge gradient loss, calculates the segmentation output S i And the edge gradient of the true mask G (through the Sobel operator):

[0036]

[0037] Boundary smoothing loss The mean square error between the predicted edge gradient and the true edge gradient is expressed as:

[0038]

[0039] The multiple loss function is expressed as:

[0040]

[0041] Among them, α, β and γ control the weights of cross entropy loss, Dice loss and boundary smoothing loss respectively. Through this multi-loss combination, the model can further optimize the refinement effect of the segmentation boundary while improving the overall segmentation accuracy.

[0042] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a remote sensing image object segmentation method based on hierarchical feature extraction and decoding, extracts multi-scale features through a hierarchical encoder, and uses a U-shaped network structure for feature fusion and decoding, thereby achieving high-precision segmentation of objects in remote sensing images. Through the combination of hierarchical feature extraction and decoding, the accuracy and efficiency of remote sensing image object segmentation are effectively improved. The preprocessing steps and multi-scale feature extraction strategies adopted can fully capture the subtle features and important contextual information in the image. This method not only improves the expressive power of features, but also enhances the model's sensitivity to details and boundaries, thereby achieving better segmentation effects in various remote sensing image processing tasks.

[0043] In addition, multiple loss functions are introduced to further enhance the expressiveness and learning ability of the model, enabling it to better adapt to complex and changeable remote sensing image segmentation tasks. The design of multiple loss functions provides richer supervision signals during the model training process, prompting the model to optimize its output at multiple levels. This innovative combination strategy not only enhances the accuracy of segmentation, but also effectively smoothes the segmentation boundaries, overcoming the shortcomings of traditional methods in dealing with complex edges, thereby achieving more delicate and natural segmentation results. Moreover, through efficient fine-tuning of parameters and the use of adapters, the resource requirements for model training and deployment are reduced. The present invention aims to promote the development of remote sensing image segmentation technology, make it suitable for remote sensing image processing tasks in various computing environments, and provide more accurate and efficient segmentation solutions for applications in related fields. . BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0045] Figure 1 A schematic diagram of the structure of the remote sensing image object segmentation model provided by the present invention;

[0046] Figure 2 A schematic diagram of the structure of a hierarchical encoder provided by the present invention;

[0047] Figure 3 This is a schematic diagram of the receptive field module structure provided by the present invention. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0049] The embodiment of the present invention discloses a remote sensing image object segmentation method based on hierarchical feature extraction and decoding, comprising the following steps:

[0050] Step 1: Collect remote sensing images and preprocess them to construct a training sample set;

[0051] Step 2: Construct an image segmentation network, which includes a hierarchical encoder, a receptive field module unit, and a network decoder connected in sequence; use the hierarchical encoder to extract the features of the preprocessed image, use the receptive field module unit to process the features extracted by the hierarchical encoder, and use the classic network decoder structure to decode the processed features; the receptive field module unit includes 4 receptive field modules;

[0052] Step 3: Use the training sample set to train and optimize the image segmentation network to obtain the remote sensing image object segmentation model;

[0053] Step 4: Collect the remote sensing image to be segmented, input it into the remote sensing image ground object segmentation model, and obtain the ground target segmentation result.

[0054] Furthermore, the preprocessing in step 1 includes operations such as denoising and contrast enhancement, which are intended to improve image quality to improve the accuracy of subsequent feature extraction.

[0055] Furthermore, the hierarchical encoder is Figure 2 As shown in the figure, the pre-trained Hiera architecture can effectively capture multi-scale features and provide rich semantic information for subsequent segmentation tasks; the hierarchical encoder can be represented as a sequence model, including four encoding units connected in sequence, and the output of the previous encoding unit is used as the input of the next encoding unit; each encoding unit includes a normalization layer (Norm), an attention mechanism layer (Attention) and a multi-layer perceptron (MLP) connected in sequence; each encoding unit outputs a hierarchical feature map to form a multi-scale feature; the feature map X i =Encoder(X i-1 ), i=2, ..., N, N represents the number of coding units, N=4, C i represents the number of feature channels; X1 = Encoder (I), I represents the input image, Where H represents the image height and W represents the image width.

[0056] Furthermore, the hierarchical encoder can be expressed as:

[0057] Encoder=Layer N →Layer N-1 →…→Layer1;

[0058] Each encoding layer can be expressed as:

[0059] Layer i =Norm→Self-Attention→Norm→MLP;

[0060] The hierarchical decoder will output feature maps at four different levels where i∈{1, 2, 3, 4}:

[0061] X1=Encoder(I)

[0062] X i =Encoder(X i-1 )(i=2,3,4)

[0063] Number of feature channels C i The value is {144,288,576,1152}.

[0064] Furthermore, the receptive field module is used to reduce the number of feature channels and enhance the feature expression capability, such as Figure 3 As shown in the figure, it is a multi-branch convolution block, whose internal structure can be mainly divided into two parts: a multi-branch convolution layer and a trailing expansion pooling or convolution layer. In each branch, a convolution layer with a specific convolution kernel size is followed by a pooling or convolution layer with a corresponding expansion rate. The former is similar to the Inception network and is responsible for simulating receptive fields of various sizes. Different convolution kernel sizes are applied to achieve multi-size receptive fields. The latter simulates the relationship between the receptive field size and the eccentricity in the human visual system. The purpose is to generate a higher resolution feature map to capture contextual information in a larger range while keeping the number of parameters unchanged. The relationship between the convolution kernel size and the expansion rate is similar to the positive correlation between the receptive field size and the eccentricity in the visual cortex. In the multi-branch convolution layer, the input feature map is processed by a combination of different convolution kernels to obtain response features of different scales. Finally, the feature maps of all branches are connected to form a spatial pooling or convolution array.

[0065] Specifically, it includes a multi-branch convolutional layer, an expansion pooling or convolutional layer, a merging layer, a skip connection layer, and an activation function layer connected in sequence, wherein the multi-branch convolutional layer adopts an Inception network, such as the structure in Inception V4 and Inception-ResNet V2, including a bottleneck structure and a stacked convolutional layer connected in sequence, the bottleneck structure adopts 4 1×1 convolutional layers to reduce the number of channels in the feature map, the stacked convolutional layer includes a 3×3 convolutional layer, a 1× convolutional layer, a 3× convolutional layer and a 1×1 convolutional layer to further process and enhance the feature expression capability, and the three 1×1 convolutional layers of the bottleneck structure are respectively connected to the 1×3 convolutional layer, the 3×1 convolutional layer and the 3×3 convolutional layer of the stacked convolutional layer; the multi-scale feature is input to each 1×1 convolutional layer of the bottleneck structure of the multi-branch convolutional layer and the 1×1 convolutional layer of the stacked convolutional layer; the expansion pooling or convolutional layer includes several 3×3 expansion layers with different expansion rates. Convolutional layer, the expansion rates are 1, 3 and 5 respectively, including two 3×3 expansion convolutional layers with an expansion rate of 3, which are respectively connected to the 1×3 convolutional layer and the 3×1 convolutional layer of the stacked convolutional layer, the 3×3 expansion convolutional layer with an expansion rate of 1 is connected to the 1×1 convolutional layer of the stacked convolutional layer, and the 3×3 expansion convolutional layer with an expansion rate of 5 is connected to the 3×3 convolutional layer of the stacked convolutional layer; the expansion convolutional layer is used to capture a wider range of context information and enhance the feature expression ability; in the bottleneck structure, a 1×1 convolutional layer is sequentially connected to a convolutional layer in the stacked convolutional layer, a 3×3 expansion convolutional layer in the expansion pooling or convolutional layer to form a convolution branch, and output a branch feature map;

[0066] The merging layer merges all branch feature maps to form a spatial pooling or convolution array, expressed as:

[0067] F multi-branch =concat(f1(X i ), f2(X i ), ..., f n (X i ))

[0068] Among them, F multi-branch represents a spatial pooling or convolution array; f1, f2, ..., f n They represent the convolution operations of different convolution branches. Each branch uses a different convolution kernel or expansion rate to simulate different receptive field sizes, which can effectively capture feature information of different scales and improve the expression ability of the model while maintaining the number of parameters. i Represents the input feature map; concat() represents the concatenation operation;

[0069] The skip connection layer inputs the multi-scale features and spatial pooling or convolution array output by the hierarchical encoder into the activation function layer;

[0070] The activation function layer integrates the output feature maps of all branches together through a splicing operation and introduces a shortcut connection to form the final output feature map; different enhanced features X are obtained based on multi-scale features and spatial pooling or convolution arrays output , forming multi-scale enhanced features, enhanced features X output It is expressed as:

[0071] X output =ReLU(F multi-branch +X shortcut )

[0072] Among them, X shortcut ReLU is the activation function. ReLU is used to increase the nonlinear expression ability of the model and enhance the feature X. output It can effectively combine information from different receptive fields and enhance the expression of multi-scale features while maintaining the input feature information.

[0073] Furthermore, the stacked convolution layer of the receptive field module uses two stacked 3×3 convolution layers instead of the original 5×5 convolution layer to reduce the number of parameters and deepen the nonlinear layer. For the same reason, a combination of 1×n and n×1 convolution layers is used to replace the original n×n convolution layer. Finally, the stacked convolution layer includes a 1×1 convolution layer, a 3×3 convolution layer, a 1×3 convolution layer, and a 3×1 convolution layer. The shortcut design in ResNet and Inception-ResNet V2 is applied in this embodiment.

[0074] Furthermore, the network decoder includes three decoding blocks (DecoderBlock) connected in sequence, each decoding block includes two decoding groups (Conv-BN-ReLU), a convolution layer and an upsampling layer, and each decoding group includes a convolution layer, a batch normalization layer and an activation function layer connected in sequence;

[0075] The multi-scale enhanced features output by the fourth receptive field module and the third receptive field module are input into the first decoding block, where one scale of the enhanced features X output The 3×3 convolution layer of the first decoding group is input and then passes through the 3×3 convolution layer, batch normalization layer and activation function layer in sequence. The enhanced features X of the remaining scales outputThe second decoding group is input for residual connection. The features output by the first decoding group and the features output by the second decoding group are input into the 1×1 convolution layer to generate a segmentation head. The segmentation head is passed through the upsampling layer to obtain the first segmentation output that matches the size of the remote sensing image. The first segmentation output and the multi-scale enhanced features output by the second receptive field module are input into the second decoding block to obtain the second segmentation output. The processing process is the same as that of the first decoding block. The second segmentation output and the multi-scale enhanced features output by the first receptive field module are input into the third decoding block to obtain the third segmentation output. The three segmentation outputs are represented as S i , i∈{1, 2, 3}, as the prediction mask, the third segmentation output of the third decoding block is selected as the ground object segmentation result. Through this U-shaped decoder design, cross-level feature fusion can be achieved, thereby improving the accuracy and robustness of the segmentation result.

[0076] Furthermore, the remote sensing images in the training sample set are annotated with the true mask G, and the true mask G is used as a supervisory signal for image segmentation network training in step 3. The true mask G represents the category label value corresponding to each pixel in the remote sensing image.

[0077] Furthermore, multiple loss functions are used in the training process to measure the difference between the model output and the true label to optimize the image segmentation network, wherein the cross entropy loss, Dice loss and boundary smoothing loss are calculated respectively according to the true mask G and all segmentation outputs of the network decoder, and the weighted sum of the cross entropy loss, Dice loss and boundary smoothing loss is used to obtain multiple loss values, and the image segmentation network is optimized according to the multiple loss values ​​to obtain the remote sensing image object segmentation model. The cross entropy loss is used for pixel-level classification, the Dice loss is used to deal with category imbalance, and the boundary smoothing loss is used to reduce the jagged effect in the segmentation results. Deep supervision is applied to all segmentation outputs, so that the model can be supervised on features at different levels, further improving the model's performance in multi-scale feature learning and segmentation accuracy. Through the weighted sum of cross entropy loss, Dice loss and boundary smoothing loss, the trained model can optimize accuracy and boundary smoothness during the training process, thereby improving the overall segmentation performance.

[0078] Furthermore, the cross entropy loss is mainly used for pixel-level classification and is defined as the predicted mask S i The classification error between the real mask G and the predicted mask S i And the true mask G, the cross entropy loss formula is as follows:

[0079]

[0080] Among them, Ω represents the set of all pixels in the image, G(j) is the value of the true label at pixel j in the true mask (0 or 1), and Si (j) is the probability predicted by the model at pixel j;

[0081] Dice loss is used to measure the overlap between the predicted mask and the true mask, which can improve segmentation accuracy in the case of imbalanced categories. The definition of Dice loss is as follows:

[0082]

[0083] Among them, ∈ is a small constant used to avoid the denominator being zero, the numerator represents the intersection of the predicted mask and the true mask, and the denominator represents the sum of the predicted mask and the true mask;

[0084] The boundary smoothing loss is used to constrain the segmentation boundary and make the segmentation result smoother. It can be defined by the boundary gradient difference to reduce the jagged effect of the boundary. The boundary smoothing loss is defined using the edge gradient loss. First, the predicted mask S is calculated. i And the edge gradient of the true mask G (through the Sobel operator):

[0085]

[0086] The boundary smoothing loss is defined as the mean square error between the predicted edge gradient and the true edge gradient. This loss term can effectively reduce the roughness of the predicted edge, making the segmentation boundary more refined and accurate, expressed as:

[0087]

[0088] The multiple loss functions are expressed as follows:

[0089]

[0090] Among them, α, β and γ control the weights of cross entropy loss, Dice loss and boundary smoothing loss respectively. Through this multi-loss combination, the model can further optimize the refinement effect of the segmentation boundary while improving the overall segmentation accuracy.

[0091] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0092] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote sensing image object segmentation method based on hierarchical feature extraction and decoding, characterized in that: The following steps are involved: Step 1: Collect remote sensing images and preprocess them to construct a training sample set; Step 2: Construct an image segmentation network, which includes a hierarchical encoder, a receptive field module unit, and a network decoder connected in sequence; Step 3: Use the training sample set to train and optimize the image segmentation network to obtain the remote sensing image object segmentation model; Step 4: Collect the remote sensing image to be segmented, input it into the remote sensing image ground object segmentation model, and obtain the ground target segmentation result.

2. The remote sensing image object segmentation method based on hierarchical feature extraction and decoding according to claim 1, characterized in that: The preprocessing in step 1 includes denoising and contrast enhancement.

3. The remote sensing image object segmentation method based on hierarchical feature extraction and decoding according to claim 1, characterized in that: The hierarchical encoder includes multiple encoding units connected in sequence, and the output of the previous encoding unit serves as the input of the next encoding unit; each encoding unit includes a normalization layer, an attention mechanism layer, and a multi-layer perceptron connected in sequence; each encoding unit outputs a hierarchical feature map to form a multi-scale feature.

4. The remote sensing image object segmentation method based on hierarchical feature extraction and decoding according to claim 1, characterized in that: The receptive field module unit includes 4 receptive field modules, each of which includes a multi-branch convolution layer, an expansion pooling or convolution layer, a merging layer, a skip connection layer, and an activation function layer connected in sequence, wherein: The multi-branch convolutional layer adopts the Inception network, including a bottleneck structure and stacked convolutional layers connected in sequence. The bottleneck structure includes 4 1×1 convolutional layers, and the stacked convolutional layers include 1×1 convolutional layers, 3×3 convolutional layers, 1×3 convolutional layers, and 3×1 convolutional layers. The 3 1×1 convolutional layers of the bottleneck structure are respectively connected to the 1×3 convolutional layers, 3×1 convolutional layers, and 3×3 convolutional layers of the stacked convolutional layers; multi-scale features are input to each 1×1 convolutional layer of the bottleneck structure of the multi-branch convolutional layer and the 1×1 convolutional layer of the stacked convolutional layer; The dilated pooling or convolution layer includes four 3×3 dilated convolution layers with different dilation rates of 3, each of which is connected to a convolution layer in the stacked convolution layer. In the bottleneck structure, a 1×1 convolution layer is sequentially connected to a convolution layer in the stacked convolution layer and a dilated convolution layer in the dilated pooling or convolution layer to form a convolution branch, and outputs a branch feature map. The merging layer merges all branch feature maps to form a spatial pooling or convolution array; The skip connection layer inputs the multi-scale features and spatial pooling or convolution array output by the hierarchical encoder into the activation function layer; The activation function layer obtains multi-scale enhanced features based on multi-scale features and spatial pooling or convolution array.

5. The remote sensing image object segmentation method based on hierarchical feature extraction and decoding according to claim 4, characterized in that: The network decoder includes three decoding blocks connected in sequence, each decoding block includes two decoding groups, a 1×1 convolution layer and an upsampling layer, and each decoding group includes a 3×3 convolution layer, a batch normalization layer and an activation function layer connected in sequence; The multi-scale enhanced features output by the fourth receptive field module and the third receptive field module are input into the first decoding block, one scale of the enhanced features is input into the 3×3 convolution layer of the first decoding group, and passes through the 3×3 convolution layer, batch normalization layer and activation function layer in sequence, and the enhanced features of the remaining scales are input into the second decoding group for residual connection. The features output by the first decoding group and the features output by the second decoding group are input into the 1×1 convolution layer to generate a segmentation head. The segmentation head passes through the upsampling layer to obtain the first segmentation output matching the size of the remote sensing image; the first segmentation output and the multi-scale enhanced features output by the second receptive field module are input into the second decoding block to obtain the second segmentation output, and the second segmentation output and the multi-scale enhanced features output by the first receptive field module are input into the third decoding block to obtain the third segmentation output, and the third segmentation output of the third decoding block is selected as the ground target segmentation result.

6. The remote sensing image object segmentation method based on hierarchical feature extraction and decoding according to claim 5, characterized in that: The real mask G is annotated for the remote sensing images in the training sample set, and in step 3, the real mask G is used as the supervisory signal for the image segmentation network training.

7. The remote sensing image object segmentation method based on hierarchical feature extraction and decoding according to claim 6, characterized in that: During the training process, multiple loss functions are used to optimize the image segmentation network, wherein the cross entropy loss, Dice loss and boundary smoothing loss are calculated respectively according to the true mask G and all segmentation outputs of the network decoder. The weighted sum of cross entropy loss, Dice loss and boundary smoothing loss is used to obtain multiple loss values. The image segmentation network is optimized according to the multiple loss values ​​to obtain the remote sensing image ground object segmentation model.

Images