A remote sensing image target segmentation method based on multi-level feature aggregation and segmentation

Abstract

This invention discloses a remote sensing image target segmentation method based on multi-level feature aggregation and segmentation, which relates to the field of remote sensing image processing technology. The method comprises the following steps: Step 1: Acquire remote sensing ship images and construct a training dataset; Step 2: Construct a remote sensing image ship segmentation model and train the remote sensing image ship segmentation model using the training dataset; The remote sensing image ship segmentation model comprises a feature extraction module, an aggregation-segmentation module, an attention fusion module, a reparameterized convolution module, and an output module, which are sequentially connected; Step 3: Acquire the remote sensing image to be segmented and input it into the trained remote sensing image ship segmentation model to obtain the ship target segmentation result. This method can improve the detection and segmentation accuracy of ship targets in remote sensing images while ensuring computational efficiency and practicality.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and more particularly to a remote sensing image target segmentation method based on multi-level feature aggregation and segmentation. Background Art

[0002] In the field of remote sensing image processing, ship segmentation technology is key to enabling applications such as ocean monitoring, maritime safety, and military reconnaissance. This technology can identify and locate ship targets from large amounts of remote sensing data, providing important information for decision-making. Traditional ship segmentation methods rely primarily on image processing techniques such as edge detection, region growing, and threshold segmentation. These methods often require extensive manual intervention and have limited effectiveness in detecting ships in complex backgrounds and under varying lighting conditions. For example, edge detection may fail in noisy images, while region growing methods can be affected by the choice of initial seed points, resulting in unstable segmentation results.

[0003] With the development of deep learning technology, particularly the breakthroughs achieved by convolutional neural networks (CNNs) in image recognition and segmentation, deep learning-based methods have gradually become mainstream due to their powerful feature extraction and learning capabilities. These methods automatically learn image features, reducing reliance on handcrafted features and improving segmentation accuracy. However, existing deep learning-based ship segmentation methods still have shortcomings in handling multi-scale ship targets, complex ocean backgrounds, and real-time requirements. These methods typically require extensive computing resources, limiting their widespread deployment in practical applications. For example, some high-performance deep learning models, while highly accurate, suffer from high computational complexity, making them unsuitable for resource-constrained embedded systems or mobile devices.

[0004] Furthermore, the adaptability and generalization capabilities of existing methods for remote sensing images of varying resolutions and complexities need to be improved. The performance of existing models can significantly degrade when faced with varying sea conditions, lighting conditions, and ship types.

[0005] Therefore, how to provide a remote sensing image ship segmentation method that can efficiently process multi-scale targets, adapt to complex backgrounds and has high computational efficiency is a problem that technicians in this field urgently need to solve. Summary of the Invention

[0006] In view of this, the present invention provides a remote sensing image target segmentation method based on multi-level feature aggregation and segmentation, which can improve the detection and segmentation accuracy of ship targets in remote sensing images while ensuring computational efficiency and model practicability.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions:

[0008] A remote sensing image target segmentation method based on multi-level feature aggregation and segmentation includes the following steps:

[0009] Step 1: Collect remote sensing ship images and build a training dataset;

[0010] Step 2: Construct a remote sensing image ship segmentation model and train it using the training dataset. The remote sensing image ship segmentation model includes a feature extraction module, an aggregation-segmentation module, an attention fusion module, a reparameterized convolution module, and an output module.

[0011] Step 3: Collect the remote sensing image to be segmented and input it into the trained remote sensing image ship segmentation model to obtain the ship target segmentation result.

[0012] Preferably, the feature extraction module includes multiple sets of encoders, each set of encoders including sequentially connected convolutional layers, nonlinear activation layers, and pooling layers. Each convolutional layer performs a convolution operation to extract spatial features from remote sensing ship images to obtain a feature map. The number of channels in the feature map increases after each convolution operation to extract more high-level features. After the convolution operation, a nonlinear activation layer is applied to the feature map to perform a nonlinear transformation, and then downsampled through the pooling operation of the pooling layer to reduce the spatial resolution of the feature map. The input image passes through the convolutional layers, nonlinear activation layers, and pooling layers, and through a step-by-step downsampling process, high-level semantic features are gradually extracted. The nonlinear activation layer uses the ReLU activation function; the pooling layer performs a maximum pooling operation.

[0013] Preferably, the convolution operation of the convolutional layer in the encoder is expressed as:

[0014] F i =Conv(X i-1 )

[0015] Among them, X i-1 represents the input feature map of the i-1th layer in the remote sensing ship image, Conv represents the 2D convolution operation, F i Represents the feature map obtained after convolution, and its number of channels is generally more than the number of input channels; the remote sensing ship image is represented as Where H0, W0, and C0 represent the height, width, and number of channels of the image respectively;

[0016] The nonlinear activation layer is used to introduce nonlinear transformation to the feature map, allowing the model to learn more complex features. The expression is:

[0017] F′ i =ReLU(F i )

[0018] Among them, F′ iRepresents the feature map after activation by the ReLU activation function; ReLU represents the ReLU activation function;

[0019] The expression of the pooling operation of the pooling layer is:

[0020] P i =MaxPool(F′ i )

[0021] Among them, P i Represents the feature map after pooling; MaxPool represents the maximum pooling operation.

[0022] Preferably, the aggregation-segmentation module includes a feature alignment layer, a fused convolution layer, and a segmentation layer connected in sequence; the attention fusion module includes a weight fusion layer and a sum fusion layer using an attention mechanism; the feature map extracted by the feature extraction module is downsampled in the feature alignment layer through an average pooling operation and a bilinear interpolation operation to obtain an aligned feature map; the fused convolution layer performs a convolution operation on the aligned feature map to achieve reparameterization and obtain a fused feature; the segmentation layer segments the fused feature in the channel dimension to obtain several local feature maps; the weight fusion layer uses an attention mechanism to perform weighted fusion on the local feature map to obtain an enhanced feature map; the sum fusion layer performs a sum operation on the local feature map and the enhanced feature map to obtain a comprehensive feature map; the reparameterized convolution module performs a convolution operation on the comprehensive feature map to obtain a fused enhanced feature. The aggregation-segmentation module is used to fuse feature maps to obtain high-resolution features that retain minimum target information.

[0023] Preferably, the feature map in the aggregation-segmentation module is downsampled in the feature alignment layer through an average pooling operation and a bilinear interpolation operation to obtain an aligned feature map, which is expressed as:

[0024]

[0025] Among them, AvgPool represents the average pooling operation; Bilinear represents the bilinear interpolation operation; Represents the splicing and fusion operation; P1, P2, P3, and P4 respectively represent the feature maps output by a set of encoders of the feature extraction module;

[0026] The fused convolution layer includes the first convolution layer, the re-parameterized convolution module and the second convolution layer connected in sequence; the re-parameterized convolution module includes a 1*1 convolution layer, a fusion layer and a 3*3 convolution layer; the aligned feature map G align Input to the first convolutional layer for convolution operation to obtain the feature map F' align ; Re-parameterized convolution module for feature map F′ align After performing 1*1 convolution, fusion and 3*3 convolution in sequence, the fusion feature F is obtained. fuse ; The second convolutional layer combines the features Ffuse Perform convolution to obtain convolution fusion features, convolution fusion features and fusion features F fuse The number of channels is the same; the expression of the reparameterized convolution module is:

[0027]

[0028] Among them, F′ align Represents the feature map obtained after the convolution operation of the convolution layer; F fuse Indicates fusion features; Conv3 indicates a convolution operation with a convolution kernel of 3; Conv1 indicates a convolution operation with a convolution kernel of 1;

[0029] The segmentation layer calculates the number of channels for the convolution fusion features and performs segmentation based on the number of channels to obtain the first feature map F α and the second feature map F β .

[0030] Preferably, the process of feature processing by the attention fusion module is:

[0031] The weight fusion layer performs convolution and average pooling operations on the first feature map output by the segmentation layer to obtain the first local feature map, which is expressed as:

[0032] F α+ =AvgPool(Conv1(F α ))

[0033] Among them, F α+ represents the first local feature map; F α Represents the first feature map output by the segmentation layer; AvgPool represents the average pooling operation; Conv1 represents the convolution operation with a convolution kernel of 1;

[0034] The weight fusion layer performs convolution operation, activation function activation and average pooling operation on the first feature map output by the segmentation layer in sequence to obtain the attention weight, which is expressed as:

[0035] A α =AvgPool(σ(Conv1(F α )))

[0036] Among them, A α represents the attention weight; σ represents the Sigmoid activation function;

[0037] The weight fusion layer performs convolution and average pooling operations on the second feature map output by the segmentation layer in sequence to obtain the second local feature map, which is expressed as:

[0038] F β+ =AvgPool(Conv1(F β )) Among them, Fβ Represents the second feature map output by the segmentation layer; F β+ represents the second local feature map;

[0039] The weight fusion layer uses the attention mechanism to integrate the second local feature map F β+ and attention weight A α Perform fusion to obtain enhanced feature maps, the expression is:

[0040]

[0041] Among them, F αβ+ represents the enhanced feature map; Represents a pixel-by-pixel multiplication operation;

[0042] The sum fusion layer adds the first local feature map and the enhanced feature map to obtain a comprehensive feature map, which is expressed as:

[0043]

[0044] in, Represents the comprehensive feature map.

[0045] Preferably, the re-parameterized convolution module is used to integrate the feature map After performing 1*1 convolution, fusion and 3*3 convolution in sequence, the fusion enhancement feature is obtained. The reparameterized convolution module includes a 1*1 convolution layer, a fusion layer, and a 3*3 convolution layer.

[0046] Preferably, the output module includes an upsampling layer and a loss calculation layer; the upsampling layer includes a 1*1 convolution layer, a transposed convolution layer and an activation function layer connected in sequence, and the fusion enhancement feature After convolution by the 1x1 convolutional layer, the input is fed into the transposed convolutional layer. This layer upsamples the fused and enhanced features after convolution to the same resolution as the remote sensing ship image. The activation function layer converts the upsampled feature map into a probability distribution to obtain the ship target segmentation result. The loss calculation layer uses a hybrid loss function to calculate the loss value based on the ship target segmentation result output by the activation function layer and the ship target segmentation result annotated in the remote sensing ship image to optimize model training and improve the accuracy and robustness of the model segmentation. The activation function layer uses the Softmax activation function; the loss calculation layer uses a hybrid loss function consisting of the cross entropy loss function and the Dice loss function.

[0047] Preferably, the hybrid loss function is expressed as:

[0048] L=ηL c +γL δ

[0049] Among them, η and γ represent the hyperparameters used to balance the weights of different loss functions; L c represents the cross entropy loss; L δ represents Dice loss;

[0050] Cross entropy loss L c Expressed as:

[0051]

[0052] Among them, i represents the category index, Y i represents the segmentation result of the real ship target marked in the image, P i Represents the ship target segmentation result predicted and output by the remote sensing image ship segmentation model;

[0053] Dice loss L δ Expressed as:

[0054]

[0055] Where λ represents a smoothing term to prevent the denominator from being zero; P represents the distribution of ship target segmentation results predicted by the remote sensing image ship segmentation model; and Y represents the distribution of the actual ship target segmentation results annotated in the image.

[0056] Preferably, during the training process of the remote sensing image ship segmentation model, an Adam optimizer is used to minimize the hybrid loss function and optimize the model.

[0057] Preferably, step 1 further includes performing data enhancement processing on the collected remote sensing ship images, expanding the images by random rotation, flipping and scaling of the images, and constructing a training data set.

[0058] As can be seen from the above technical solutions, compared to existing technologies, the present invention provides a remote sensing image target segmentation method based on multi-level feature aggregation and segmentation. This method uses an aggregation-segmentation model to perform multi-level feature aggregation and re-segmentation, combined with an attention mechanism, to achieve efficient and accurate segmentation of ship targets in remote sensing images. The aggregation-segmentation model effectively captures and aggregates feature information at different levels, enhancing the model's ability to detect small targets and complex backgrounds. Re-segmentation also preserves high-resolution feature details, improving segmentation accuracy. Furthermore, the use of an attention mechanism reduces unnecessary computational burden, enabling the model to automatically focus on key areas in the image, further enhancing its ability to identify ship targets in complex backgrounds. The model maintains high accuracy while also delivering fast processing speed, making it suitable for real-time or near-real-time remote sensing image analysis. In addition, in terms of enhancing generalization ability, adaptability and reducing resource consumption, the present invention introduces a combination of data augmentation and multiple loss functions, which improves the robustness of the model under different remote sensing image conditions, enabling the model to work stably under changing environments and conditions. By optimizing the model structure and calculation process, it reduces dependence on high-performance computing resources, making the model easier to deploy on various computing platforms, including edge computing devices, and expanding the application scope of the model.

[0059] The present invention can significantly reduce the computational complexity of the model while maintaining high accuracy, making it applicable to remote sensing image processing tasks in various computing environments. It is of great significance for improving the practicality and effectiveness of remote sensing image processing. It can not only improve the efficiency of ocean monitoring, but also realize real-time or near real-time ship detection in resource-constrained environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0061] Figure 1 This is a schematic diagram of the structure of the remote sensing image ship segmentation model provided by the present invention;

[0062] Figure 2 This is a schematic diagram of the aggregation-splitting module structure provided by the present invention. DETAILED DESCRIPTION

[0063] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0064] The embodiment of the present invention discloses a remote sensing image target segmentation method based on multi-level feature aggregation and segmentation, which is characterized by comprising the following steps:

[0065] Step 1: Collect remote sensing ship images and build a training dataset;

[0066] Step 2: Construct a remote sensing image ship segmentation model and train it using the training dataset. The remote sensing image ship segmentation model includes a feature extraction module, an aggregation-segmentation module, an attention fusion module, a reparameterized convolution module, and an output module.

[0067] Step 3: Collect the remote sensing image to be segmented and input it into the trained remote sensing image ship segmentation model to obtain the ship target segmentation result.

[0068] The remote sensing image based multi-level feature aggregation and re-segmentation of the present invention not only improves the segmentation accuracy, but also reduces the resource requirements of model training and deployment through efficient parameter fine-tuning and the use of adapters, making it suitable for remote sensing image processing tasks in various computing environments.

[0069] Furthermore, the feature extraction module includes multiple groups of encoders, each group of encoders includes a convolutional layer, a nonlinear activation layer and a pooling layer connected in sequence. The convolution operation of each convolutional layer extracts the spatial features in the remote sensing ship image to obtain a feature map. After the convolution operation, the nonlinear activation layer is applied to perform a nonlinear transformation on the feature map, and then down-sampled through the pooling operation of the pooling layer to reduce the spatial resolution of the feature map.

[0070] Furthermore, the aggregation-segmentation module includes a feature alignment layer, a fusion convolution layer, and a segmentation layer connected in sequence. The feature map extracted by the feature extraction module is downsampled in the feature alignment layer through average pooling and bilinear interpolation to obtain an aligned feature map. The fusion convolution layer of the aggregation-segmentation module performs a convolution operation on the aligned feature map to obtain a fusion feature. The segmentation layer segments the fusion feature in the channel dimension to obtain several local feature maps.

[0071] Furthermore, the attention fusion module includes a weight fusion layer and a sum fusion layer that adopt an attention mechanism; the weight fusion layer adopts the attention mechanism to perform weight fusion on the local feature map to obtain an enhanced feature map; the sum fusion layer performs a sum operation on the local feature map and the enhanced feature map to obtain a comprehensive feature map.

[0072] Furthermore, the re-parameterized convolution module performs convolution operations on the comprehensive feature map to obtain fused enhanced features.

[0073] Furthermore, the output module includes an upsampling layer and a loss calculation layer; the upsampling layer includes a 1*1 convolution layer, a transposed convolution layer and an activation function layer connected in sequence, and the fusion enhancement feature After the convolution operation of the 1*1 convolutional layer, the input is sent to the transposed convolutional layer. The transposed convolutional layer upsamples the convolved fusion enhancement features to the same resolution as the remote sensing ship image. The activation function layer converts the upsampled feature map into a probability distribution to obtain the ship target segmentation result. The loss calculation layer uses a hybrid loss function to calculate the loss value based on the ship target segmentation result output by the activation function layer and the ship target segmentation result marked in the remote sensing ship image to optimize model training and improve the accuracy and robustness of model segmentation.

[0074] In a specific embodiment, the process of processing an image by the remote sensing image ship segmentation model is as follows:

[0075] S1: Input and feature extraction of remote sensing ship images;

[0076] The input remote sensing ship image is Where H0, W0, and C0 are the height, width, and number of channels of the input image, respectively. The purpose of feature extraction is to extract high-level semantic features of the image through a gradual downsampling process. The remote sensing image ship segmentation model consists of four sets of encoders. In each encoder layer, the image undergoes a series of convolution and pooling operations, gradually reducing the spatial resolution while increasing the number of channels to capture richer contextual information, ultimately obtaining four sets of feature maps. The specific process is as follows:

[0077] S11: In each layer of the encoder, the input image is processed by convolution operation. The purpose of convolution is to extract the spatial features in the feature map. The convolution operation for each layer can be expressed as:

[0078] F i =COnv(X i-1 )

[0079] Among them, X i-1 is the input feature map of the i-1th layer, Conv is the convolution operation (2D convolution), F iIt is the feature map obtained after convolution; the number of channels of the feature map will be increased after each convolution layer. This is to extract more advanced features and help the network capture more complex semantic information. The increment of the number of channels is usually set according to the design requirements, that is, Among them, H i and W i are the height and width of the feature map, C i is the number of channels of the feature map, which is usually more than the number of channels in the previous layer;

[0080] S12: After convolution, a nonlinear activation function is usually applied to the feature map to introduce nonlinear transformation. This is to enable the network to learn more complex features. The nonlinear activation function is expressed as:

[0081] F′ i =ReLU(F i )

[0082] Among them, F′ i It is the feature map after ReLU activation; the ReLU activation function is usually applied to the output of the convolutional layer, and the calculation formula is:

[0083] ReLU(x)=max(0,x);

[0084] S13: Perform pooling operations, especially max pooling, which is usually used for downsampling. In the encoder part, the feature maps of each layer will be pooled to reduce the spatial resolution and further compress the information. The role of pooling is to reduce the amount of computation while maintaining important spatial features. The formula for the pooling operation can be expressed as:

[0085] P i =MaxPool(F′ i )

[0086] Among them, P i It is the feature map after pooling. MaxPool is the maximum pooling operation with a pooling window of 2×2 and a step size of 2. The pooling operation reduces the width and height of the feature map but retains the most significant features of each local area.

[0087] S2: Output from S1 The features are input to the aggregation-segmentation module for fusion to obtain high-resolution features that retain small target information. The structure of the aggregation-segmentation module is as follows: Figure 2 As shown, the specific processing process is:

[0088] S21: First, the input feature map is downsampled by average pooling and bilinear interpolation to ensure the uniformity of the feature map size; the aligned feature map F is obtained by adjusting the feature map to the smallest feature size in the group. align , expressed as:

[0089]

[0090] Among them, AvgPool represents the average pooling operation, Bilinear represents the bilinear interpolation operation, represents the splicing and fusion operation; the average pooling operation reduces the computational complexity by reducing the resolution of the feature map while maintaining important low-level features; bilinear interpolation helps smooth the image and prevent excessive information loss; during the alignment process, the size of the feature map is ensured to be consistent so that subsequent modules can process it effectively. In order to control the computational delay and strike a balance between speed and accuracy, the feature map F4 output by the fourth encoder is selected as the target size for feature alignment, thereby ensuring effective computational control and information retention; the above operations ensure efficient information aggregation while minimizing the computational complexity of subsequent modules;

[0091] S22: The fused convolution layer includes a first convolution layer, a reparameterized convolution module, and a second convolution layer connected in sequence; the reparameterized convolution module includes a 1*1 convolution layer, a fusion layer, and a 3*3 convolution layer;

[0092] Align feature map F align Input to the first convolutional layer for convolution operation to obtain the feature map F' align ;

[0093] Reparameterized convolution module for feature map F′ align After performing 1*1 convolution, fusion and 3*3 convolution in sequence, the fusion feature F is obtained. fuse , the expression is:

[0094]

[0095] The number of intermediate channels is an adjustable parameter to adapt to models of different sizes. The reparameterized convolution module optimizes the representation of feature maps by flexibly adjusting the parameters of the convolution kernel and the number of intermediate channels, and enables the model to adapt to input features of different sizes. In this process, feature extraction and aggregation are performed through convolution operations.

[0096] The second convolutional layer combines the fusion features F fuse Perform convolution to obtain convolution fusion features, convolution fusion features and fusion features F fuse The number of channels is the same;

[0097] S23: The convolution fusion feature generated by the fusion convolution module is input to the segmentation layer to calculate the number of channels. It will be segmented in the channel dimension to obtain F α and F β , so the input of the aggregation-segmentation module is P i , the output of the segmentation layer is Fα and F β ;

[0098] S3: The attention fusion module uses the attention mechanism to fuse the two sets of features output by the segmentation layer;

[0099] S31: The weight fusion layer applies the attention mechanism to further improve the fusion effect of information, such as Figure 1 As shown;

[0100] S311: First feature map F α Use the convolution operation and then apply the average pooling operation to obtain the feature map F α+ :

[0101] F α+ =AvgPool(Conv1(F α ))

[0102] The attention weight A is obtained based on the feature map through the activation function α :

[0103] A α =AvgPool(σ(Conv1(F α )))

[0104] Among them, σ represents the Sigmoid activation function; the Sigmoid activation function can generate a normalized weight value to reflect the importance of the input feature map;

[0105] S312: Feature map F β Perform convolution operation and obtain F by average pooling β+ :

[0106] F β+ =AvgPool(Conv1)(F β ))

[0107] S313: Through the attention mechanism, F β+ With attention weight A α Fusion is performed to obtain the final enhanced feature map F αβ+ :

[0108]

[0109] in, Represents an element-by-element multiplication operation; attention weights give greater weight to more important parts of the feature map. Through this attention mechanism, the model can automatically focus on key areas, effectively enhance small target information, and thus strengthen the information in key areas;

[0110] S32: The sum fusion layer combines the feature map F through the sum operationα+ and enhanced feature map F αβ+ , further improving the ability to express features and ensuring the comprehensiveness and accuracy of information:

[0111]

[0112] S4: The fused and enhanced features are obtained through the re-parameterized convolution module That is, the final output of this step is The re-parameterized convolution module includes 1*1 convolution layer, fusion layer and 3*3 convolution layer, which is used to integrate the feature map. After performing 1*1 convolution, fusion and 3*3 convolution in sequence, the fusion enhancement feature is obtained.

[0113] S5: Fusion of these enhanced features Convert to the final segmentation result;

[0114] S51: Perform upsampling operation; since convolution operation usually reduces the spatial dimension of feature map, it is necessary to Upsample to the same resolution as the input image;

[0115] Fusion Enhancement Features After the convolution operation of the 1*1 convolutional layer, the input is sent to the transposed convolutional layer, which is implemented through transposed convolution (also known as deconvolution). Let the upsampled feature map be U, and then a softmax activation function is used to convert U into a probability distribution, thereby obtaining the final ship target segmentation result P.

[0116] Furthermore, during the training process, a loss function is used to measure the difference between the segmentation results predicted by the model and the true labels; a combination of multiple loss functions is used to train the model to improve the accuracy and robustness of the segmentation; the hybrid loss function used in the training model in this embodiment is expressed as:

[0117] L=ηL c +γL δ

[0118] Among them, η and γ are hyperparameters used to balance the weights of different loss functions; by minimizing these loss functions, the model can be trained to improve its performance on the segmentation task;

[0119] Cross quotient loss L c as follows:

[0120]

[0121] Among them, i represents the category index, Y i and P iRepresent the true label and predicted probability respectively;

[0122] Dice loss is a loss function that measures the similarity between two samples. δ Expressed as:

[0123]

[0124] Among them, λ is a smoothing term to prevent the denominator from being zero; P is the probability distribution predicted by the model, and Y is the one-hot encoding representation of the true input label.

[0125] Furthermore, during the training process, the Adam optimizer is used to minimize the total loss function L. Through iterative optimization, the model learns how to more accurately predict the category label of each pixel, thereby improving the accuracy and robustness of the segmentation.

[0126] Furthermore, in order to further improve the performance of the model, data augmentation techniques such as random rotation, flipping, and scaling are introduced to increase the diversity of training data, prevent model overfitting, and improve the model's adaptability to different image transformations.

[0127] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0128] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote sensing image target segmentation method based on multi-level feature aggregation and segmentation, characterized in that: The following steps are involved: Step 1: Collect remote sensing ship images and build a training dataset; Step 2: Construct a remote sensing image ship segmentation model and train it using the training dataset. The remote sensing image ship segmentation model includes a feature extraction module, an aggregation-segmentation module, an attention fusion module, a reparameterized convolution module, and an output module. The aggregation-segmentation module includes a feature alignment layer, a fusion convolution layer, and a segmentation layer connected in sequence; the attention fusion module includes a weight fusion layer and a sum fusion layer using an attention mechanism; the feature map extracted by the feature extraction module is downsampled in the feature alignment layer through average pooling and bilinear interpolation to obtain an aligned feature map; the fusion convolution layer performs a convolution operation on the aligned feature map to obtain a fused feature; The segmentation layer segments the fusion features in the channel dimension to obtain several local feature maps; the weight fusion layer uses the attention mechanism to perform weight fusion on the local feature maps to obtain enhanced feature maps; the sum fusion layer adds the local feature maps and the enhanced feature maps to obtain a comprehensive feature map; the re-parameterized convolution module performs a convolution operation on the comprehensive feature map to obtain fusion enhanced features; Step 3: Collect the remote sensing image to be segmented and input it into the trained remote sensing image ship segmentation model to obtain the ship target segmentation result.

2. The remote sensing image target segmentation method based on multi-level feature aggregation and segmentation according to claim 1, characterized in that: The feature extraction module includes multiple groups of encoders, each of which includes a convolutional layer, a nonlinear activation layer, and a pooling layer connected in sequence. Each convolutional layer performs a convolution operation to extract spatial features from the remote sensing ship image to obtain a feature map. After the convolution operation, a nonlinear activation layer is applied to perform a nonlinear transformation on the feature map, and then down-sampled through the pooling operation of the pooling layer to reduce the spatial resolution of the feature map.

3. The method for remote sensing image target segmentation based on multi-level feature aggregation and segmentation according to claim 2, characterized in that: The convolution operation of the convolutional layer in the encoder is expressed as: F i =Conv(X i-1 ) Among them, X i-1 represents the input feature map of the i-1th layer in the remote sensing ship image, Conv represents the 2D convolution operation, F i Represents the feature map obtained after convolution; The expression of the nonlinear transformation of the feature map by the nonlinear activation layer is: F′ i =ReLU(F i ) where F′ i Represents the feature map after activation by the ReLU activation function; ReLU represents the ReLU activation function; The expression of the pooling operation of the pooling layer is: P i =MaxPool(F′ i ) Among them, P i Represents the feature map after pooling; MaxPool represents the maximum pooling operation.

4. The method for remote sensing image target segmentation based on multi-level feature aggregation and segmentation according to claim 3, characterized in that: The feature map in the aggregation-segmentation module is downsampled in the feature alignment layer through average pooling and bilinear interpolation to obtain the aligned feature map, which is expressed as: Among them, AvgPool represents the average pooling operation; Bilinear represents the bilinear interpolation operation; Represents the splicing and fusion operation; P1, P2, P3, and P4 respectively represent the feature maps output by a set of encoders of the feature extraction module; The fused convolution layer includes the first convolution layer, the re-parameterized convolution module and the second convolution layer connected in sequence; the re-parameterized convolution module includes a 1*1 convolution layer, a fusion layer and a 3*3 convolution layer; the aligned feature map F align Input to the first convolutional layer for convolution operation to obtain the feature map F' align ; Re-parameterized convolution module for feature map F′ align After performing 1*1 convolution, fusion and 3*3 convolution in sequence, the fusion feature F is obtained. fuse ; The second convolutional layer combines the features F fuse Perform convolution to obtain convolution fusion features, convolution fusion features and fusion features F fuse The number of channels is the same; the expression of the reparameterized convolution module is: Among them, F′ align Represents the feature map obtained after the convolution operation of the convolution layer; F fuse Indicates fusion features; Conv3 indicates a convolution operation with a convolution kernel of 3; Conv1 indicates a convolution operation with a convolution kernel of 1; The segmentation layer calculates the number of channels for the convolution fusion features and performs segmentation based on the number of channels to obtain the first feature map F α and the second feature map F β .

5. The method for remote sensing image target segmentation based on multi-level feature aggregation and segmentation according to claim 4, characterized in that: The process of feature processing by the attention fusion module and the reparameterized convolution module is as follows: The weight fusion layer performs convolution and average pooling operations on the first feature map output by the segmentation layer to obtain the first local feature map, which is expressed as: F α+ =AvgPool(Conv1(F α )) Among them, F α+ represents the first local feature map; F α Represents the first feature map output by the segmentation layer; AvgPool represents the average pooling operation; Conv1 represents the convolution operation with a convolution kernel of 1; The weight fusion layer performs convolution operation, activation function activation and average pooling operation on the first feature map output by the segmentation layer in sequence to obtain the attention weight, which is expressed as: A α =AvgPool(σ(Conv1(F α ))) Among them, A α represents the attention weight; σ represents the Sigmoid activation function; The weight fusion layer performs convolution and average pooling operations on the second feature map output by the segmentation layer in sequence to obtain the second local feature map, which is expressed as: F β+ =AvgPool(Conv1(F β )) Among them, F β Represents the second feature map output by the segmentation layer; F β+ represents the second local feature map; The weight fusion layer uses the attention mechanism to integrate the second local feature map F β+ and attention weight A α Perform fusion to obtain enhanced feature maps, the expression is: Among them, F αβ+ represents the enhanced feature map; Represents a pixel-by-pixel multiplication operation; The sum fusion layer adds the first local feature map and the enhanced feature map to obtain a comprehensive feature map, which is expressed as: in, represents the comprehensive feature map; Re-parameterized convolution module for comprehensive feature maps After performing 1*1 convolution, fusion and 3*3 convolution in sequence, the fusion enhancement feature is obtained.

6. The method for remote sensing image target segmentation based on multi-level feature aggregation and segmentation according to claim 1, characterized in that: The output module includes an upsampling layer and a loss calculation layer. The upsampling layer includes a 1*1 convolutional layer, a transposed convolutional layer, and an activation function layer connected in sequence. The fused enhanced features are convolved by the 1*1 convolutional layer and then input into the transposed convolutional layer. The transposed convolutional layer upsamples the convolved fused enhanced features to the same resolution as the remote sensing ship image. The activation function layer converts the upsampled feature map into a probability distribution to obtain the ship target segmentation result. The loss calculation layer uses a hybrid loss function to calculate the loss value based on the ship target segmentation results output by the activation function layer and the ship target segmentation results marked in the remote sensing ship image to optimize model training.

7. The method for remote sensing image target segmentation based on multi-level feature aggregation and segmentation according to claim 6, characterized in that: The hybrid loss function used in the loss calculation layer includes the cross entropy loss function and the Dice loss function; the hybrid loss function is expressed as: L=ηL c +γL δ Among them, η and γ represent the hyperparameters used to balance the weights of different loss functions; L c represents the cross entropy loss; L δ represents Dice loss; Cross entropy loss L c Expressed as: Among them, i represents the category index, Y i represents the segmentation result of the real ship target marked in the image, P i Represents the ship target segmentation result predicted and output by the remote sensing image ship segmentation model; Dice loss L δ Expressed as: Where λ represents a smoothing term to prevent the denominator from being zero; P represents the distribution of ship target segmentation results predicted by the remote sensing image ship segmentation model; and Y represents the distribution of actual ship target segmentation results annotated in the image.

8. The method for remote sensing image target segmentation based on multi-level feature aggregation and segmentation according to claim 7, characterized in that: During the training process of the remote sensing image ship segmentation model, the Adam optimizer is used to minimize the hybrid loss function and optimize the model.

9. The method for remote sensing image target segmentation based on multi-level feature aggregation and segmentation according to claim 1, characterized in that: Step 1 also includes data augmentation processing on the collected remote sensing ship images, expanding the images through random rotation, flipping and scaling, and constructing a training dataset.

Images