Remote sensing image target segmentation method based on feature adaptive processing

Abstract

The invention discloses a remote sensing image target segmentation method based on feature adaptive processing, which relates to the technical field of remote sensing image processing, and comprises the following steps: 1, collecting a remote sensing image, and carrying out preprocessing and data enhancement to obtain a training data set; 2, constructing a remote sensing image segmentation network, and training by using the training data set to obtain a remote sensing image segmentation model; the remote sensing image segmentation network comprises an encoder, a feature adaptive processing module, a multi-scale feature fusion module and a decoder which are connected in sequence; and 3, collecting a remote sensing image to be segmented, and inputting the remote sensing image to the remote sensing image segmentation model to obtain a target segmentation result. According to the invention, the feature adaptive processing module and the multi-scale feature fusion technology are introduced, so that the segmentation precision of the target in the remote sensing image can be effectively improved.

Description

Technical Field

[0001] The invention relates to the technical field of remote sensing image processing, and more particularly to a remote sensing image target segmentation method based on feature adaptive processing. Background Art

[0002] With the rapid development of remote sensing technology, remote sensing images have been widely used in many fields such as military, geology, agriculture, and environmental monitoring. The core problem of remote sensing image target segmentation tasks is how to effectively extract targets from complex backgrounds and accurately segment them. Traditional segmentation methods often rely on manually designed feature extraction algorithms. Although these methods are effective in certain specific scenarios, they perform poorly in complex environments. Specifically, traditional methods often misidentify or miss when dealing with problems such as illumination changes, terrain undulations, and the similarity between targets and backgrounds. With the rise of deep learning, especially convolutional neural networks (CNNs), data-driven automatic feature learning has gradually become the mainstream method in the field of remote sensing image segmentation. Deep learning methods can automatically extract features through training data, overcoming the limitations of manual feature design in traditional methods. In particular, segmentation models based on network architectures such as fully convolutional neural networks (FCNs) and U-Net have made significant progress. However, despite the excellent performance of these methods in many tasks, there are still some difficult-to-overcome challenges in target segmentation of remote sensing images.

[0003] First, the distinction between the target and the background in remote sensing images is often weak, especially in the monitoring and recognition of aircraft targets. When the texture or color of the aircraft target is similar to that of the background objects such as the ground and clouds, the existing deep learning model is easily disturbed by the background, resulting in a decrease in segmentation accuracy. Secondly, remote sensing images usually have a high resolution, which requires the segmentation model to have a strong multi-scale feature extraction capability to cope with aircraft targets of different sizes, shapes and directions. At present, a variety of improvement methods have been proposed to address these problems. Some methods improve the recognition accuracy of the target area by enhancing the contextual information of the model or introducing an attention mechanism. However, these methods usually focus on the fusion of information at a single scale and ignore the multi-scale feature expression in remote sensing images. Other methods try to use multi-channel or multi-view image information to enhance the recognizability of the target, but often pay a high computational cost. Therefore, how to efficiently fuse multi-scale features, enhance the model's attention to key areas, and reduce the consumption of computing resources is still an urgent problem to be solved in remote sensing image target segmentation.

[0004] In order to solve these problems, feature adaptive processing technology has become a research hotspot in the field of remote sensing image segmentation in recent years. Feature adaptive processing methods can effectively highlight key areas and suppress irrelevant background by dynamically adjusting the weights of features, thereby improving the accuracy and robustness of segmentation. Unlike traditional static feature extraction methods, feature adaptive processing can adaptively select appropriate features for processing according to changes in image content. This method is particularly effective when the difference between the target area and the background in the image is small. However, most of the existing feature adaptive processing methods focus on simple single-scale feature fusion. For complex remote sensing images, especially when there are large scale differences or multiple interference factors, the effect of existing technologies is still limited.

[0005] Therefore, how to further improve the feature adaptive processing capability and improve the segmentation accuracy of targets in remote sensing images through adaptive fusion of multi-scale features is an urgent problem that technical personnel in this field need to solve. Summary of the invention

[0006] In view of this, the present invention provides a remote sensing image target segmentation method based on feature adaptive processing. By introducing feature adaptive processing module and multi-scale feature fusion technology, the segmentation accuracy of targets in remote sensing images can be effectively improved, especially in complex backgrounds and diverse environments. Through the application of this technology, targets can be segmented more accurately from remote sensing images, providing strong technical support for remote sensing monitoring, military reconnaissance and other fields.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A remote sensing image target segmentation method based on feature adaptive processing comprises the following steps:

[0009] Step 1: Collect remote sensing images and perform preprocessing and data enhancement to obtain training data sets;

[0010] Step 2: Construct a remote sensing image segmentation network and use the training data set for training to obtain a remote sensing image segmentation model; the remote sensing image segmentation network includes an encoder, a feature adaptive processing module, a multi-scale feature fusion module and a decoder connected in sequence;

[0011] Step 3: Collect the remote sensing image to be segmented, input it into the remote sensing image segmentation model, and obtain the target segmentation result.

[0012] Preferably, the preprocessing in step 1 includes cropping, scaling and labeling, and the data enhancement includes rotation, inversion and color adjustment. The image is cropped and scaled to adapt to the model input size; the image is labeled to determine the position and shape of the ship target, generate an accurate segmentation mask, and provide a learning target for the model; data enhancement operations include rotation, flipping, color adjustment, etc., to simulate various situations that may occur in the training set and enhance the model's ability to recognize ship targets under different conditions; preprocessing and data enhancement before training the network can improve the training efficiency and generalization ability of the model.

[0013] Preferably, the process of processing the training image by the remote sensing image segmentation network includes:

[0014] The encoder includes multiple convolution modules connected in sequence. The images in the training data set are input into the convolution modules, and each convolution module extracts a high-dimensional feature map of a scale;

[0015] The feature adaptive processing module performs pooling operations, fusion operations, and multi-head attention mechanism enhancement operations on the high-dimensional feature map in sequence to obtain an adaptive feature map;

[0016] The multi-scale feature fusion module performs feature extraction and adaptive fusion on the adaptive feature map to obtain a fused feature map;

[0017] The decoder performs weighted fusion, convolution adjustment and upsampling on the fused feature map in sequence to obtain the target segmentation result.

[0018] The technical effect of the above technical solution is that the high-dimensional feature maps of multiple scales output by the encoder represent semantic information at different levels, including edges, regional contexts and global structures; the feature adaptive processing module can enhance the semantic information of the features; the multi-scale feature fusion module further improves the feature representation capability and helps the network capture important areas and detail information; the decoder generates high-quality segmentation result images.

[0019] Preferably, the network weights are updated by back propagation algorithm and gradient descent method, the loss function is minimized, the early stopping technique is used to determine the best training round, and the training is stopped after iterating to the best training round to obtain the remote sensing image segmentation model; the remote sensing image segmentation model is fine-tuned to optimize the model hyperparameters, including learning rate, batch size, etc. The segmentation accuracy of the model is improved by updating the network weights and minimizing the loss function, and the model performance is further improved by fine-tuning the model.

[0020] Preferably, during the training process, the model needs to be verified regularly, and the model performance is evaluated using the validation set to prevent overfitting. After the remote sensing image segmentation model is obtained, the model is finally evaluated using the test set to verify the generalization ability of the model on unseen data, and necessary adjustments are made based on the test results.

[0021] Preferably, the process of preprocessing and data enhancement of the remote sensing image in step 1 includes:

[0022] Step 11: Annotate the remote sensing images, normalize the annotated remote sensing images, and normalize the pixels to [0,1] to eliminate the brightness and contrast differences between different images;

[0023] Step 12: Crop the standardized remote sensing image according to the preset target size (H, W) to adapt to the input requirements of the model, using random cropping or center cropping methods, expressed as:

[0024]

[0025] in, Represents a remote sensing image after standardization; Represents the cropped image; Crop represents the cropping operation;

[0026] Step 13: Randomly rotate or flip horizontally or vertically the cropped image to obtain the transformed image, which is expressed as:

[0027]

[0028] Among them, θ represents the rotation angle, θ∈[-θ max ,θ max ];θ max Indicates the preset maximum rotation angle; Rotate indicates the rotation transformation operation; Flip indicates the flip transformation operation; axis indicates the flip axis direction; Represents the transformed image;

[0029] Step 14: Perform color adjustment on the transformed image; perform random perturbations on brightness, contrast, and saturation to obtain the adjusted image

[0030] Step 15: Generate segmentation label ψ based on the annotation information of the remote sensing image and adjust the image Paired with the segmentation label ψ to form a training data set

[0031] Preferably, the encoder includes three groups of convolution modules, each of which includes a convolution layer, an activation function layer and a pooling layer connected in sequence, and each convolution module processes the image as follows:

[0032] X i =MaxPool(ReLU(Conv(X i-1 ))) for i = 1, 2, 3

[0033] Among them, X i-1 represents the image input to the convolution module, X i Represents the feature map output by the convolution module; i represents the i-th convolution module, and the first convolution module inputs the adjusted image in the training dataset Conv represents the convolution operation; ReLU represents the activation function; MaxPool represents the maximum pooling operation; the first convolution module outputs a high-dimensional feature map Xx with a size of Contains underlying edge and texture information; the second convolution module outputs a high-dimensional feature map X2 with a size of Contains higher-level regional context information; the third convolution module outputs a high-dimensional feature map X3 with a size of Contains global structure and complex pattern information.

[0034] Preferably, the feature adaptive processing module includes a first average pooling layer, a second average pooling layer, an initial fusion layer, a multi-head attention layer, a first 1×1 convolution layer, an activation function layer, a second 1×1 convolution layer and an attention fusion layer. The specific processing process is:

[0035] Step 211: The high-dimensional feature map X1 is passed through the first average pooling layer to obtain a pooled feature map X′1, and the high-dimensional feature map X2 is passed through the second average pooling layer to obtain a pooled feature map X′2;

[0036] Step 212: The pooled feature map X′1, the pooled feature map X′2 and the high-dimensional feature map X3 are concatenated or weighted fused at the initial fusion layer to obtain the initial fusion feature map X fusion ;

[0037] Step 213: Initial fusion feature X fusion In the multi-head attention layer, it is mapped to multiple subspaces, and multiple detection heads are used for parallel calculation to obtain multiple attention maps, and all attention maps are combined with the initial fusion feature X fusion Multiply to get the attention weighted feature map X attn ;

[0038] Step 214: The attention weighted feature map is convolved in the first 1×1 convolutional layer to obtain a compressed feature map;

[0039] Step 215: The compressed feature map is activated in the activation function layer to obtain an activated feature map;

[0040] Step 216: The activated feature map is convolved in the second 1×1 convolutional layer to obtain a weight map W;

[0041] Step 217: The attention fusion layer adds the attention weighted feature map X attn Multiply it with the weight map W to obtain the adaptive feature map Xenhanced .

[0042] Preferably, the multi-scale feature fusion module includes an upper branch, a middle branch and a lower branch; the upper branch includes a third 1×1 convolution layer and a third average pooling layer connected in sequence; the middle branch includes a fourth 1×1 convolution layer, an activation function layer, a fourth average pooling layer, a scale fusion layer and a scale splicing layer connected in sequence; the lower branch includes a fifth average pooling layer, a branch splicing layer, a fifth 1×1 convolution layer and a sixth 1×1 convolution layer; the processing process of the multi-scale feature fusion module is:

[0043] Step 221: Adaptive feature map X enhanced After the convolution operation of the third 1×1 convolutional layer, an average pooling operation is performed in the third average pooling layer to obtain the global feature x α , expressed as:

[0044] x α =AvgPool(1×1Conv(X enhanced ))

[0045] Among them, AvgPool represents average pooling; 1×1Conv represents 1*1 convolution operation;

[0046] Adaptive Feature Map X enhanced After the convolution operation of the fourth 1×1 convolutional layer, the weight map A is obtained by sequentially passing through the activation operation of the activation function layer and the average pooling operation of the fourth average pooling layer. β , expressed as:

[0047] A β =AvgPool(ReLU(1×1Conv(X enhanced )))

[0048] Among them, RELU means applying the RELU activation function;

[0049] Adaptive Feature Map X enhanced After the average pooling operation of the fifth average pooling layer, the feature map is obtained Feature Map and adaptive feature map X enhanced The concatenated feature map is obtained by adding the branch concatenation layers, and the enhanced feature map is sequentially convolved through the fifth 1×1 convolution layer and the sixth 1×1 convolution layer to obtain the enhanced feature map x γ , expressed as:

[0050]

[0051] Step 222: Enhance feature map x γ Input to the scale fusion layer and weight map A βMultiply to obtain the enhanced weighted feature map x β , expressed as:

[0052] x β =A β ·x γ ;

[0053] Step 223: Enhance the weighted feature map x β and the global feature map x α Input to the scale concatenation layer for addition to obtain the fusion feature map X″ enhanced .

[0054] Preferably, the decoder comprises a convolution layer, an upsampling layer and an activation function layer connected in sequence; the fusion feature map X″ enhanced The target segmentation result is obtained by sequentially undergoing the convolution operation of the convolution layer, the bilinear interpolation upsampling operation of the upsampling layer, and the activation operation of the activation function layer; the convolution layer adopts a 1×1 convolution layer.

[0055] Preferably, the remote sensing image segmentation network uses a mixed loss function L to optimize the model during the training process, and the expression is:

[0056] L=εL1+δL2+μL3

[0057]

[0058] Among them, L1 represents the cross entropy loss; L2 represents the Dice loss; L3 represents the IoU value; N represents the total number of pixels of the adjusted image in the training dataset; P i represents the predicted probability at pixel i in the target segmentation result predicted by the remote sensing image segmentation network; ψ i represents the segmentation label of pixel i in the training dataset; ε, δ, and μ are the scaling coefficients respectively.

[0059] Preferably, the remote sensing image segmentation network minimizes the mixed loss function through back propagation and gradient descent during the training process; the back propagation algorithm calculates the gradient of the model parameters according to the mixed loss function, and then updates the network weights through gradient descent. The network weight update expression is:

[0060]

[0061] Among them, η represents the learning rate; represents the gradient of the hybrid loss function with respect to the parameter θ.

[0062] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a remote sensing image target segmentation method based on feature adaptive processing. By introducing a feature adaptive processing module, it can effectively enhance the semantic information of the image and adaptively fuse multi-scale features, thereby more accurately identifying and segmenting the target. This method combines the advantages of deep learning models, and through data preprocessing and enhancement, feature adaptive processing, multi-scale feature fusion and other steps, it improves the model's ability to recognize targets under complex backgrounds and different environmental conditions, and is suitable for various remote sensing image processing tasks. Specifically, the present invention has the following beneficial effects:

[0063] 1) Through the feature adaptive processing module, multi-scale features can be adaptively integrated to improve the segmentation accuracy of targets in remote sensing images, especially the target recognition capability under complex backgrounds and different environmental conditions;

[0064] 2) The introduction of multi-head attention mechanism and weighted fusion strategy enhances the model's focus on key areas, resulting in better performance in detail capture and segmentation accuracy. By efficiently fine-tuning parameters, the resource requirements during training and deployment are reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0066] Figure 1 This is a schematic diagram of the remote sensing image segmentation model processing process provided by the present invention. DETAILED DESCRIPTION

[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] In terms of monitoring and identifying aircraft targets, aircraft targets in remote sensing images are usually highly complex and diverse, and are affected by multiple factors such as weather, lighting, terrain, and the appearance characteristics of the target itself. Therefore, it is extremely challenging to accurately segment aircraft targets in remote sensing images. The embodiment of the present invention discloses a remote sensing image target segmentation method based on feature adaptive processing, which segments aircraft targets in remote sensing images, including the following steps:

[0069] S1: Collect remote sensing images and perform preprocessing and data enhancement to obtain training data sets;

[0070] S2: construct a remote sensing image segmentation network and use the training data set for training to obtain a remote sensing image segmentation model; the remote sensing image segmentation network includes an encoder, a feature adaptive processing module, a multi-scale feature fusion module and a decoder connected in sequence;

[0071] S3: Collect the remote sensing image to be segmented, input it into the remote sensing image segmentation model, and obtain the aircraft target segmentation result.

[0072] Furthermore, the preprocessing in S1 includes cropping, scaling and annotation, and data enhancement includes rotation, inversion and color adjustment. The image is cropped and scaled to adapt to the model input size; the image is annotated to determine the position and shape of the ship target, generate an accurate segmentation mask, and provide a learning target for the model; data enhancement operations include rotation, flipping, color adjustment, etc., to simulate various situations that may occur in the training set and enhance the model's ability to recognize ship targets under different conditions; preprocessing and data enhancement before training the network can improve the training efficiency and generalization ability of the model.

[0073] Furthermore, a remote sensing image segmentation network is constructed and trained to obtain a remote sensing image segmentation model; first, the processed image in S1 is used as the input image, and high-dimensional feature maps are extracted through multiple convolution modules of the encoder. These feature maps represent semantic information at different levels, including edges, regional contexts, and global structures; next, the feature adaptive processing module is used to enhance the semantic information of the features through pooling, splicing, weighted fusion, and multi-head attention mechanisms, and obtain weighted feature maps; then, the feature maps are subjected to multi-scale feature extraction and adaptive fusion through a multi-scale feature fusion module with three branches (upper branch, middle branch, and lower branch), further improving the feature representation capability and helping the network capture important areas and detail information; finally, the decoder performs weighted fusion, convolution adjustment, and upsampling operations to generate high-quality segmentation result images. This process improves segmentation accuracy and image detail performance through multi-level feature fusion and spatial restoration.

[0074] Furthermore, in the training and optimization process of the remote sensing image segmentation model, the network is trained using preprocessed and enhanced remote sensing image data, and the model weights are updated through the back-propagation algorithm and gradient descent method to minimize the loss function to improve the segmentation accuracy of the model; during the training process, the model needs to be verified regularly, and the model performance is evaluated using the validation set to prevent overfitting; the early stopping technique is used to determine the optimal training rounds; after the training is completed, the model is fine-tuned and hyperparameters such as learning rate and batch size are optimized to further improve the model performance; finally, the model is finally evaluated using the test set to verify the generalization ability of the model on unseen data, and necessary adjustments are made based on the test results.

[0075] The remote sensing image target segmentation method based on the feature adaptive processing module of the remote sensing image of the present invention significantly improves the segmentation accuracy of aircraft targets in remote sensing images by enhancing the feature representation ability of the image and multi-scale information fusion, and reduces the resource requirements for model training and deployment through efficient parameter fine-tuning and the use of adapters. It can be effectively deployed under limited computing resources, making it suitable for remote sensing image processing tasks in various computing environments.

[0076] In a specific embodiment, before the deep learning model is trained, the input original remote sensing image is Perform data preprocessing and enhancement operations to improve the training efficiency and generalization ability of the model; the specific process is:

[0077] S1, perform image annotation, standardization and cropping; annotate the image content; input image Normalize the pixel values ​​to [0, 1] to eliminate the brightness and contrast differences between different images; crop the image to the target size (H, W) to adapt to the input requirements of the model, using random cropping or center cropping, denoted as:

[0078]

[0079] in, is the cropped image;

[0080] S2, diversified data enhancement; through data enhancement, we simulate the complex situations that may occur in the training set to enhance the robustness of the model. First, we use the random rotation θ∈[-θ max ,θ max ], or horizontal and vertical flipping, defined as:

[0081]

[0082] Where θ represents the rotation angle (in degrees), θ maxis the maximum rotation angle set, Rotate is an operation function used to rotate the input image, Flip is an operation function used to flip the input image, and axis indicates the axis direction of the flip;

[0083] Then the image Perform color adjustment and randomly perturb the brightness, contrast, and saturation of the image to obtain the image

[0084] S3, based on the annotation information of the original remote sensing image, generates the corresponding segmentation label (mask) ψ, where the target area (aircraft) is marked as the foreground (value 1), and the rest of the area is marked as the background (value 0); the processed image Paired with the corresponding mask ψ to form a training data set

[0085] In a specific embodiment, the remote sensing image segmentation model processes the image as follows: Figure 1 As shown, specifically:

[0086] S1: processed image As the input image of the network, the input image The size is C×H×W, where C is the number of channels, H and W are the height and width of the image respectively;

[0087] The image passes through multiple convolution modules of the encoder in sequence. Each convolution module is regarded as a stage. Each convolution module consists of several convolution layers, activation functions (such as ReLU) and pooling layers (such as maximum pooling). By extracting high-dimensional features of different semantic levels layer by layer, three high-dimensional features X1, X2 and X3 are obtained, where i = 1, 2, 3, and each feature map represents semantic information at different levels:

[0088] X i =MaxPool(ReLU(Conv(X i-1 ))) for i = 1, 2, 3

[0089] Among them, X1 represents the features after the first stage, and the size is Contains the underlying edge and texture information; X2 represents the features after the second stage, with a size of Contains higher-level regional context information; X3 represents the features after the third stage, with a size of Contains global structure and complex pattern information; these feature maps provide rich semantic information for subsequent decoding and segmentation tasks;

[0090] S2: Apply average pooling operation to feature maps X1 and X2 respectively to obtain pooled feature maps X′1 and X′2;

[0091] The pooled features X′1 and X′2 are concatenated or weighted fused with the feature map X3 to obtain the fused feature X fusion , the size of the feature map is This feature will serve as the input of the subsequent multi-head attention mechanism;

[0092] Adopt multi-head attention mechanism and fuse feature X fusion It is mapped to multiple subspaces and computed in parallel by multiple heads to capture different attention information;

[0093] By multiplying the attention map with the fusion feature, we get the feature map X weighted by the attention mechanism. attn ;

[0094] After the attention is applied to the feature map X through a 1×1 convolutional layer attn The number of channels is compressed or adjusted, and then nonlinear features are introduced through the activation function ReLU. Then, the features are adjusted through a 1×1 convolution layer to obtain the final weight map W.

[0095] The obtained weight map W is combined with the feature map X that has been weighted by the attention mechanism. attn Multiply by to obtain the enhanced adaptive feature X enhanced , whose size is Where C′ is the number of channels output by the convolutional layer; the feature map contains information after multi-head attention and adaptive weight adjustment, which can better represent the important features in the input image;

[0096] S3: Adaptive Feature X enhanced The quality of feature representation will be improved through multi-scale feature extraction and adaptive fusion through the upper, middle and lower branches, so that the network can better capture semantic information at different levels when processing image segmentation tasks, and finally obtain the enhanced feature map X″ enhanced ; The specific process is as follows:

[0097] S31: The upper branch first performs adaptive feature X enhanced A 1×1 convolution operation is performed to extract channel information and reduce the amount of calculation. Then, an average pooling operation is performed. The purpose is to compress the spatial dimension through the pooling layer, thereby retaining the information of a larger semantic area without losing important global features. Through this processing, an overall, high-level global feature x can be obtained. α , expressed as:

[0098] x α =AvgPool(1×1Conv(X enhanced))AvgPool represents average pooling, and 1×1Conv represents 1*1 convolutional layer; the purpose of this branch is to extract large-scale contextual information in the image through pooling, retaining global features without paying too much attention to local details;

[0099] The middle branch firstly performs adaptive feature X enhanced A 1×1 convolution is performed, followed by a nonlinear transformation through the ReLU activation function to increase the expressiveness of the features. Subsequently, an average pooling operation is applied to further reduce the spatial resolution of the feature map, thereby highlighting the salient areas or object structures in the image. Finally, the output is the weight map A β , these weights will be used in the subsequent weighted fusion stage and are expressed as:

[0100] A β =AvgPool(ReLU(1×1Conv(X enhanced )))

[0101] Among them, RELU means applying the RELU activation function; the design purpose of the middle branch is to learn the importance of different areas in the feature map, and enhance or suppress the features of certain areas through the weight map to help the network focus more on the key areas;

[0102] The lower branch first performs adaptive feature X enhanced Perform an average pooling operation to obtain a lower resolution feature map This provides a basis for subsequent feature weighting, and then the pooled feature map Compared with the original adaptive feature X enhanced Add, combine the low-resolution information and the original features to enhance the detail information of the image, then, through two 1×1 convolution operations, further adjust and refine the feature map, and finally obtain the enhanced feature map x γ :

[0103]

[0104] The lower branch can improve the network's ability to capture details by combining low-resolution features with high-resolution features. Especially in segmentation tasks, capturing fine boundaries and local information is crucial.

[0105] S32: Weight map A β And the adjusted feature map x γ Multiply them together to get the weighted feature map x β ; The key to this step is to adjust the contribution of different parts in the feature map by introducing weights, so that the network can pay more attention to important areas, expressed as:

[0106] x β =Aβ ·x γ The weight map here plays a role in weighting important feature areas, enhancing the model's response to key areas and helping to improve segmentation accuracy;

[0107] S33: The feature map x of the upper branch α The weighted feature map x of the middle branch β Add together to get the final fusion feature map X″ enhanced ; In this way, the network combines feature information at different levels to comprehensively improve the expressive power of features, expressed as:

[0108] X″ enhanced =x α +x β ;

[0109] The upper branch provides global semantic information, and the middle branch provides weighted information of important areas. The combination of the two can better describe the overall and local features of the image, thereby improving the segmentation performance.

[0110] S4: Enhanced fusion feature map X″ enhanced After further processing through a 1×1 convolutional layer, the purpose is to reduce the number of channels and optimize the feature expression, and then upsampled through bilinear interpolation to restore the spatial resolution and make the image details clearer. Finally, the upsampled feature map passes through an activation function to obtain the final output aircraft target segmentation result P; the upsampling operation restores the spatial resolution of the image, allowing the network to generate high-quality segmentation results, and the 1×1 convolutional layer plays a role in further optimizing and integrating features.

[0111] In a specific embodiment, in the training and optimization of the remote sensing image segmentation model, a loss function is needed to measure the difference between the predicted result and the true label (mask ψ). The output of the network is the aircraft target segmentation result P, which represents the segmentation result image, and the total loss function is expressed as a mixed loss function L:

[0112] L=εL1+δL2+μL3

[0113]

[0114] Among them, ε, δ, μ are proportional coefficients; L1 represents cross entropy loss, which measures the difference between the predicted probability distribution and the true distribution; L2 represents Dice loss, which is used to measure the similarity between two sets and is suitable for binary classification tasks, especially in the case of class imbalance; L3 represents IoU value, which is another common indicator for evaluating segmentation accuracy, calculating the intersection over union ratio of the predicted area and the true area; N represents the total number of pixels of the adjusted image in the training data set; P irepresents the predicted probability at pixel i in the aircraft target segmentation result predicted by the remote sensing image segmentation network; ψ i represents the segmentation label of pixel i in the training dataset, ψ i is the label (0 or 1) of pixel i in the ground-truth mask.

[0115] Furthermore, during the training process, the loss function is minimized through back propagation and gradient descent. The back propagation algorithm calculates the gradient of the model parameters according to the loss function, and then updates the model weights through gradient descent. The weights are updated by the following formula:

[0116]

[0117] Where η is the learning rate, is the gradient of the loss function with respect to the parameter θ; through iterative optimization of backpropagation and gradient descent, the model continuously adjusts parameters to reduce losses and improve performance.

[0118] Furthermore, after the initial training, the model usually needs to be fine-tuned. The fine-tuning process includes adjusting hyperparameters, such as learning rate, batch size, etc. By further training the model on the training set and validation set, fine-tuning can help the model better adapt to new data and tasks, thereby improving segmentation accuracy. The fine-tuned model can usually further improve performance and achieve better segmentation results.

[0119] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0120] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A remote sensing image target segmentation method based on feature adaptive processing, characterized in that: The following steps are involved: Step 1: Collect remote sensing images and perform preprocessing and data enhancement to obtain training data sets; Step 2: Construct a remote sensing image segmentation network and use the training data set for training to obtain a remote sensing image segmentation model; the remote sensing image segmentation network includes an encoder, a feature adaptive processing module, a multi-scale feature fusion module and a decoder connected in sequence; Step 3: Collect the remote sensing image to be segmented, input it into the remote sensing image segmentation model, and obtain the target segmentation result.

2. The remote sensing image target segmentation method based on feature adaptive processing according to claim 1, characterized in that: In step 1, preprocessing includes cropping, scaling and labeling, and data enhancement includes rotation, inversion and color adjustment.

3. The remote sensing image target segmentation method based on feature adaptive processing according to claim 2, characterized in that: The process of preprocessing and data enhancement of remote sensing images in step 1 includes: Step 11: Annotate the remote sensing image, normalize the annotated remote sensing image, and normalize the pixels to [0,1]; Step 12: Crop the standardized remote sensing image according to the preset target size; Step 13: Randomly rotate or horizontally flip or vertically flip the cropped image to obtain a transformed image; Step 14: Perform color adjustment on the transformed image to obtain the adjusted image Step 15: Generate segmentation label ψ based on the annotation information of the remote sensing image and adjust the image Paired with the segmentation label ψ to form a training data set 4. The remote sensing image target segmentation method based on feature adaptive processing according to claim 1, characterized in that: The process of remote sensing image segmentation network processing training images includes: The images in the training data set are input into the encoder for feature map extraction to obtain high-dimensional feature maps of multiple scales; The feature adaptive processing module performs pooling operations, fusion operations, and multi-head attention mechanism enhancement operations on the high-dimensional feature map in sequence to obtain an adaptive feature map; The multi-scale feature fusion module performs feature extraction and adaptive fusion on the adaptive feature map to obtain a fused feature map; The decoder performs weighted fusion, convolution adjustment and upsampling on the fused feature map in sequence to obtain the target segmentation result.

5. The method for remote sensing image target segmentation based on feature adaptive processing according to claim 4, characterized in that: The encoder consists of three groups of convolution modules. Each convolution module consists of a convolution layer, an activation function layer, and a pooling layer connected in sequence. Each convolution module processes the image as follows: X i =MaxPool(ReLU(Conv(X i-1 ))) for i=1,2,3 Among them, X i-1 represents the image input to the convolution module, X i Represents the feature map output by the convolution module; i represents the i-th convolution module, and the first convolution module inputs the image in the training data set; Conv represents the convolution operation; ReLU represents the activation function; MaxPool represents the maximum pooling operation; the first convolution module outputs the high-dimensional feature map X1, the second convolution module outputs the high-dimensional feature map X2, and the third convolution module outputs the high-dimensional feature map X3.

6. The method for remote sensing image target segmentation based on feature adaptive processing according to claim 5, characterized in that: The feature adaptive processing module includes the first average pooling layer, the second average pooling layer, the initial fusion layer, the multi-head attention layer, the first 1×1 convolution layer, the activation function layer, the second 1×1 convolution layer and the attention fusion layer. The specific processing process is as follows: Step 211: The high-dimensional feature map X1 is passed through the first average pooling layer to obtain a pooled feature map X′1, and the high-dimensional feature map X2 is passed through the second average pooling layer to obtain a pooled feature map X′2; Step 212: The pooled feature map X′1, the pooled feature map X′2 and the high-dimensional feature map X3 are concatenated or weighted fused at the initial fusion layer to obtain the initial fusion feature map X fusion ; Step 213: Initial fusion feature X fusion In the multi-head attention layer, it is mapped to multiple subspaces, and multiple detection heads are used for parallel calculation to obtain multiple attention maps, and all attention maps are combined with the initial fusion feature X fusion Multiply to get the attention weighted feature map X attn ; Step 214: Attention weighted feature map X attn Perform convolution operation in the first 1×1 convolution layer to obtain a compressed feature map; Step 215: The compressed feature map is activated in the activation function layer to obtain an activated feature map; Step 216: The activated feature map is convolved in the second 1×1 convolutional layer to obtain a weight map W; Step 217: The attention fusion layer adds the attention weighted feature map X attn Multiply it with the weight map W to obtain the adaptive feature map X enhanced .

7. The method for remote sensing image target segmentation based on feature adaptive processing according to claim 4, characterized in that: The multi-scale feature fusion module includes an upper branch, a middle branch and a lower branch; the upper branch includes the third 1×1 convolution layer and the third average pooling layer connected in sequence; the middle branch includes the fourth 1×1 convolution layer, the activation function layer, the fourth average pooling layer, the scale fusion layer and the scale splicing layer connected in sequence; the lower branch includes the fifth average pooling layer, the branch splicing layer, the fifth 1×1 convolution layer and the sixth 1×1 convolution layer; the processing process of the multi-scale feature fusion module is: Step 221: After the adaptive feature map is convolved by the third 1×1 convolutional layer, an average pooling operation is performed in the third average pooling layer to obtain the global feature x α , expressed as: x α =AvgPool(1×1Conv(X enhanced )) Among them, AvgPool represents average pooling; 1×1Conv represents 1*1 convolution operation; X enhanced represents the adaptive feature map; After the adaptive feature map passes through the convolution operation of the fourth 1×1 convolution layer, it passes through the activation operation of the activation function layer and the average pooling operation of the fourth average pooling layer in sequence to obtain the weight map A β , expressed as: A β =AvgPool(ReLU(1×1Conv(X enhanced ))) Among them, RELU means applying the RELU activation function; The adaptive feature map is obtained after the average pooling operation of the fifth average pooling layer. Feature Map The adaptive feature map is added to the branch concatenation layer to obtain the concatenated feature map. The enhanced feature map is sequentially convolved through the fifth 1×1 convolution layer and the sixth 1×1 convolution layer to obtain the enhanced feature map x γ , expressed as: Step 222: Enhance feature map x γ Input to the scale fusion layer and weight map A β Multiply to obtain the enhanced weighted feature map x β , expressed as: x β =A β ·x γ ; Step 223: Enhance the weighted feature map x β and the global feature map x α Input to the scale concatenation layer for addition to obtain the fusion feature map X″ enhanced .

8. The method for remote sensing image target segmentation based on feature adaptive processing according to claim 4, characterized in that: The decoder consists of a convolutional layer, an upsampling layer, and an activation function layer connected in sequence; The fused feature map undergoes the convolution operation of the convolution layer, the bilinear interpolation upsampling operation of the upsampling layer, and the activation operation of the activation function layer in sequence to obtain the target segmentation result.

9. The method for remote sensing image target segmentation based on feature adaptive processing according to claim 3, characterized in that: The remote sensing image segmentation network uses a mixed loss function L to optimize the model during the training process, and the expression is: L=εL1+δL2+μL3 Among them, L1 represents the cross entropy loss; L2 represents the Dice loss; L3 represents the IoU value; N represents the total number of pixels of the adjusted image in the training dataset; P i represents the predicted probability at pixel i in the target segmentation result predicted by the remote sensing image segmentation network; ψ i represents the segmentation label of pixel i in the training dataset; ε, δ, and μ are the scaling coefficients respectively.

10. The method for remote sensing image target segmentation based on feature adaptive processing according to claim 9, characterized in that: The remote sensing image segmentation network minimizes the hybrid loss function through back propagation and gradient descent during the training process; the back propagation algorithm calculates the gradient of the model parameters according to the hybrid loss function, and then updates the network weights through gradient descent.

Images