Remote sensing image segmentation error correction method, system, medium and equipment

Abstract

The invention discloses a remote sensing image segmentation error correction method and system, a medium and equipment in the technical field of deep learning, and aims to solve the optimization problem of remote sensing image segmentation error correction repair. The method comprises the following steps: performing data preprocessing on acquired remote sensing image data to obtain a remote sensing image and initial segmentation; inputting the remote sensing image and the initial segmentation into a trained remote sensing image segmentation error correction model; performing feature extraction through an IS feature extraction module to obtain multi-scale advanced semantic features and initial segmentation features; performing error estimation on the multi-scale advanced semantic features and the initial segmentation features through a channel space attention module to obtain error estimation features and error estimation results; and performing error correction on the initial segmentation feature, the error estimation feature and the error estimation result through an error notification refining module to obtain a segmentation repair result. According to the method, the remote sensing image segmentation precision can be effectively improved, and high adaptability is shown in tasks with complex backgrounds and high detail requirements.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning, and particularly to a method, system, medium and device for remote sensing image segmentation error correction. Background Art

[0002] Remote sensing images usually have high resolution and multi-spectral characteristics, and contain complex ground object information (such as buildings, roads, vegetation, etc.), which makes it difficult for traditional image segmentation methods to accurately extract the target area. There are often problems such as noise, shadows, and occlusions in remote sensing images, resulting in inaccurate segmentation results.

[0003] Currently, significant progress has been made in remote sensing image segmentation methods based on deep learning, but there are still the following problems: loss of spatial information: traditional convolutional neural networks may lose spatial detail information during feature extraction, resulting in unclear segmentation boundaries; insufficient utilization of channel information: different channels of multi-spectral remote sensing images contain rich information, but existing methods have not fully explored the correlation between channels; accumulation of segmentation errors: deep learning models may generate local errors during the segmentation process, and these errors will accumulate as the network depth increases, affecting the final segmentation accuracy.

[0004] Existing error correction methods mostly rely on post-processing techniques, but these methods have high computational complexity and are difficult to be combined with deep learning models end-to-end. There is a lack of an error correction mechanism that can simultaneously utilize spatial and channel information to dynamically correct errors in the segmentation results. To overcome these challenges, researchers pursue the refinement of the initial segmentation result by predicting error pixels, where error pixels represent pixels that are inconsistent with the ground truth in the predicted segmentation result. Summary of the Invention

[0005] The purpose of the present invention is to solve the problems of loss of spatial information, insufficient utilization of channel information, and accumulation of segmentation errors existing in the existing remote sensing image segmentation technology, and provide a method, system, medium and device for remote sensing image segmentation error correction. By introducing a spatial attention mechanism and a channel attention mechanism, and combining error pixel prediction and dynamic correction strategies, it can effectively improve the accuracy of remote sensing image segmentation, especially in tasks with complex backgrounds and high detail requirements, showing strong adaptability.

[0006] To solve the above technical problems, the present invention is implemented by adopting the following technical solutions:

[0007] The present invention provides a method for remote sensing image segmentation error correction, including:

[0008] Performing data preprocessing on the obtained remote sensing image data to obtain a remote sensing image and an initial segmentation;

[0009] Input the remote sensing image and the initial segmentation into the trained error correction model for remote sensing image segmentation: Through the IS feature extraction module, extract features from the remote sensing image and the initial segmentation to obtain multi-scale high-level semantic features and initial segmentation features; Through the channel spatial attention module, estimate the errors of the multi-scale high-level semantic features and the initial segmentation features to obtain error estimation features and error estimation results; Through the error notification refinement module, correct the errors of the initial segmentation features, error estimation features and error estimation results to obtain the segmentation repair result.

[0010] Optionally, the data preprocessing includes image normalization, image cropping, and label perturbation.

[0011] Optionally, the IS feature extraction module uses HRNet-W48 as the backbone network, and the data processing process includes:

[0012] Stitch the remote sensing image and the initial segmentation and input them into an Encoder-Decoder structure for feature extraction to obtain multi-scale features;

[0013] According to the multi-scale features, perform feature encoding through a 1×1 convolutional layer and sample to different scales to obtain multi-scale high-level semantic features;

[0014] According to the initial segmentation, perform feature encoding through two 3×3 convolutional layers to obtain initial segmentation features;

[0015] Upsample the multi-scale features, input the sampling results into the auxiliary branch, and pass through a classifier with 2 output channels to obtain the auxiliary branch segmentation result.

[0016] Optionally, the data processing process of the channel spatial attention module includes:

[0017] Input the difference between the multi-scale high-level semantic features and the initial segmentation features into the channel attention module CAM: Through average pooling and max pooling operations, obtain the average response feature and the max response feature in the spatial dimension; Send the average response feature and the max response feature in the spatial dimension into a shared multi-layer perceptron MLP for learning to obtain two enhanced features; Through the Add operation, fuse the two enhanced features to obtain a fused feature; Use the sigmoid function to calculate the frequency of each channel, and weight the fused feature according to the frequency to obtain the enhanced feature based on the channel relationship;

[0018] Input the difference between the multi-scale high-level semantic features and the initial segmentation features into the spatial attention module SAM: obtain the average response feature and the maximum response feature in the channel dimension through average pooling and maximum pooling operations; concatenate the average response feature and the maximum response feature in the channel dimension according to the channel dimension to obtain the concatenated feature; perform convolution processing on the concatenated feature using a 7X7 convolutional layer and weight it using the sigmoid function to obtain the enhanced feature based on the spatial relationship.

[0019] Fuse the enhanced feature based on the channel relationship and the enhanced feature based on the spatial relationship to obtain the error estimation feature.

[0020] According to the error estimation feature, map it to the output of 3 channels through a 1×1 convolutional layer, corresponding to correctly segmented pixels, under-segmented pixels, and over-segmented pixels respectively, to obtain the error estimation result.

[0021] Optionally, the data processing flow of the error notification refinement module includes:

[0022] Fuse the initial segmentation feature, the error estimation feature, and the error estimation result through a 1×1 convolutional layer to obtain the fused feature.

[0023] Reduce the number of channels of the error notification refinement module from 2c + 3 to c, perform feature extraction on the fused feature through a 3×3 convolution, and go through a 5×5 depthwise separable convolution and a 1×1 convolution to obtain the segmentation repair result.

[0024] Optionally, the overall loss of the remote sensing image segmentation error correction model includes the auxiliary branch loss , the error correction branch loss and the repair branch loss ;

[0025] The auxiliary branch loss is obtained through the following formula:

[0026]

[0027] where and represent preset parameters, represents the cross-entropy loss function, represents the Dice loss function, represents the auxiliary branch segmentation result, represents the ground truth of the auxiliary branch segmentation result;

[0028] The error correction branch loss is obtained through the following formula:

[0029]

[0030] Among them, and represent preset parameters, represents the error estimation result, represents the ground truth of the error estimation result;

[0031] The repair branch loss is obtained by the following formula:

[0032]

[0033] Among them, and represent preset parameters, represents the segmentation repair result;

[0034] The overall loss of the remote sensing image segmentation error correction model is obtained by the following formula:

[0035]

[0036] Among them, , and represent preset parameters.

[0037] In a second aspect, the present invention provides a remote sensing image segmentation error correction system, including:

[0038] A preprocessing module for: performing data preprocessing on the obtained remote sensing image data to obtain preprocessed data, including a remote sensing image and an initial segmentation;

[0039] An IS feature extraction module for: extracting features from the remote sensing image and the initial segmentation to obtain multi-scale high-level semantic features and initial segmentation features;

[0040] A channel spatial attention module for: performing error estimation on the multi-scale high-level semantic features and the initial segmentation features to obtain error estimation features and an error estimation result;

[0041] An error notification refinement module for: performing error correction based on the initial segmentation features, the error estimation features, and the error estimation result to obtain a segmentation repair result.

[0042] In a third aspect, the present invention provides a computer-readable storage medium, on which a computer program / instructions are stored. When the computer program / instructions are executed by a processor, the steps of any of the remote sensing image segmentation error correction methods in the first aspect are implemented.

[0043] In a fourth aspect, the present invention provides a computer device / equipment / system, including:

[0044] A memory for storing computer programs / instructions;

[0045] A processor for executing the computer programs / instructions to implement the steps of the remote sensing image segmentation error correction method according to any one of the first aspect.

[0046] In a fifth aspect, the present invention provides a computer program product, including computer programs / instructions, characterized in that when the computer programs / instructions are executed by a processor, the steps of the remote sensing image segmentation error correction method according to any one of the first aspect are implemented.

[0047] Compared with the prior art, the beneficial effects achieved by the present invention:

[0048] 1. For the remote sensing image segmentation error correction method provided by the present invention, by combining spatial and channel attention mechanisms, it effectively enhances the model's feature representation ability for different scales; through end-to-end training for optimization, it avoids the problems of high computational complexity commonly found in traditional post-processing methods (such as conditional random fields, etc.) and the difficulty of seamless integration with deep learning models; through error pixel prediction and dynamic correction strategies, it can timely identify and correct errors in the initial segmentation results, avoiding the accumulation of errors during the segmentation process.

[0049] 2. For the remote sensing image segmentation error correction system provided by the present invention, by setting a preprocessing module, an IS feature extraction module, a channel spatial attention module, and an error notification refinement module, it can effectively improve the accuracy of remote sensing image segmentation, not only improving the computational efficiency but also reducing the need for manual intervention, being applicable to remote sensing image processing tasks with high automation and strict processing speed requirements, facilitating engineering applications, and having practical significance and good application prospects.

[0050] 3. The computer-readable storage medium, computer device / equipment / system, and computer program product provided by the present invention can execute the steps of the remote sensing image segmentation error correction method provided by the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a flowchart of the remote sensing image segmentation error correction method according to an embodiment of the present invention;

[0052] Figure 2 It is a network structure diagram of the remote sensing image segmentation error correction model according to an embodiment of the present invention;

[0053] Figure 3 It is a structure diagram of the channel spatial attention module according to an embodiment of the present invention;

[0054] Figure 4 It is a remote sensing image according to an embodiment of the present invention;

[0055] Figure 5 It is the ground truth map of the initial segmentation of remote sensing images provided according to the embodiments of the present invention;

[0056] Figure 6 It is the initial segmentation map of remote sensing images provided according to the embodiments of the present invention;

[0057] Figure 7 It is the output map of the auxiliary branch of remote sensing images provided according to the embodiments of the present invention;

[0058] Figure 8 It is the error estimation result map of remote sensing images provided according to the embodiments of the present invention;

[0059] Figure 9 It is the ground truth map of the error estimation result of remote sensing images provided according to the embodiments of the present invention;

[0060] Figure 10 It is the segmentation repair result map of remote sensing images provided according to the embodiments of the present invention. Specific embodiments

[0061] The technical solution of the present invention will be described in detail below through the accompanying drawings and specific embodiments. It should be understood that the specific features in the embodiments of the present application and the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations on the technical solution of the present application. Without conflict, the technical features in the embodiments of the present application and the embodiments can be combined with each other.

[0062] It should be noted that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after.

[0063] Embodiment 1:

[0064] The embodiments of the present invention disclose a method for correcting errors in remote sensing image segmentation. Referring to Figure 1 as shown, the specific steps are as follows:

[0065] S1, perform data preprocessing on the obtained remote sensing image data to obtain a remote sensing image and an initial segmentation;

[0066] S2. Input the remote sensing image and the initial segmentation into the trained error correction model for remote sensing image segmentation: extract features from the remote sensing image and the initial segmentation through the IS feature extraction module to obtain multi-scale high-level semantic features and initial segmentation features; through the channel spatial attention module, estimate the error of the multi-scale high-level semantic features and the initial segmentation features to obtain the error estimation features and the error estimation results; through the error notification refinement module, correct the error of the initial segmentation features, the error estimation features and the error estimation results to obtain the segmentation repair result.

[0067] Specifically,

[0068] In step S1, select a suitable remote sensing image dataset and preprocess the data, including image normalization, image cropping, and label perturbation, etc., and organize it into a format convenient for subsequent processing; according to the task requirements, a part of the data can be randomly selected from the remote sensing image dataset as the training set, the validation set, and the test set to ensure the diversity and representativeness of the data.

[0069] In this embodiment, the error pixel MaskEme is defined as the pixel consistency between the perturbed label and its ground truth, which is expressed as:

[0070]

[0071] Where, is used to evaluate the consistency of the values of the th pixel in the perturbed label and the ground truth, and respectively represent the values of the th pixel in the perturbed label and the ground truth; if , it means that the perturbed label is accurate and does not need to be corrected, otherwise, the prediction is either under-segmented (i.e., , or over-segmented (i.e., ); the process of label perturbation is carried out online during the training process.

[0072] In step S2, the error correction model for remote sensing image segmentation includes an IS feature extraction module, a channel spatial attention module, and an error notification refinement module.

[0073] The IS feature extraction module uses HRNet-W48 as the backbone network, and the data processing flow includes:

[0074] Stitch and input the remote sensing image and the initial segmentation into the Encoder-Decoder structure for feature extraction to obtain multi-scale features;

[0075] According to the multi-scale features, feature encoding is performed through a 1×1 convolutional layer and sampled to different scales to obtain multi-scale high-level semantic features;

[0076] According to the initial segmentation, feature encoding is performed through two 3×3 convolutional layers to obtain initial segmentation features;

[0077] Upsample the multi-scale features, input the sampling result into the auxiliary branch, and pass through a classifier with 2 output channels to obtain the auxiliary branch segmentation result, which directly focuses on specific objects or regions in the input sample.

[0078] Reference Figure 2 As shown, the IS Feature Extractor includes image feature extraction (as the backbone) and initial segmentation feature extraction; First, image feature extraction is used to extract features of objects or regions of interest in the image. Its input is the image and the corresponding initial segmentation result. HRNet-W48 is used as the backbone, which is an Encoder-Decoder structure and the classifier is removed; The multi-scale features of the backbone are first upsampled to obtain the sampling result, and this sampling result is input into the classifier of the auxiliary branch to obtain the auxiliary branch segmentation result; Then, the features before being input into the classifier are concatenated and passed through a convolutional layer with a kernel of 1×1 and sampled to different scales (1 / 4, 1 / 8, 1 / 16, and 1 / 32), which are called multi-scale high-level semantic features (Multi-scale and High-level semantic features); At the same time, two 3×3 convolutional layers are used for initial segmentation feature extraction to encode the features of the initial segmentation and obtain the initial segmentation features (Initial Feature); The multi-scale high-level semantic features and the initial segmentation features will be used for error estimation in the next stage, and the auxiliary branch segmentation result is used to calculate the auxiliary branch loss.

[0079] The auxiliary branch loss architecture of this embodiment is as follows: For the output and its corresponding ground truth , the cross-entropy (CE) loss and Dice loss are used, where and are 0.5 and 0.5 respectively; The auxiliary branch loss is obtained through the following formula:

[0080]

[0081] where, and represent preset parameters, represents the cross - entropy loss function, represents the Dice loss function, represents the segmentation result of the auxiliary branch, represents the ground truth of the segmentation result of the auxiliary branch.

[0082] Assume that the result output by the auxiliary branch is , and its corresponding truth matrix (i.e., ) is . By comparing the two, it is obtained that the elements in the red part of

[0083] The data processing flow of the channel - spatial attention module includes:

[0084] Input the difference between the multi - scale high - level semantic feature and the initial segmentation feature into the channel attention module CAM: Through average pooling and max - pooling operations, obtain the average response feature and the max response feature in the spatial dimension; Send the average response feature and the max response feature in the spatial dimension into a shared multi - layer perceptron MLP for learning respectively, to obtain two enhanced features; Through the Add operation, fuse the two enhanced features to obtain a fused feature; Use the sigmoid function to calculate the frequency of each channel, and weight the fused feature according to the frequency to obtain an enhanced feature based on channel relationships;

[0085] Input the difference between the multi - scale high - level semantic feature and the initial segmentation feature into the spatial attention module SAM: Through average pooling and max - pooling operations, obtain the average response feature and the max response feature in the channel dimension; Concatenate the average response feature and the max response feature in the channel dimension according to the channel dimension to obtain a concatenated feature; Use a 7X7 convolutional layer to perform convolutional processing on the concatenated feature, and use the sigmoid function for weighting to obtain an enhanced feature based on spatial relationships;

[0086] Fuse the enhanced feature based on channel relationships and the enhanced feature based on spatial relationships to obtain an error estimation feature;

[0087] According to the error estimation feature, map it to the output of 3 channels through a 1×1 convolutional layer, corresponding to correctly segmented pixels, under - segmented pixels, and over - segmented pixels respectively, to obtain an error estimation result.

[0088] In this embodiment, the channel - spatial attention module is used for error estimation (Error Estimator). When correcting errors in image segmentation, the image and the corresponding initial segmentation need to be analyzed from both global and local scales:

[0089] Global scale: The network can easily detect large annotation errors, such as large-scale error regions;

[0090] Local scale: For small annotation errors, such as errors in object boundaries or details, the network needs to carefully examine at the local scale.

[0091] The Channel-Spatial Attention Module (CSAM) is a module for enhancing multi-scale feature representations, aiming to improve the model's performance in object detection and image segmentation; it includes a Channel Attention Module (CAM) and a Spatial Attention Module (SAM), which enhance features from the channel dimension and spatial dimension respectively to ensure that the model can capture key features at different scales; the multi-scale high-level semantic features obtained in the previous step and the initial segmentation features are subtracted in the same scale for image features, and the resulting result is used as the input feature of the CSAM module.

[0092] In the Channel Attention Module (CAM), the input is the difference between the multi-scale high-level semantic features and the initial segmentation features; through Average Pooling and Max Pooling operations, CAM can capture the maximum response signals in the spatial dimension; then, a fully connected layer is used to learn and enhance these response signals, and finally, the sigmoid function is used to calculate the frequency of each channel, and these frequencies are multiplied by the input features to obtain the enhanced information based on channel relationships.

[0093] In the Spatial Attention Module (SAM), the input is the difference between the multi-scale high-level semantic features and the initial segmentation features, mainly focusing on the response signals in the spatial dimension; SAM pools the image features through windows of different sizes (such as 1×1, 3×3, 7×7) to extract the most significant features on the channel; subsequently, a 1×1 convolutional layer is used to further process these signals, and finally, the input features are weighted through sigmoid operations to enhance spatially related features.

[0094] The final output of CSAM is obtained by adding the enhanced features of CAM and SAM to the original input features; this process effectively strengthens the model's feature expression in different dimensions and improves the model's ability to understand complex information; the features processed by CSAM are called Error Estimator Features, and through a 1×1 convolutional layer, the input out_channels channels are mapped to the output of 3 channels, corresponding to correctly segmented pixels, under-segmented pixels, and over-segmented pixels respectively, and this result is called Error Estimator Prediction, which focuses on the error-occurring regions in the input samples.

[0095] This embodiment also includes an error correction branch loss architecture as follows: Using Dice and CE losses to optimize the error estimation result and its error ground truth The relationship between them; where and are 0.5 and 0.5 respectively; the error correction branch loss is obtained through the following formula:

[0096]

[0097] where and represent preset parameters, represents the error estimation result, represents the ground truth of the error estimation result.

[0098] Assume that the initial segmentation matrix of the current remote sensing image is , the error estimation result is , and its corresponding truth matrix (i.e., ) is , and In, 0 indicates that the pixel at this position is correctly segmented, 1 indicates that the pixel at this position is under-segmented, and 2 indicates that the pixel at this position is over-segmented.

[0099] The error estimation result is used to evaluate the segmentation situation of each pixel point in the initial segmentation matrix, represents the correct evaluation of the segmentation situation of each pixel point in the initial segmentation matrix. After comparison, the red parts of the two are different, indicating that the error estimation result evaluates the segmentation situation of the pixel values in the red part incorrectly.

[0100] The data processing flow of the error notification refinement module includes:

[0101] Fuse the initial segmentation feature, error estimation feature, and error estimation result through a 1×1 convolutional layer to obtain a fused feature;

[0102] Reduce the number of channels of the error notification refinement module from 2c + 3 to c, perform feature extraction on the fused feature through a 3×3 convolution, and through a 5×5 depthwise separable convolution and a 1×1 convolution to obtain a segmentation repair result.

[0103] The error notification refinement module is a refinement component designed to guide the correction of the prediction result through error information in the error correction model; it is implemented through the EGF module, and the input of the EGF module includes:

[0104] Initial Feature: The output from the IS feature extraction module;

[0105] Error Estimator Feature: The feature after being processed by CSAM, representing the error or uncertainty generated during the prediction process;

[0106] Error Estimator Prediction: The error pixel prediction result obtained by CSAM, representing the error regions or inaccurate regions identified in the model prediction.

[0107] The EGF module first uses a 1×1 convolutional layer to fuse the input features (Initial Feature, Error Estimator Feature, and Error Estimator Prediction), and reduces the number of channels from 2c + 3 to c. Then, it uses a 3×3 convolution for feature extraction; this helps to extract more detailed information, and then inputs it into a 5×5 depthwise separable convolution and a 1×1 convolution to obtain the final segmentation repair result, which focuses on the global and local segmentation results of the image.

[0108] This embodiment also includes a repair branch loss architecture as follows: Using a loss function similar to that of the auxiliary branch; Using Dice and CE losses to optimize the segmentation repair result With its corresponding ground truth The relationship between them, where and are 0.5 and 0.5 respectively; The repair branch loss is obtained through the following formula:

[0109]

[0110] where and represent preset parameters, represents the segmentation repair result.

[0111] Assume the segmentation repair result is , and the ground truth matrix (i.e., ) is , and the error estimation result is a further correction to the initial segmentation matrix; By comparing the two matrices, it can be seen that the segmentation repair result is correct, and each pixel value is correctly segmented, which is the ideal state that the model hopes to achieve and can correctly repair each mis-segmented point.

[0112] This embodiment adopts end-to-end training and optimization, integrating the IS feature extraction module, the spatial and channel attention module, and the error pixel correction module into a deep learning network for end-to-end training. During the training process, the network weights are updated by calculating the loss function and backpropagation, gradually optimizing the segmentation accuracy of the model. The loss function is designed to consider the contributions of both spatial information and channel information simultaneously, enabling the model to achieve an effective balance between the two.

[0113] The overall loss of the remote sensing image segmentation error correction model includes the auxiliary branch loss and the error correction branch loss and the repair branch loss ; the overall loss of the remote sensing image segmentation error correction model is obtained through the following formula:

[0114]

[0115] where , and represent preset parameters, which are 0.2, 0.4, and 0.4 respectively.

[0116] In this embodiment, the WHU Building Dataset is selected for training and testing. This dataset provides high-resolution remote sensing image data and detailed location information of buildings in the images. Each image contains a large number of building annotations with high annotation quality and uniform image quality, which can well train and verify the building extraction and segmentation tasks in remote sensing images. The dataset includes the following parts:

[0117] (1) The overall dataset contains 7,152 images, with approximately 200,000 buildings annotated. Ground resolution: 0.3 meters. The training set is a set of 4,736 images (512x512 pixel RGB images). The validation set is a set of 1,036 images (512x512 pixel RGB images). The test set provides 2,416 images (512x512 pixel RGB images);

[0118] (2) The label data is provided using two semantic classes at the pixel level, including buildings (white 255) and non-buildings (black 0);

[0119] (3) To enhance the generalization of the error correction model in any segmentation model, the initial segmentation of the existing model is avoided during the training process. Different perturbations are applied to the label mask in this application to simulate instance-level segmentation errors.

[0120] In this embodiment, all training set images are selected from the dataset for training, and the validation set images are used for model evaluation. The remote sensing dataset is read to obtain image and label information, including the image name image_id, the image size image_size, and the label categories (building 255 and background 0). In the data loading stage, online data augmentation is adopted, including random rotation, shearing, shifting, scaling, and horizontal and vertical flipping, to expand the training dataset. At the same time, instance-level errors are simulated by randomly removing masks and segmenting adjacent masks, and false positive instances are randomly added using graph-based segmentation. This method generates a wide distribution of potential segmentation errors, thus achieving more robust training.

[0121] This training is implemented using the Pytorch framework on an NVIDIA Tesla V100 GPU. The Adam optimizer is used, and the training runs for 25,600 iterations with a learning rate of 0.000125 and a batch size of 8. Color and depth augmentation are applied to simulate the real-world transfer. The label perturbation is Online_dataset during the training stage and Offline_dataset during the validation and testing stages. RGB images with a resolution of 512×512 are used for training, testing, and validation. At the same time, the loss function is calculated, the gradients of the model parameters are calculated, and the network weights are updated according to the gradients.

[0122] Figures 4 to 10 are the inference results of this embodiment on the test set, where Figure 4 is the remote sensing image, Figure 5 is the ground truth map of the initial segmentation of the remote sensing image, Figure 6 is the initial segmentation map of the remote sensing image after label perturbation, Figure 7 is the output map of the auxiliary branch of the remote sensing image, Figure 8 is the error estimation result map of the remote sensing image, Figure 9 is the ground truth map of the error estimation result of the remote sensing image, Figure 10 is the segmentation repair result map of the remote sensing image; the error notification refinement module can Figure 8 repair the initial segmentation of Figure 6 according to the error estimation result, making it closer to the ground truth Figure 5 .

[0123] After the network training is completed, the network weights are saved to a pth file. With the pth file, the initial segmentation results of any input model can be directly corrected and repaired, and the final result is output after visualization.

[0124] In summary, the remote sensing image segmentation error correction method proposed in this embodiment solves the common problems of feature loss and insufficient information in existing methods by fully combining spatial and channel attention mechanisms; different from traditional post-processing error correction methods, it can be trained end-to-end and does not rely on complex post-processing steps, improving the processing efficiency and segmentation accuracy; its advantages include:

[0125] (1) Improving segmentation accuracy and detail capture ability: By combining spatial and channel attention mechanisms, the model's ability to represent features at different scales is effectively enhanced; the spatial attention mechanism can focus on key regions in the image, especially when dealing with boundaries and details, while the channel attention mechanism can fully explore the correlations between the multi-spectral channels of remote sensing images; this multi-dimensional information fusion greatly improves the model's performance in complex regions and detail capture, making the segmentation results more accurate, especially in environments with challenging factors such as noise, occlusion, and shadows, where the segmentation accuracy can be significantly improved;

[0126] (2) End-to-end training and efficient optimization: This method is fully optimized through end-to-end training, avoiding the common problems of high computational complexity and difficulty in seamless integration with deep learning models in traditional post-processing methods (such as conditional random fields, etc.); through the network architecture that combines spatial and channel attention mechanisms, the model can adaptively adjust feature extraction and error correction during training, ultimately achieving an efficient and accurate segmentation effect; this end-to-end training method makes the model training process simpler, more adaptable, and capable of processing large-scale remote sensing image data to meet the requirements of different application scenarios;

[0127] (3) Dynamically correcting segmentation errors and enhancing robustness: Through error pixel prediction and dynamic correction strategies, errors in the initial segmentation results can be timely identified and corrected, avoiding the accumulation of errors during the segmentation process; the error-guided refinement module (EGF) accurately identifies inconsistent pixel regions and effectively repairs them by fusing error estimation features, error estimation results, and image features; this mechanism not only improves the segmentation accuracy but also enables the model to maintain high robustness when processing remote sensing images with complex backgrounds and rich details, ensuring the stability and consistency of the segmentation results;

[0128] (4) Eliminating the need for complex post-processing and improving processing efficiency: Compared with traditional image segmentation methods, this method does not rely on complex post-processing steps, but directly optimizes the segmentation results through the error estimation and correction mechanism within the model; this simplified processing flow not only improves the computational efficiency but also reduces the need for manual intervention, making it suitable for remote sensing image processing tasks with high automation requirements and strict processing speed requirements; in addition, the end-to-end training method reduces the latency of the model during the inference stage, making real-time or large-scale image segmentation possible;

[0129] (5) Wide applicability and scalability: This method is not only applicable to remote sensing image segmentation tasks, but also has strong generality and scalability; The error correction mechanism combining spatial and channel attention can be extended and applied to image segmentation tasks in other fields, such as medical image segmentation, satellite image analysis, autonomous driving, etc.; Due to its efficient feature expression and dynamic correction ability, this technology can flexibly adapt to various complex scenarios, provide accurate segmentation results, and has wide practical application value.

[0130] Example Two:

[0131] Based on the same inventive concept as Example One, this embodiment of the present invention discloses a remote sensing image segmentation error correction system, including:

[0132] A preprocessing module, configured to: perform data preprocessing on the acquired remote sensing image data to obtain preprocessed data, including remote sensing images and initial segmentations;

[0133] An IS feature extraction module, configured to: extract features from the remote sensing image and the initial segmentation to obtain multi-scale high-level semantic features and initial segmentation features;

[0134] A channel spatial attention module, configured to: perform error estimation on the multi-scale high-level semantic features and the initial segmentation features to obtain error estimation features and error estimation results;

[0135] An error notification refinement module, configured to: perform error correction based on the initial segmentation features, error estimation features, and error estimation results to obtain a segmentation repair result.

[0136] For the specific function implementation of the above modules, refer to the relevant content in the method of Example One, which will not be elaborated here.

[0137] Example Three:

[0138] This embodiment provides a computer-readable storage medium, on which computer programs / instructions are stored. When the computer programs / instructions are executed by a processor, the steps of the remote sensing image segmentation error correction method described in any one of Example One are implemented.

[0139] Example Four:

[0140] This embodiment provides a computer device / system, including:

[0141] A memory, configured to store computer programs / instructions;

[0142] A processor, configured to execute the computer programs / instructions to implement the steps of the remote sensing image segmentation error correction method described in any one of the first aspect.

[0143] Example Five:

[0144] This embodiment provides a computer program product, including computer programs / instructions, characterized in that when the computer programs / instructions are executed by a processor, the steps of the remote sensing image segmentation error correction method described in any one of the first embodiment are implemented.

[0145] Those skilled in the art should understand that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0146] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0147] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0149] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit of the present invention and the scope protected by the claims. All of these fall within the protection scope of the present invention.

Claims

1. A method for correcting remote sensing image segmentation errors, characterized in that Including: Performing data preprocessing on the acquired remote sensing image data to obtain a remote sensing image and an initial segmentation; Inputting the remote sensing image and the initial segmentation into a trained remote sensing image segmentation error correction model: extracting features from the remote sensing image and the initial segmentation through an IS feature extraction module to obtain multi-scale high-level semantic features and initial segmentation features; estimating errors for the multi-scale high-level semantic features and the initial segmentation features through a channel spatial attention module to obtain error estimation features and an error estimation result; correcting errors for the initial segmentation features, the error estimation features, and the error estimation result through an error notification refinement module to obtain a segmentation repair result.

2. The remote sensing image segmentation error correction method according to claim 1, characterized in that, The data preprocessing includes image normalization, image cropping, and label perturbation.

3. The remote sensing image segmentation error correction method according to claim 1, characterized in that: The IS feature extraction module uses HRNet-W48 as the backbone network, and the data processing process includes: Stitching the remote sensing image and the initial segmentation and inputting them into an Encoder-Decoder structure for feature extraction to obtain multi-scale features; Encoding the multi-scale features through a 1×1 convolutional layer and sampling them to different scales to obtain multi-scale high-level semantic features; Encoding the initial segmentation through two 3×3 convolutional layers to obtain initial segmentation features; Upsampling the multi-scale features, inputting the sampling result into an auxiliary branch, and passing it through a classifier with 2 output channels to obtain an auxiliary branch segmentation result.

4. The remote sensing image segmentation error correction method according to claim 1, wherein The data processing process of the channel spatial attention module includes: Inputting the difference between the multi-scale high-level semantic features and the initial segmentation features into a channel attention module CAM: obtaining the average response feature and the maximum response feature in the spatial dimension through average pooling and maximum pooling operations; respectively sending the average response feature and the maximum response feature in the spatial dimension into a shared multi-layer perceptron MLP for learning to obtain two enhanced features; fusing the two enhanced features through an Add operation to obtain a fused feature; calculating the frequency of each channel using a sigmoid function, and weighting the fused feature according to the frequency to obtain an enhanced feature based on channel relationships; Inputting the difference between the multi-scale high-level semantic features and the initial segmentation features into a spatial attention module SAM: obtaining the average response feature and the maximum response feature in the channel dimension through average pooling and maximum pooling operations; concatenating the average response feature and the maximum response feature in the channel dimension according to the channel dimension to obtain a concatenated feature; performing convolutional processing on the concatenated feature using a 7X7 convolutional layer and weighting it using a sigmoid function to obtain an enhanced feature based on spatial relationships; Fusing the enhanced feature based on channel relationships and the enhanced feature based on spatial relationships to obtain error estimation features; Mapping the error estimation features to the output of 3 channels through a 1×1 convolutional layer, corresponding to correctly segmented pixels, under-segmented pixels, and over-segmented pixels respectively, to obtain an error estimation result.

5. The remote sensing image segmentation error correction method according to claim 1, characterized in that The data processing process of the error notification refinement module includes: Fuse the initial segmentation features, error estimation features, and error estimation results through a 1×1 convolutional layer to obtain fused features; Reduce the number of channels of the error notification refinement module from 2c + 3 to c, perform feature extraction on the fused features through a 3×3 convolution, and obtain a segmentation repair result after a 5×5 depthwise separable convolution and a 1×1 convolution.

6. The remote sensing image segmentation error correction method according to claim 3, characterized in that, The overall loss of the remote sensing image segmentation error correction model includes the auxiliary branch loss , the error correction branch loss and the repair branch loss ; The auxiliary branch loss is obtained by the following formula: Among them, and represent preset parameters, represents the cross-entropy loss function, represents the Dice loss function, represents the segmentation result of the auxiliary branch, represents the ground truth of the segmentation result of the auxiliary branch; The error correction branch loss is obtained by the following formula: Among them, and represent preset parameters, represents the error estimation result, represents the ground truth of the error estimation result; The repaired branch loss is obtained by the following formula: Among them, and represent preset parameters, represents the split repair result; The overall loss of the remote sensing image segmentation error correction model is obtained through the following formula: Among them, , and represent preset parameters.

7. A remote sensing image segmentation error correction system, characterized in that, Comprising: A preprocessing module for preprocessing the acquired remote sensing image data to obtain preprocessed data, including a remote sensing image and an initial segmentation; An IS feature extraction module for extracting features from the remote sensing image and the initial segmentation to obtain multi-scale high-level semantic features and initial segmentation features; A channel spatial attention module for estimating errors in the multi-scale high-level semantic features and initial segmentation features to obtain error estimation features and error estimation results; An error notification refinement module for correcting errors based on the initial segmentation features, error estimation features, and error estimation results to obtain a segmentation repair result.

8. A computer-readable storage medium having computer programs / instructions stored thereon, characterized in that, When the computer program / instructions are executed by a processor, the steps of the remote sensing image segmentation error correction method according to any one of claims 1-6 are implemented.

9. A computer device / equipment / system, characterized in that, Comprising: A memory for storing computer programs / instructions; A processor for executing the computer programs / instructions to implement the steps of the remote sensing image segmentation error correction method according to any one of claims 1-6.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by a processor, the steps of the remote sensing image segmentation error correction method according to any one of claims 1-6 are implemented.

Images