A remote sensing image segmentation error correction method, system, medium and device

Abstract

The application discloses a kind of remote sensing image segmentation error correction method, system, medium and equipment in the technical field of deep learning, to solve the optimization problem of remote sensing image segmentation error correction repair. Method includes: the remote sensing image data obtained is carried out data preprocessing, obtains remote sensing image and initial segmentation;Remote sensing image and initial segmentation are input into trained remote sensing image segmentation error correction model: carry out feature extraction by IS feature extraction module, obtain multi-scale high-level semantic feature and initial segmentation feature;Through channel space attention module, error estimation is carried out to multi-scale high-level semantic feature and initial segmentation feature, to obtain error estimation feature and error estimation result;Through error notification refinement module, error correction is carried out to initial segmentation feature, error estimation feature and error estimation result, to obtain segmentation repair result.The application can effectively improve the segmentation accuracy of remote sensing image, and shows strong adaptability in complex background and high detail requirement task.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of deep learning, in particular to a remote sensing image segmentation error correction method, system, medium and equipment. BACKGROUND

[0002] Remote sensing images usually have high resolution, multi-spectral characteristics, and contain complex ground information (such as buildings, roads, vegetation, etc.), which makes it difficult for traditional image segmentation methods to accurately extract target regions. There are often problems such as noise, shadow, occlusion in remote sensing images, which lead to inaccurate segmentation results.

[0003] At present, the deep learning-based remote sensing image segmentation method has made significant progress, but there are still the following problems: spatial information loss: traditional convolutional neural networks may lose spatial detail information during feature extraction, resulting in unclear segmentation boundaries; insufficient utilization of channel information: different channels of multi-spectral remote sensing images contain rich information, but existing methods fail to fully exploit the correlation between channels; segmentation error accumulation: deep learning models may produce local errors during segmentation, which will accumulate with the increase of network depth, affecting the final segmentation accuracy.

[0004] Existing error correction methods rely heavily on post-processing techniques, but these methods have high computational complexity and are difficult to integrate with deep learning models end-to-end. There is a lack of an error correction mechanism that can simultaneously utilize spatial and channel information to dynamically correct errors in the segmentation results. To overcome these challenges, researchers pursue the re-refinement of the initial segmentation results by predicting error pixels, where error pixels represent pixels that are inconsistent with the underlying truth in the predicted segmentation results. SUMMARY

[0005] The present application aims to solve the problems of spatial information loss, insufficient utilization of channel information, and segmentation error accumulation in existing remote sensing image segmentation techniques, and provides a remote sensing image segmentation error correction method, system, medium and equipment. By introducing spatial attention mechanisms and channel attention mechanisms, combining error pixel prediction and dynamic correction strategies, the remote sensing image segmentation accuracy can be effectively improved, especially in complex backgrounds and tasks with high detail requirements, showing strong adaptability.

[0006] To solve the above technical problems, the present application is implemented by using the following technical solutions:

[0007] The present application provides a remote sensing image segmentation error correction method, comprising:

[0008] performing data preprocessing on the obtained remote sensing image data to obtain a remote sensing image and an initial segmentation;

[0009] inputting the remote sensing image and the initial segmentation into the trained remote sensing image segmentation error correction model: performing feature extraction on the remote sensing image and the initial segmentation through an IS feature extraction module to obtain multi-scale high-level semantic features and initial segmentation features; performing error estimation on the multi-scale high-level semantic features and the initial segmentation features through a channel space attention module to obtain error estimation features and error estimation results; performing error correction on the initial segmentation features, the error estimation features and the error estimation results through an error notification refinement module to obtain a segmentation repair result.

[0010] Optionally, the data preprocessing includes image standardization, image cropping and label disturbance.

[0011] Optionally, the IS feature extraction module adopts HRNet-W48 as a backbone network, and a data processing procedure includes:

[0012] concatenating the remote sensing image and the initial segmentation and inputting them into an Encoder-Decoder structure for feature extraction to obtain multi-scale features;

[0013] According to the multi-scale features, feature encoding is performed through a 1x1 convolutional layer, and sampling is performed to different scales to obtain multi-scale high-level semantic features;

[0014] According to the initial segmentation, feature encoding is performed through two 3x3 convolutional layers to obtain initial segmentation features;

[0015] The multi-scale features are up-sampled, and the sampling results are input into an auxiliary branch, and a classifier with an output channel of 2 is used to obtain an auxiliary branch segmentation result.

[0016] Optionally, a data processing procedure of the channel space attention module includes:

[0017] The difference between the multi-scale high-level semantic features and the initial segmentation features is input into a channel attention module CAM: average pooling and maximum pooling operations are used to obtain average response features and maximum response features in a spatial dimension; the average response features and the maximum response features in the spatial dimension are respectively input into a shared multi-layer perceptron MLP to learn, and two enhanced features are obtained; the two enhanced features are fused through an Add operation to obtain a fusion feature; a sigmoid function is used to calculate the frequency of each channel, and the fusion feature is weighted according to the frequency to obtain an enhanced feature based on channel relationship;

[0018] inputting the difference between the multi-scale advanced semantic feature and the initial segmentation feature into a spatial attention module SAM: obtaining average response features and maximum response features in a channel dimension through average pooling and maximum pooling operations; concatenating the average response features and the maximum response features in the channel dimension according to the channel dimension to obtain concatenated features; performing convolution processing on the concatenated features by using a 7X7 convolution layer, and performing weighting by using a sigmoid function to obtain an enhanced feature based on spatial relationship;

[0019] fusing the enhanced feature based on the channel relationship and the enhanced feature based on the spatial relationship to obtain an error estimation feature;

[0020] According to the error estimation feature, mapping to an output of 3 channels through a 1x1 convolution layer, respectively corresponding to correct segmentation pixels, under-segmented pixels and over-segmented pixels, to obtain an error estimation result.

[0021] Optionally, the data processing flow of the error notification refinement module includes:

[0022] fuse the initial segmentation feature, the error estimation feature and the error estimation result through a 1x1 convolution layer to obtain a fused feature;

[0023] reduce the number of channels of the error notification refinement module from 2c+3 to c, perform feature extraction on the fused feature through a 3x3 convolution, and obtain a segmentation repair result through a 5x5 depth separable convolution and a 1x1 convolution.

[0024] Optionally, the overall loss of the remote sensing image segmentation error correction model includes an auxiliary branch loss , an error correction branch loss and a repair branch loss .

[0025] The auxiliary branch loss is obtained by the following formula:

[0026]

[0027] wherein, and represent preset parameters, represents a cross-entropy loss function, represents a Dice loss function, represents an auxiliary branch segmentation result, represents a ground truth value of the auxiliary branch segmentation result;

[0028] The error correction branch loss is obtained by the following formula:

[0029]

[0030] wherein, and denote preset parameters, denote error estimation results, denote ground truth values of error estimation results;

[0031] the repair branch loss is obtained by the following formula:

[0032]

[0033] wherein, and denote preset parameters, denote segmentation repair results;

[0034] the overall loss of the remote sensing image segmentation error correction model is obtained by the following formula:

[0035]

[0036] wherein, , and denote preset parameters.

[0037] In a second aspect, the present application provides a remote sensing image segmentation error correction system, comprising:

[0038] a preprocessing module, configured to perform data preprocessing on the obtained remote sensing image data to obtain preprocessed data, including a remote sensing image and an initial segmentation;

[0039] an IS feature extraction module, configured to perform feature extraction on the remote sensing image and the initial segmentation to obtain multi-scale high-level semantic features and initial segmentation features;

[0040] a channel space attention module, configured to perform error estimation on the multi-scale high-level semantic features and the initial segmentation features to obtain error estimation features and error estimation results;

[0041] an error notification refinement module, configured to perform error correction according to the initial segmentation features, the error estimation features and the error estimation results to obtain segmentation repair results.

[0042] In a third aspect, the present application provides a computer readable storage medium having a computer program / instruction stored thereon, wherein the computer program / instruction is executed by a processor to implement the steps of the remote sensing image segmentation error correction method of any one of the first aspect.

[0043] In a fourth aspect, the present application provides a computer device / equipment / system, comprising:

[0044] a memory for storing computer programs / instructions;

[0045] a processor for executing the computer programs / instructions to implement the steps of the remote sensing image segmentation error correction method according to any one of the first aspect.

[0046] In a fifth aspect, the present application provides a computer program product comprising computer programs / instructions, wherein the computer programs / instructions, when executed by a processor, implement the steps of the remote sensing image segmentation error correction method according to any one of the first aspect.

[0047] Compared with the prior art, the present application has the following beneficial effects:

[0048] 1. The remote sensing image segmentation error correction method provided by the present application effectively enhances the model's feature representation ability for different scales by combining spatial and channel attention mechanisms; through end-to-end training for optimization, the method avoids the high computational complexity and the problem of difficult seamless combination with deep learning models that are common in traditional post-processing methods (such as conditional random fields, etc.); through error pixel prediction and dynamic correction strategies, the method can timely identify and correct errors in the initial segmentation results, avoiding the accumulation of errors in the segmentation process;

[0049] 2. The remote sensing image segmentation error correction system provided by the present application, by setting a preprocessing module, an IS feature extraction module, a channel spatial attention module, and an error notification refinement module, can effectively improve the segmentation accuracy of remote sensing images, not only improving the computational efficiency, but also reducing the need for manual intervention, suitable for remote sensing image processing tasks with high automation degree and strict processing speed requirements, facilitating engineering application, and having practical significance and good application prospects;

[0050] 3. The computer readable storage medium, computer device / system, and computer program product provided by the present application can execute the steps of the remote sensing image segmentation error correction method provided by the present application. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 A remote sensing image segmentation error correction method flowchart is provided according to an embodiment of the present application;

[0052] Figure 2 A network structure diagram of a remote sensing image segmentation error correction model is provided according to an embodiment of the present application;

[0053] Figure 3 A channel spatial attention module structure diagram is provided according to an embodiment of the present application;

[0054] Figure 4 A remote sensing image is provided according to an embodiment of the present application;

[0055] Figure 5 an initial segmentation ground truth map of a remote sensing image provided by an embodiment of the present application;

[0056] Figure 6 an initial segmentation map of a remote sensing image provided by an embodiment of the present application;

[0057] Figure 7 an auxiliary branch output map of a remote sensing image provided by an embodiment of the present application;

[0058] Figure 8 an error estimation result map of a remote sensing image provided by an embodiment of the present application;

[0059] Figure 9 an error estimation result ground truth map of a remote sensing image provided by an embodiment of the present application;

[0060] Figure 10 a segmentation repair result map of a remote sensing image provided by an embodiment of the present application. DETAILED DESCRIPTION

[0061] The technical solutions of the present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific features in the embodiments and the specific features in the embodiments of the present application are detailed descriptions of the technical solutions of the present application, and are not limitations of the technical solutions of the present application. In the case of no conflict, the technical features in the embodiments and the embodiments of the present application can be combined with each other.

[0062] It should be noted that the term "and / or" in this paper is only a description of the association relationship between the associated objects, which means that there can be three kinds of relationships, for example, A and / or B, which can represent the existence of A alone, the existence of A and B, and the existence of B alone. In addition, the character " / " in this paper generally represents a "or" relationship between the associated objects before and after it.

[0063] Embodiment one:

[0064] The embodiment of the present application discloses a remote sensing image segmentation error correction method, as shown in Figure 1 , specifically comprising the following steps:

[0065] S1, data preprocessing is performed on the obtained remote sensing image data to obtain a remote sensing image and an initial segmentation;

[0066] S2, input the remote sensing image and the initial segmentation into the trained remote sensing image segmentation error correction model: the remote sensing image and the initial segmentation are subjected to feature extraction through an IS feature extraction module to obtain multi-scale high-level semantic features and initial segmentation features; error estimation is performed on the multi-scale high-level semantic features and the initial segmentation features through a channel space attention module to obtain error estimation features and error estimation results; error correction is performed on the initial segmentation features, the error estimation features and the error estimation results through an error notification refinement module to obtain a segmentation repair result.

[0067] Specifically,

[0068] In step S1, appropriate remote sensing image data sets are selected, and the data is preprocessed, including image standardization, image cropping and label perturbation, etc., and is organized into a format convenient for subsequent processing; according to the task requirements, part of the data in the remote sensing image data set can be randomly selected as the training set, the verification set and the test set to ensure the diversity and representativeness of the data.

[0069] In this embodiment, the error pixel MaskEme is defined as the pixel consistency between the perturbed label and the ground truth, which is expressed as:

[0070]

[0071] Among them, is used to evaluate the consistency of the values of the i-th pixel in the perturbed label and the ground truth, and respectively represent the values of the i-th pixel in the perturbed label and the ground truth; if , it means that the perturbed label is accurate and does not need to be corrected, otherwise, the prediction is either under-segmentation (i.e. , or over-segmentation (i.e. ); the label perturbation process is performed online during training. In step S2, the remote sensing image segmentation error correction model includes an IS feature extraction module, a channel space attention module and an error notification refinement module.

[0072] The IS feature extraction module adopts HRNet-W48 as the backbone network, and the data processing process includes:

[0073] The remote sensing image and the initial segmentation are spliced and input into an Encoder-Decoder structure for feature extraction to obtain multi-scale features;

[0074] The remote sensing image and the initial segmentation are spliced and input into an Encoder-Decoder structure for feature extraction to obtain multi-scale features;

[0075] ​According to the multi-scale features, feature encoding is performed through a 1x1 convolutional layer, and sampling is performed to different scales to obtain multi-scale high-level semantic features;

[0076] According to the initial segmentation, feature encoding is performed through two 3x3 convolutional layers to obtain initial segmentation features;

[0077] The multi-scale features are up-sampled, and the sampling results are input into an auxiliary branch, and a classifier with an output channel of 2 is used to obtain an auxiliary branch segmentation result, which directly focuses on specific objects or regions in the input sample.

[0078] Reference Figure 2 As shown, the IS feature extractor includes image feature extraction (as a backbone) and initial segmentation feature extraction; first, the image feature extraction is used to extract the features of the objects or regions of interest in the image, and the input is the image and the corresponding initial segmentation result, and HRNet-W48 is used as the backbone, which is an Encoder-Decoder structure and removes the classifier; the multi-scale features of the backbone are first up-sampled to obtain sampling results, and the sampling results are input into the classifier of the auxiliary branch to obtain the auxiliary branch segmentation result; then, the features before being input into the classifier are connected, and a convolutional layer with a kernel of 1x1 is used to sample to different scales (1 / 4, 1 / 8, 1 / 16 and 1 / 32), which are called multi-scale and high-level semantic features; at the same time, two 3x3 convolutional layers are used for initial segmentation feature extraction, and the features of the initial segmentation are encoded to obtain initial features; the multi-scale and high-level semantic features and the initial features will be used for error estimation in the next stage, and the auxiliary branch segmentation result is used for calculating the auxiliary branch loss.

[0079] The auxiliary branch loss architecture of the embodiment is as follows: the output and the corresponding true value are used to calculate the cross-entropy (CE) loss and the Dice loss, wherein and are 0.5 and 0.5 respectively; the auxiliary branch loss is obtained by the following formula:

[0080]

[0081] wherein, and represent preset parameters, denotes a cross-entropy loss function, denotes a Dice loss function, denotes an auxiliary branch segmentation result, denotes a ground truth of the auxiliary branch segmentation result.

[0082] Assuming that the result output by the auxiliary branch is , the corresponding truth matrix (i.e. ) is , and by comparing the two, the elements in the red part of are segmented incorrectly.

[0083] The data processing procedure of the channel spatial attention module comprises:

[0084] The difference between the multi-scale high-level semantic feature and the initial segmentation feature is input into a channel attention module CAM: average response features and maximum response features in a spatial dimension are obtained through average pooling and maximum pooling operations; the average response features and the maximum response features in the spatial dimension are respectively sent into a shared multi-layer perceptron MLP to learn, to obtain two enhanced features; the two enhanced features are fused through an Add operation to obtain a fused feature; the frequency of each channel is calculated using a sigmoid function, and the fused feature is weighted according to the frequency to obtain an enhanced feature based on channel relationships;

[0085] The difference between the multi-scale high-level semantic feature and the initial segmentation feature is input into a spatial attention module SAM: average response features and maximum response features in a channel dimension are obtained through average pooling and maximum pooling operations; the average response features and the maximum response features in the channel dimension are spliced according to the channel dimension to obtain spliced features; the spliced features are convoluted using a 7X7 convolution layer, and are weighted using a sigmoid function to obtain an enhanced feature based on spatial relationships;

[0086] The enhanced feature based on channel relationships and the enhanced feature based on spatial relationships are fused to obtain an error estimation feature;

[0087] According to the error estimation feature, an output of 3 channels is mapped through a 1X1 convolution layer, respectively corresponding to correct segmentation pixels, under-segmented pixels and over-segmented pixels, to obtain an error estimation result.

[0088] This embodiment estimates errors (Error Estimator) through a channel spatial attention module, and when correcting errors in image segmentation, the image and the corresponding initial segmentation are analyzed from two scales of global and local:

[0089] ​Global scale: the network can easily find larger annotation errors, such as large-scale error regions;

[0090] Local scale: for smaller annotation errors, such as object boundaries or details, the network needs to carefully examine the local scale.

[0091] The channel spatial attention module CSAM is a module for enhancing multi-scale feature representation, aiming to improve the performance of the model in object detection and image segmentation; it includes a channel attention module (CAM) and a spatial attention module (SAM), which enhance features from the channel dimension and the spatial dimension respectively, ensuring that the model can capture key features at different scales; the multi-scale high-level semantic features obtained in the previous step and the initial segmentation features are subtracted at the same scale to obtain the result as the input feature of the CSAM module.

[0092] In the channel attention module (CAM), the input is the difference between the multi-scale high-level semantic features and the initial segmentation features; through average pooling (Average Pooling) and max pooling (Max Pooling) operations, CAM can capture the maximum response signal in the spatial dimension; then, a fully connected layer is used to learn and enhance these response signals, and finally the frequency of each channel is calculated through the sigmoid function, and these frequencies are multiplied with the input features to obtain enhancement information based on channel relationships.

[0093] In the spatial attention module (SAM), the input is the difference between the multi-scale high-level semantic features and the initial segmentation features, mainly focusing on the response signal in the spatial dimension; SAM uses different size windows (such as 1x1, 3x3, 7x7) to pool image features, extracting the most prominent features in the channel; then, a 1x1 convolution layer is used to further process these signals, and finally a sigmoid operation is used to weight the input features to enhance spatially related features.

[0094] The final output of CSAM is obtained by adding the enhanced features of CAM and SAM to the original input features; this process effectively strengthens the model's feature expression in different dimensions, improving the model's ability to understand complex information; the features processed by CSAM are called error estimator features, and through a 1x1 convolution layer, the input out_channels channels are mapped to 3-channel output, corresponding to correct segmentation pixels, under-segmented pixels and over-segmented pixels; the result is called error estimator prediction, which focuses on the areas where errors occur in the input sample.

[0095] The embodiment also comprises an error correction branch loss architecture as follows: Dice and CE loss are used to optimize the error estimation result The relationship between the error correction branch loss and its ground truth ; wherein and are 0.5 and 0.5 respectively; the error correction branch loss is obtained by the following formula:

[0096]

[0097] wherein, and represent preset parameters, represents the error estimation result, and represents the ground truth of the error estimation result.

[0098] Suppose the initial segmentation matrix of the current remote sensing image is , the error estimation result is , and the corresponding truth matrix (i.e. ) is , and , wherein 0 represents that the pixel at the position is correctly segmented, 1 represents that the pixel at the position is under-segmented, and 2 represents that the pixel at the position is over-segmented.

[0099] The error estimation result is used to evaluate the segmentation of each pixel point in the initial segmentation matrix, represents the correct evaluation of the segmentation of each pixel point in the initial segmentation matrix, and after comparison, the red parts are different, indicating that the error estimation result is wrong in evaluating the segmentation of the red part pixel value.

[0100] The data processing process of the error notification refinement module comprises:

[0101] The initial segmentation feature, the error estimation feature and the error estimation result are fused through a 1x1 convolution layer to obtain a fusion feature;

[0102] The number of channels of the error notification refinement module is reduced from 2c+3 to c, the fusion feature is extracted through a 3x3 convolution, and after a 5x5 depth separable convolution and a 1x1 convolution, a segmentation repair result is obtained.

[0103] The error notification refinement module is a refiner component designed to guide the correction of the prediction result in the error correction model through error information; it is implemented through an EGF module, and the input of the EGF module comprises:

[0104] Initial Feature: the output from the IS Feature Extraction module;

[0105] Error Estimator Feature: the feature after CSAM processing, representing the error or uncertainty produced in the prediction process;

[0106] Error Estimator Prediction: the error pixel prediction result obtained by CSAM, representing the error area or inaccurate area identified in the model prediction.

[0107] The EGF module first uses a 1x1 convolution layer to fuse the input features (initial segmentation feature, error estimation feature and error estimation result), and reduces the channel number from 2c+3 to c. Then a 3x3 convolution is used for feature extraction; this helps to extract more detailed information, which is then input to a 5x5 depthwise separable convolution and a 1x1 convolution to obtain the final segmentation repair result, which focuses on the global and local segmentation results of the image.

[0108] The embodiment also includes a repair branch loss architecture as follows: using a similar loss function as the auxiliary branch; using Dice and CE loss to optimize the segmentation repair result and its corresponding ground truth , where and are 0.5 and 0.5 respectively; the repair branch loss is obtained by the following formula:

[0109]

[0110] where, and represent preset parameters, and the segmentation repair result.

[0111] Assuming that the segmentation repair result is , the ground truth matrix (i.e. ) is , and the error estimation result is a further correction to the initial segmentation matrix; comparing the two matrices, the segmentation repair result is correct, and each pixel value is correctly segmented, which is the ideal state that the model hopes to achieve, and can correctly repair each error segmentation point.

[0112] The end-to-end training and optimization are adopted in the embodiment, the IS feature extraction module, the spatial and channel attention module and the error pixel correction module are integrated into the deep learning network for end-to-end training; in the training process, the network weight is updated by calculating the loss function and backward propagation, and the segmentation accuracy of the model is gradually optimized; the loss function is designed to consider the contribution of spatial information and channel information, so that the model can effectively balance between the two.

[0113] The overall loss of the remote sensing image segmentation error correction model includes an auxiliary branch loss , an error correction branch loss and a repair branch loss ; the overall loss of the remote sensing image segmentation error correction model is obtained by the following formula:

[0114]

[0115] Among them, , and represent preset parameters, which are 0.2, 0.4 and 0.4 respectively.

[0116] In the embodiment, WHU Building Dataset is selected for training and testing; the dataset provides high-resolution remote sensing image data and detailed location information of buildings in the image, and each image contains a large number of building labels, with high label quality and uniform image quality, which can well train and verify the building extraction and segmentation task in remote sensing images; the dataset includes the following parts:

[0117] (1) The dataset contains 7,152 images in total, and about 200,000 buildings are labeled. The ground resolution is 0.3 meters. The training set is a set of 4,736 images (512x512 pixel RGB images. The verification set is a set of 1036 images (512x512 pixel RGB images. The test set provides 2416 images (512x512 pixel RGB images);

[0118] (2) The label data is provided using two semantic classes at the pixel level, including buildings (white 255) and non-buildings (black 0);

[0119] (3) In order to enhance the generalization of the error correction model in any segmentation model, the initial segmentation of the existing model is avoided during the training process; different perturbations are applied to the label mask to simulate instance-level segmentation errors.

[0120] The embodiment selects all training set images from the data set for training and uses the validation set images for model evaluation; the remote sensing data set reads the image and label information, including the image name image_id, the image size image_size and the label category (building 255 and background 0); in the data loading stage, online data argumentation is adopted, including random rotation, cutting, shifting, scaling and horizontal and vertical flipping, to expand the training data set; at the same time, random removal of masks and segmentation of adjacent masks are used to simulate instance-level errors, and graph-based segmentation is also used to randomly add false positive instances; this method produces widely distributed potential segmentation errors, thereby achieving more robust training.

[0121] The training is implemented on an NVIDIA Tesla V100 GPU using the Pytorch framework, using the Adam optimizer, training running 25600 iterations, learning rate 0.000125, batch size 8; color and depth enhancement is applied to simulate real migration, label disturbance is Online_dataset in the training stage, and label disturbance is Offline_dataset in the verification and test stage; training, testing and verification all use 512x512 resolution RGB images; at the same time, the loss function is calculated, and the gradient of the model parameters is calculated, and the network weight is updated according to the gradient.

[0122] Figures 4-10 is the inference result of the embodiment on the test set, wherein Figure 4 is a remote sensing image, Figure 5 is a remote sensing image initial segmentation ground truth map, Figure 6 is a remote sensing image initial segmentation map after label disturbance, Figure 7 is a remote sensing image auxiliary branch output map, Figure 8 is a remote sensing image error estimation result map, Figure 9 is a remote sensing image error estimation result ground truth map, Figure 10 is a remote sensing image segmentation repair result map; the error notification refinement module can repair the initial segmentation of Figure 8 according to the error estimation result of Figure 6 to achieve a ground truth Figure 5 closer to .

[0123] After the network training is completed, the network weight is kept in the pth file, and the pth file can directly correct and repair the initial segmentation result of any input model, and the final result is output after visualization.

[0124] In summary, the remote sensing image segmentation correction method proposed in this embodiment solves the problems of feature loss and insufficient information in existing methods by fully combining spatial and channel attention mechanisms. Unlike traditional post-processing correction methods, it can be trained end-to-end and does not rely on complex post-processing steps, improving processing efficiency and segmentation accuracy. The advantages include:

[0125] (1) Improve segmentation accuracy and detail capture ability: By combining spatial and channel attention mechanisms, the model's ability to represent features of different scales is effectively enhanced. Spatial attention mechanism can focus on key areas in the image, especially when dealing with boundaries and details, while channel attention mechanism can fully exploit the correlation between multi-spectral channels of remote sensing images. This multi-dimensional information fusion greatly improves the model's performance in complex regions and detail capture, making the segmentation result more accurate, especially in challenging environments with noise, occlusion, and shadows, significantly improving segmentation accuracy.

[0126] (2) End-to-end training and efficient optimization: This method is optimized entirely through end-to-end training, avoiding the high computational complexity and difficulty in seamlessly integrating with deep learning models that are common in traditional post-processing methods such as Conditional Random Fields. Through the network architecture combining spatial and channel attention mechanisms, the model can adaptively adjust feature extraction and error correction during training, ultimately achieving efficient and accurate segmentation results. This end-to-end training method makes the model training process more convenient and adaptable, and can handle large-scale remote sensing image data, meeting the needs of different application scenarios.

[0127] (3) Dynamic correction of segmentation errors and enhanced robustness: Through error pixel prediction and dynamic correction strategies, errors in the initial segmentation results can be identified and corrected in a timely manner, avoiding the accumulation of errors during segmentation. The Error Guided Refinement module (EGF) accurately identifies inconsistent pixel regions by fusing error estimation features, error estimation results, and image features, and effectively repairs them. This mechanism not only improves segmentation accuracy but also maintains high robustness when processing complex backgrounds and detailed remote sensing images, ensuring the stability and consistency of the segmentation results.

[0128] (4) No need for complex post-processing, improve processing efficiency: Compared with traditional image segmentation methods, this method does not rely on complex post-processing steps, but directly optimizes the segmentation results through error estimation and correction mechanisms within the model. This simplified processing flow not only improves computational efficiency but also reduces the need for human intervention, making it suitable for high automation and strict processing speed requirements in remote sensing image processing tasks. In addition, the end-to-end training method reduces the delay of the model during inference, making real-time or large-scale image segmentation possible.

[0129] (5) Wide applicability and scalability: The method is not only suitable for remote sensing image segmentation tasks, but also has strong universality and scalability; the error correction mechanism combining spatial and channel attention can be applied to other image segmentation tasks in other fields, such as medical image segmentation, satellite image analysis, autonomous driving, etc.; due to its efficient feature expression and dynamic correction ability, the technology can flexibly adapt to various complex scenes, provide accurate segmentation results, and has wide practical application value.

[0130] Embodiment two:

[0131] Based on the same inventive concept as embodiment one, the present embodiment discloses a remote sensing image segmentation error correction system, comprising:

[0132] A preprocessing module is configured to perform data preprocessing on the obtained remote sensing image data to obtain preprocessed data, including a remote sensing image and an initial segmentation.

[0133] An IS feature extraction module is configured to perform feature extraction on the remote sensing image and the initial segmentation to obtain multi-scale high-level semantic features and initial segmentation features.

[0134] A channel spatial attention module is configured to perform error estimation on the multi-scale high-level semantic features and the initial segmentation features to obtain error estimation features and error estimation results.

[0135] An error notification refinement module is configured to perform error correction based on the initial segmentation features, the error estimation features, and the error estimation results to obtain a segmentation repair result.

[0136] The specific function implementation of each module is referred to the related content in the method of embodiment one, and will not be repeated here.

[0137] Embodiment three:

[0138] The present embodiment provides a computer readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps of the remote sensing image segmentation error correction method of any one of embodiments one.

[0139] Embodiment four:

[0140] The present embodiment provides a computer device / system, comprising:

[0141] A memory is configured to store a computer program / instruction.

[0142] A processor is configured to execute the computer program / instruction to implement the steps of the remote sensing image segmentation error correction method of any one of the first aspect.

[0143] Embodiment five:

[0144] The embodiments provide a computer program product comprising computer programs / instructions, characterized in that the computer programs / instructions, when executed by a processor, implement the steps of the remote sensing image segmentation error correction method according to any one of the embodiments.

[0145] Those skilled in the art will understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) having computer usable program code embodied therein.

[0146] The present application is described with reference to the flowcharts and / or block diagrams of the methods, apparatus (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions, which are executed via the processor of the computer or other programmable data processing apparatus, generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the functions specified in the flowcharts and / or block diagrams.

[0147] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction means, which implement the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the functions specified in the flowcharts and / or block diagrams.

[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 means for carrying out the functions specified in the flowcharts and / or block diagrams.

[0149] The embodiments of the present application are described above with reference to the accompanying drawings, but the present application is not limited to the above-described specific embodiments, and the above-described specific embodiments are merely illustrative, but not restrictive, and a person of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims, and these all belong to the protection of the present application.

Claims

1. A method for segmenting and correcting errors in remote sensing images, characterized in that, The method comprises the following steps: data preprocessing is performed on the obtained remote sensing image data to obtain a remote sensing image and an initial segmentation; the remote sensing image and the initial segmentation are input into a trained remote sensing image segmentation error correction model: feature extraction is performed on the remote sensing image and the initial segmentation by an IS feature extraction module to obtain multi-scale high-level semantic features and initial segmentation features; error estimation is performed on the multi-scale high-level semantic features and the initial segmentation features by a channel-spatial attention module to obtain error estimation features and error estimation results; error correction is performed on the initial segmentation features, the error estimation features and the error estimation results by an error notification refinement module to obtain a segmentation repair result; the data processing procedure of the channel-spatial attention module comprises: the difference between the multi-scale high-level semantic features and the initial segmentation features is input into a channel attention module CAM: average response features and maximum response features in the spatial dimension are obtained through average pooling and maximum pooling operations; the average response features and the maximum response features in the spatial dimension are respectively input into a shared multi-layer perception MLP to learn, and two enhanced features are obtained; the two enhanced features are fused through an Add operation to obtain a fused feature; the frequency of each channel is calculated by using a sigmoid function, and the fused feature is weighted according to the frequency to obtain an enhanced feature based on channel relationship; the difference between the multi-scale high-level semantic features and the initial segmentation features is input into a spatial attention module SAM: average response features and maximum response features in the channel dimension are obtained through average pooling and maximum pooling operations; the average response features and the maximum response features in the channel dimension are spliced according to the channel dimension to obtain spliced features; the spliced features are convoluted by using a 7X7 convolution layer, and are weighted by using a sigmoid function to obtain an enhanced feature based on spatial relationship; the enhanced feature based on channel relationship and the enhanced feature based on spatial relationship are fused to obtain the error estimation features; the error estimation features are mapped to the output of 3 channels corresponding to correct segmentation pixels, under-segmentation pixels and over-segmentation pixels through a 1x1 convolution layer to obtain the error estimation results.

2. The method of claim 1, wherein, The data preprocessing comprises image standardization, image cropping and label disturbance.

3. The method of claim 1, wherein, The data processing procedure of the IS feature extraction module comprises: the remote sensing image and the initial segmentation are spliced and input into an Encoder-Decoder structure for feature extraction to obtain multi-scale features; the multi-scale features are encoded by a 1x1 convolution layer and are sampled to different scales to obtain multi-scale high-level semantic features; the initial segmentation is encoded by two 3x3 convolution layers to obtain initial segmentation features; the multi-scale features are up-sampled, and the sampling results are input into an auxiliary branch, and a classifier with an output channel of 2 is used to obtain an auxiliary branch segmentation result.

4. The method of claim 1, wherein, The data processing procedure of the error notification refinement module comprises: The initial segmentation feature, the error estimation feature and the error estimation result are fused through a 1*1 convolution layer to obtain a fused feature; The number of channels of the error notification refinement module is reduced from 2c+3 to c, feature extraction is performed on the fused feature through a 3*3 convolution, and a segmentation repair result is obtained through a 5*5 depth separable convolution and a 1*1 convolution.

5. The method of claim 3, wherein, The overall loss of the remote sensing image segmentation error correction model includes an auxiliary branch loss , an error correction branch loss , and a repair branch loss ; the auxiliary branch loss is obtained by the following equation: ， wherein, and denotes a preset parameter, denotes a cross-entropy loss function, denotes a Dice loss function, denotes an auxiliary branch segmentation result, denotes a ground truth of the auxiliary branch segmentation result; The error correction branch loss is obtained by the following equation: ， wherein, and denotes a preset parameter, denotes an error estimation result, denotes a ground truth of the error estimation result; The repair branch loss is obtained by the following equation: ， wherein, and denotes a preset parameter, denotes a segmentation repair result; The overall loss of the remote sensing image segmentation error correction model is obtained by the following formula: ， wherein, , and denote preset parameters.

6. A remote sensing image segmentation and error correction system, characterized in that, It comprises: a preprocessing module configured to perform data preprocessing on the obtained remote sensing image data to obtain preprocessed data including a remote sensing image and an initial segmentation; an IS feature extraction module configured to perform feature extraction on the remote sensing image and the initial segmentation to obtain multi-scale high-level semantic features and initial segmentation features; a channel spatial attention module configured to perform error estimation on the multi-scale high-level semantic features and the initial segmentation features to obtain error estimation features and error estimation results; an error notification refinement module configured to perform error correction according to the initial segmentation features, the error estimation features and the error estimation results to obtain a segmentation repair result; The data processing procedure of the channel spatial attention module comprises: the difference between the multi-scale high-level semantic features and the initial segmentation features is input into a channel attention module CAM: average response features and maximum response features in a spatial dimension are obtained through average pooling and maximum pooling operations; the average response features and the maximum response features in the spatial dimension are respectively input into a shared multi-layer perception MLP to learn two enhanced features; the two enhanced features are fused through an Add operation to obtain a fused feature; the frequency of each channel is calculated using a sigmoid function, and the fused feature is weighted according to the frequency to obtain an enhanced feature based on channel relationship; the difference between the multi-scale high-level semantic features and the initial segmentation features is input into a spatial attention module SAM: average response features and maximum response features in a channel dimension are obtained through average pooling and maximum pooling operations; the average response features and the maximum response features in the channel dimension are spliced according to the channel dimension to obtain spliced features; the spliced features are convoluted using a 7*7 convolution layer, and are weighted using a sigmoid function to obtain an enhanced feature based on spatial relationship; the enhanced feature based on channel relationship and the enhanced feature based on spatial relationship are fused to obtain an error estimation feature; the error estimation feature is mapped to 3-channel outputs corresponding to correct segmentation pixels, under-segmented pixels and over-segmented pixels through a 1*1 convolution layer to obtain an error estimation result.

7. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, The computer program / instructions are executed by the processor to implement the steps of the remote sensing image segmentation error correction method of any one of claims 1-5.

8. A computer device, comprising: It comprises: a memory configured to store computer programs / instructions; a processor configured to execute the computer programs / instructions to implement the steps of the remote sensing image segmentation error correction method of any one of claims 1-5.

9. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions are executed by the processor to implement the steps of the remote sensing image segmentation error correction method of any one of claims 1-5.

Images