Remote sensing image segmentation method based on local consistency and pseudo-label filtering

Abstract

The present invention discloses a remote sensing image segmentation method based on local consistency and pseudo-label filtering, comprising the following steps: obtaining an optical remote sensing dataset; constructing a remote sensing image segmentation model, wherein the remote sensing image segmentation model includes a local consistency loss calculation module and a pseudo-label filtering module; and inputting the optical remote sensing dataset into the remote sensing image segmentation model to obtain a remote sensing image segmentation result map. A local consistency loss and pseudo-label filtering strategy is proposed, which effectively learns local consistency features, removes pseudo-label noise, and achieves excellent remote sensing image segmentation results. Furthermore, more unlabeled remote sensing images are utilized to improve remote sensing image segmentation accuracy. Remote sensing images acquired by satellites can be segmented to obtain a segmentation map with specific ground object category labels. This solves the problems of densely populated objects, large background differences, wide size variations, large color and texture differences, difficulty in obtaining local consistency features, and the inevitable presence of large amounts of noise in pseudo-labels in remote sensing images.

Description

Technical Field

[0001] The present invention belongs to the technical field of image segmentation, and in particular relates to a remote sensing image segmentation method based on local consistency and pseudo-label filtering. Background Art

[0002] Remote sensing image segmentation is the process and technology of pixel-level classification of remote sensing images, assigning each pixel in a remote sensing image to a specific class label. Supervised learning-based image segmentation models have achieved remarkable success in natural image and remote sensing image interpretation by leveraging sufficient amounts of accurate, manually labeled data. However, in some practical applications, high-quality labeled data is difficult to obtain, while large amounts of raw data are readily available, especially for high-resolution remote sensing imagery. This consumes significant resources, requiring high-level expertise and labor costs for labelers. To alleviate this problem, a natural approach is to leverage large amounts of raw data to assist model learning when only a small amount of labeled data is available, a strategy known as semi-supervised learning. Existing SSL methods mostly focus on image classification tasks and can be categorized into two main strategies: self-training (ST) and consistency regularization (CR). ST improves the model by expanding the training set with high-quality pseudo-labels from unlabeled data at a predefined confidence threshold and then retraining. CR improves SSL performance by enforcing model predictions to remain consistent across various perturbations, including input, feature, or network perturbations. Another popular SSL family adopts strong data augmentation (SDA) to enhance the robustness and generalization of the model, leading to better performance.

[0003] Although semi-supervised image segmentation has made significant progress in the field of natural image interpretation, there are few studies on semi-supervised RS image segmentation. Among them, the paper Simple and Efficient: ASemisupervised Learning Framework for Remote Sensing Image Semantic Segmentation published in the 2022 TGARS journal provides a strong baseline for the field of semi-supervised image segmentation and proposes a two-stage teacher-student network LSST with sample-level category-related adaptive thresholds. The paper Self Pseudo Entropy Knowledge Distillation for Semi-supervised Semantic Segmentation published in the 2023 TGARS journal proposed an effective framework based on consistency learning, designed a novel unsupervised data enhancement strategy EntropyMix, and proposed self-pseudo entropy knowledge distillation (SPEED) to optimize the linear combination of feature and logit-level distillation losses, learn global pixel relationships from multi-view and cross-view perturbations, and thus improve model performance without the need for additional auxiliary segmentation heads or complex pre-trained teacher models.

[0004] However, these methods don't address the challenges of obtaining locally consistent features in remote sensing images, which often involve densely packed objects, diverse backgrounds, large size variations, and significant color and texture variations. Furthermore, noise filtering in pseudo-labeling is performed solely based on category or overall features, without considering the edge information and color morphology of unlabeled data. Summary of the Invention

[0005] The purpose of the present invention is to overcome the problems that it is difficult to obtain local consistency features in remote sensing images due to dense objects and large background differences, and there is a lot of noise in pseudo labels. A remote sensing image segmentation method based on local consistency and pseudo label filtering is proposed.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a remote sensing image segmentation method based on local consistency and pseudo-label filtering, comprising the following steps:

[0008] Acquisition of optical remote sensing datasets;

[0009] Constructing a remote sensing image segmentation model, wherein the remote sensing image segmentation model includes a local consistency loss calculation module and a pseudo label filtering module;

[0010] The optical remote sensing dataset is input into the remote sensing image segmentation model to obtain the remote sensing image segmentation result map.

[0011] Furthermore, the obtaining of the optical remote sensing data set includes the following steps: obtaining a remote sensing image, annotating the remote sensing image to obtain labeled data and unlabeled data, and constructing an optical remote sensing data set using the labeled data and the unlabeled data; the remote sensing image is a visible light modality remote sensing image.

[0012] Furthermore, the step of inputting the optical remote sensing data set into the remote sensing image segmentation model to obtain the remote sensing image segmentation result graph comprises the following steps:

[0013] Divide the labeled data into training set, validation set and test set;

[0014] The unlabeled data is processed through the remote sensing segmentation network and the pseudo-label filtering module to generate pseudo-labels to obtain unlabeled data with pseudo-labels;

[0015] Enhance the training set to obtain enhanced labeled data;

[0016] Use pseudo-labeled unlabeled data and enhanced labeled data to perform supervised training on the remote sensing segmentation network to generate a remote sensing image segmentation model;

[0017] The validation set is input into the remote sensing image segmentation model for model validation. The task types of model validation include model evaluation tasks and model reasoning tasks. The remote sensing image segmentation result map is obtained through the model reasoning task.

[0018] Furthermore, the remote sensing segmentation network uses Swin Transformer Base as the encoder and UperNetHead as the decoder, and the encoder is initialized using ImageNet weights.

[0019] Furthermore, the unlabeled data is subjected to a remote sensing segmentation network and a pseudo-label filtering module to generate pseudo-labels to obtain unlabeled data with pseudo-labels, which includes the following steps:

[0020] Input unlabeled data into the remote sensing segmentation network to generate rough pseudo labels;

[0021] The coarse pseudo labels are filtered by the pseudo label filtering module to obtain pseudo labels;

[0022] The coarse pseudo labels are filtered by the pseudo label filtering module to obtain pseudo labels, specifically:

[0023] Input the training set into Swin Transformer Base to obtain feature maps, and cluster the feature maps to obtain clustered images;

[0024] Add the clustered images and the rough pseudo-labels to get the cluster blocks, and calculate the entropy value of each cluster block;

[0025] The entropy value of each cluster block is calculated using the following formula:

[0026]

[0027] in, It is a segmentation network For the first The softmax probability value generated by an unlabeled image at pixel (h, w), It is Victoria , The higher the value, the higher the uncertainty of the model prediction, which means sorting the entropy map pixel by pixel in ascending order;

[0028] All cluster blocks are sorted according to the entropy value, and the top 5% of cluster blocks are deleted to obtain the remaining cluster blocks after removing the high entropy value cluster blocks;

[0029] The boundary information of the image is obtained through the Canny operator, and the category of each area within the boundary is the maximum value of the accumulated pixel point categories to obtain the area category determined by boundary detection;

[0030] The pseudo labels are obtained by combining the remaining cluster blocks after removing the high entropy cluster blocks and the region categories determined by boundary detection;

[0031] The step of enhancing the training set to obtain enhanced labeled data includes the following steps:

[0032] The training set is randomly scaled, randomly cropped, and randomly flipped horizontally before being subjected to color conversion and strong enhancement to obtain enhanced labeled data. The scale of the random scaling is between 0.5 and 2, where 0.5 represents reduction to 50% of the original size and 2 represents enlargement to 200% of the original size.

[0033] Furthermore, the supervised training is specifically as follows:

[0034] Calculate the loss according to the cross entropy loss calculation formula and the local consistency contrast loss calculation formula;

[0035] The calculation formula of the cross entropy loss is as follows:

[0036]

[0037] in, is the category predicted by the pseudo label, is the category of the true label;

[0038] The calculation formula of the local consistency contrast loss is as follows:

[0039]

[0040] in, represents the local consistency contrast loss, represents mutual information, represents the Shannon entropy,) is given hour The conditional entropy of is the number of ROIs in a patch, There is a corresponding ROI in the label enhancement image, It is the corresponding ROI of the label map;

[0041] The network weights are optimized and updated according to the loss, using the AdamW optimizer and the Step learning rate strategy. The learning rate for the first two-thirds of the training rounds is , the learning rate for the last one-third of the training rounds is ;

[0042] Determine whether the number of Epochs of the current supervised training has reached the preset number of epochs. If so, save the model parameters and end the training. If not, enter the verification phase and set Epoch=Epoch+1.

[0043] Furthermore, the inputting of the validation set into the remote sensing image segmentation model for model validation is specifically as follows:

[0044] After normalizing the remote sensing image, it is fed into the remote sensing image segmentation model for calculation to obtain the pixel-by-pixel classification results. The pixel-by-pixel classification result presentation method is selected according to the task type of model verification.

[0045] If it is a model inference task, a different color is assigned to each category based on the pixel-by-pixel classification results to obtain a segmentation result map;

[0046] If it is a model evaluation task, the intersection of the true value and the predicted value is calculated based on the pixel-by-pixel classification result and the read true classification. The model evaluation index of this round is compared with the previous highest evaluation index. If it is greater than the highest index, the weight is updated to the model parameter of this round. Otherwise, it enters the next round of supervised training.

[0047] In a second aspect, the present invention provides a remote sensing image segmentation system based on local consistency and pseudo-label filtering, comprising:

[0048] Dataset acquisition module, used to acquire optical remote sensing datasets;

[0049] A segmentation model construction module is used to construct a remote sensing image segmentation model, wherein the remote sensing image segmentation model includes a local consistency loss calculation module and a pseudo label filtering module;

[0050] The segmentation result obtaining module is used to input the optical remote sensing data set into the remote sensing image segmentation model to obtain the remote sensing image segmentation result map.

[0051] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the remote sensing image segmentation method based on local consistency and pseudo-label filtering when executing the computer program.

[0052] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the remote sensing image segmentation method based on local consistency and pseudo-label filtering is implemented.

[0053] Compared with the prior art, the present invention has the following beneficial technical effects:

[0054] The remote sensing image segmentation method based on local consistency and pseudo-label filtering proposed in the present invention proposes a local consistency loss and pseudo-label filtering strategy, effectively learns local consistency features, removes the noise of pseudo-labels, and obtains good remote sensing image segmentation effect. It uses more unlabeled remote sensing images to improve the accuracy of remote sensing image segmentation, and can segment remote sensing images acquired by satellites to obtain segmentation maps with specific ground object category labels, solving the problems of remote sensing images with dense objects, large background differences, large size variation range, large color and texture differences, difficulty in obtaining local consistency features, and the inevitable presence of a large amount of noise in pseudo-labels.

[0055] Furthermore, to address the problems of dense objects and large background differences in RS images, the present invention adopts a semi-supervised segmentation method based on local consistency and pseudo-label filtering in the field of remote sensing single-modal image segmentation, and uses local consistency loss to extract local semantic details in remote sensing images; to address the problem of a large amount of noise in pseudo-labels, a pseudo-label filtering module is proposed, which clusters feature maps and sorts them by entropy values, and can adaptively remove noise with inconsistent color and morphology in the labels; and uses the Canny operator to extract image boundaries, remove boundary noise, and effectively remove noise in pseudo-labels, thereby better obtaining segmentation result maps. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The drawings described herein are for illustrative purposes only and are not intended to limit the scope of the present invention in any way. In addition, the shapes and proportional dimensions of the components in the drawings are only schematic and are used to help understand the present invention, and are not intended to specifically limit the shapes and proportional dimensions of the components of the present invention. In the drawings:

[0057] Figure 1 This is a flow chart of the remote sensing image segmentation method based on local consistency and pseudo-label filtering of the present invention.

[0058] Figure 2 This is a structural diagram of the remote sensing image segmentation system based on local consistency and pseudo-label filtering of the present invention.

[0059] Figure 3 This is a flow chart of the remote sensing image segmentation method based on local consistency and pseudo-label filtering of the present invention.

[0060] Figure 4 This is a flowchart of a remote sensing image segmentation method based on local consistency and pseudo-label filtering provided in an embodiment of the present invention.

[0061] Figure 5 This is the pseudo-label filtering module in the embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0063] Example 1

[0064] See also Figure 1 , a remote sensing image segmentation method based on local consistency and pseudo label filtering, comprising the following steps:

[0065] Acquisition of optical remote sensing datasets;

[0066] Constructing a remote sensing image segmentation model, wherein the remote sensing image segmentation model includes a local consistency loss calculation module and a pseudo label filtering module;

[0067] The optical remote sensing dataset is input into the remote sensing image segmentation model to obtain the remote sensing image segmentation result map.

[0068] The remote sensing image segmentation method based on local consistency and pseudo-label filtering provided in this embodiment improves the automation level of remote sensing image segmentation by introducing technical means such as local consistency and pseudo-label filtering. Local consistency loss calculation helps the model more flexibly adjust local features during training, reducing overfitting and improving the model's generalization ability. Remote sensing images often contain complex surface coverage and diverse object categories. The combination of local consistency and pseudo-label filtering helps the model better adapt to these complex scenarios. By calculating local consistency loss, the model can learn the characteristic relationships between local regions in the image, thereby more accurately identifying different object categories. Pseudo-labeling technology uses a model trained on labeled data to make predictions on unlabeled data and filter samples based on the prediction results. The pseudo-label filtering module can identify and remove pseudo-labels that have unreliable or noisy predictions, thereby preventing these erroneous labels from negatively impacting model training. By filtering out poor-quality pseudo-labels, the model can be trained on more accurate data, further improving segmentation accuracy. Pseudo-labeling technology enables the model to be trained on large amounts of unlabeled data, thereby expanding the size of the training dataset. The pseudo-label filtering module allows the model to filter valuable samples from this unlabeled data for training, improving training efficiency. The combination of the local consistency loss calculation module and the pseudo-label filtering module helps the model converge to the optimal solution more quickly. By optimizing local features and removing noisy labels, the model can achieve high segmentation accuracy in fewer iterations.

[0069] Acquiring an optical remote sensing dataset includes the following steps: acquiring a remote sensing image, annotating the remote sensing image to obtain labeled data and unlabeled data, and constructing an optical remote sensing dataset using the labeled data and unlabeled data; the remote sensing image is a visible light modality remote sensing image.

[0070] The visible light remote sensing images in this embodiment can directly reflect the shape, color, texture, and other characteristics of surface objects. These characteristics are clarified and enhanced during the annotation process, thereby improving data accuracy. The annotation process helps highlight key information in remote sensing images, making the data more intuitive and easy to understand. By acquiring visible light remote sensing images at different times, locations, and conditions, it is possible to construct a dataset that contains rich diversity and variability, which is crucial for improving the generalization ability of the model.

[0071] Inputting the optical remote sensing dataset into the remote sensing image segmentation model to obtain the remote sensing image segmentation result graph includes the following steps:

[0072] Divide the labeled data into training set, validation set and test set;

[0073] The unlabeled data is processed through the remote sensing segmentation network and the pseudo-label filtering module to generate pseudo-labels to obtain unlabeled data with pseudo-labels;

[0074] Enhance the training set to obtain enhanced labeled data;

[0075] Use pseudo-labeled unlabeled data and enhanced labeled data to perform supervised training on the remote sensing segmentation network to generate a remote sensing image segmentation model;

[0076] The validation set is input into the remote sensing image segmentation model for model validation. The task types of model validation include model evaluation tasks and model reasoning tasks. The remote sensing image segmentation result map is obtained through the model reasoning task.

[0077] By dividing the labeled data into training, validation, and test sets, we can ensure that the model is fully learned during training while accurately evaluating its performance during validation and testing. This division helps avoid overfitting and improves the model's generalization ability.

[0078] This embodiment enhances the training set to increase data diversity and richness, enabling the model to learn more features and patterns. This helps improve the model's recognition capabilities in complex scenarios and enhances its robustness. Through the remote sensing segmentation network and pseudo-label filtering module, unlabeled data can be assigned pseudo-labels, thereby converting them into pseudo-labeled unlabeled data. While these pseudo-labels are not as accurate as manual annotation, they still provide valuable information for model training. Using pseudo-labeled unlabeled data and enhanced labeled data for supervised training of the remote sensing segmentation network can fully utilize the information in the unlabeled data and further improve model performance. This semi-supervised learning method is particularly effective when labeled data is scarce. During the model validation phase, model evaluation tasks can accurately assess model performance, helping to understand the model's performance in different scenarios and providing a basis for model optimization and improvement. Model inference tasks can generate remote sensing image segmentation result graphs. These result graphs can intuitively demonstrate the model's segmentation results, providing important foundational data for subsequent remote sensing applications. Furthermore, model inference tasks can help identify potential problems and deficiencies in the model's segmentation process, providing guidance for further model optimization.

[0079] The remote sensing segmentation network uses Swin Transformer Base as the encoder and UperNetHead as the decoder, and the encoder is initialized using ImageNet weights.

[0080] Swin Transformer Base is a deep learning model based on the Transformer architecture, particularly suitable for processing high-resolution remote sensing imagery. Using a spatial self-attention mechanism, it adaptively acquires information from remote sensing images while effectively integrating and modeling spatial information. This not only retains the Transformer's advantage in long-range dependencies but also avoids memory limitations when computing large images. Swin Transformer Base captures details at different levels of the image through the fusion of windows and features at different scales. This multi-scale feature representation capability makes the model more flexible and accurate when processing complex remote sensing imagery. Swin Transformer Base boasts high computational efficiency, ensuring accuracy while reducing memory and computing resource requirements, which is crucial for processing large-scale remote sensing image data.

[0081] UperNetHead is commonly used for image segmentation tasks. Through its symmetrical encoder-decoder structure and skip connections, it can achieve accurate segmentation of remote sensing images. Skip connections help capture multi-scale information, thereby improving the accuracy of image segmentation. UperNetHead can capture contextual information in the image, which is crucial for remote sensing image segmentation tasks. Contextual information helps the model understand the image content more accurately, thereby improving the accuracy of the segmentation results.

[0082] Using ImageNet weights to initialize the encoder can utilize the model weights pre-trained on the ImageNet dataset to accelerate the model training process, which helps the model converge to the optimal solution faster. The ImageNet dataset contains a large number of natural images, which have certain similarities with remote sensing images in structure and texture. Therefore, using ImageNet weights for initialization can enable the encoder to have better generalization ability when processing remote sensing images, thereby improving the performance of the model. Using pre-trained weights for initialization can also reduce the risk of overfitting of the model during training. This is because the pre-trained weights have learned a large number of image features and patterns, which are also useful for new remote sensing image segmentation tasks.

[0083] The unlabeled data is processed through the remote sensing segmentation network and the pseudo-label filtering module to generate pseudo-labels to obtain unlabeled data with pseudo-labels, including the following steps:

[0084] Input unlabeled data into the remote sensing segmentation network to generate rough pseudo labels;

[0085] The coarse pseudo labels are filtered by the pseudo label filtering module to obtain pseudo labels;

[0086] The coarse pseudo labels are filtered by the pseudo label filtering module to obtain pseudo labels, specifically:

[0087] Input the training set into Swin Transformer Base to obtain feature maps, and cluster the feature maps to obtain clustered images;

[0088] Add the clustered images and the rough pseudo-labels to get the cluster blocks, and calculate the entropy value of each cluster block;

[0089] The entropy value of each cluster block is calculated using the following formula:

[0090]

[0091] in, It is a segmentation network For the first The softmax probability value generated by an unlabeled image at pixel (h, w), It is Victoria , The higher the value, the higher the uncertainty of the model prediction, which means sorting the entropy map pixel by pixel in ascending order;

[0092] All cluster blocks are sorted according to the entropy value, and the top 5% of cluster blocks are deleted to obtain the remaining cluster blocks after removing the high entropy value cluster blocks;

[0093] The boundary information of the image is obtained through the Canny operator, and the category of each area within the boundary is the maximum value of the accumulated pixel point categories to obtain the area category determined by boundary detection;

[0094] The pseudo labels are obtained by combining the remaining cluster blocks after removing the high entropy cluster blocks and the region categories determined by boundary detection;

[0095] The training set is enhanced to obtain the enhanced labeled data, including the following steps:

[0096] The training set is randomly scaled, randomly cropped, and randomly flipped horizontally before being subjected to color conversion and strong enhancement to obtain the enhanced labeled data. The scale of random scaling is between 0.5 and 2, where 0.5 means reducing it to 50% of the original size and 2 means enlarging it to 200% of the original size.

[0097] By generating pseudo-labels for unlabeled data, this embodiment can fully utilize these unlabeled data resources, expand the size of the training set, and thus improve the model's generalization ability. During the pseudo-label generation process, clustering, entropy calculation, and boundary detection are used to filter out areas with high model prediction uncertainty, thereby improving the quality of the pseudo-labels. Using high-quality pseudo-labels for training can further enhance the performance of remote sensing segmentation networks, making them more accurate and efficient in processing remote sensing images. Enhancements to the training set, such as random scaling, random cropping, random horizontal flipping, and color conversion, can increase data diversity and improve the model's adaptability to various image transformations. This helps improve the model's robustness when processing complex and variable remote sensing images. Traditional supervised learning methods require a large amount of labeled data, which is often labor-intensive and time-consuming. Generating pseudo-labels can reduce the reliance on labeled data to a certain extent, thereby lowering the cost of data annotation. This process combines elements of unsupervised learning (such as clustering) and supervised learning (such as training with pseudo-labels), embodying the principles of semi-supervised learning and self-supervised learning.

[0098] Supervised training is as follows:

[0099] Calculate the loss according to the cross entropy loss calculation formula and the local consistency contrast loss calculation formula;

[0100] The calculation formula of cross entropy loss is as follows:

[0101]

[0102] in, is the category predicted by the pseudo label, is the category of the true label;

[0103] The calculation formula of local consistency contrast loss is as follows:

[0104]

[0105] in, represents the local consistency contrast loss, represents mutual information, represents the Shannon entropy,) is given hour The conditional entropy of is the number of ROIs in a patch, There is a corresponding ROI in the label enhancement image, It is the corresponding ROI of the label map;

[0106] The network weights are optimized and updated according to the loss, using the AdamW optimizer and the Step learning rate strategy. The learning rate for the first two-thirds of the training rounds is , the learning rate for the last one-third of the training rounds is ;

[0107] Determine whether the number of Epochs of the current supervised training has reached the preset number of epochs. If so, save the model parameters and end the training. If not, enter the verification phase and set Epoch=Epoch+1.

[0108] The cross-entropy loss function is a common method for measuring the difference between two probability distributions, particularly well-suited for processing one-hot encoded labels. By minimizing the cross-entropy between the predicted and actual distributions, the model's classification performance can be optimized, enabling it to more accurately learn the differences between categories.

[0109] Local consistency contrast loss focuses on the similarities and differences between data samples, which helps the model learn more discriminative feature representations, thereby improving the accuracy of the model.

[0110] The AdamW optimizer in this embodiment combines the advantages of the momentum method and the RMSProp algorithm to achieve the goal of dynamically adjusting the learning rate. By calculating the first-order moment estimate and the second-order moment estimate of the gradient, the learning rate is adaptively adjusted for each parameter, thereby performing well on problems with sparse gradients or noise. This helps to speed up the training of the model and improve training efficiency. The Step learning rate strategy dynamically adjusts the learning rate according to the number of training rounds, which helps the model converge quickly in the early stages of training while maintaining stability in the later stages of training to avoid overfitting. Through supervised training, the model can learn more data patterns and features, thereby enhancing the generalization ability of the model. When faced with new and unseen data, the model can better predict and classify. Local consistency contrast loss also helps the model understand and learn abstract concepts, explore deep representations in the data, and further improve the generalization ability of the model. The loss value calculated based on the cross entropy loss and local consistency contrast loss can be used to optimize and update the network weights. This helps the model continuously adjust parameters during training to better fit the data distribution and characteristics. By setting the number of training epochs, you can clearly define the end conditions of the training process. When the preset number of epochs is reached, the model parameters are saved and the training ends, which helps avoid overfitting problems caused by excessive training.

[0111] The validation set is input into the remote sensing image segmentation model for model validation as follows:

[0112] After normalizing the remote sensing image, it is fed into the remote sensing image segmentation model for calculation to obtain the pixel-by-pixel classification results. The pixel-by-pixel classification result presentation method is selected according to the task type of model verification.

[0113] If it is a model inference task, a different color is assigned to each category based on the pixel-by-pixel classification results to obtain a segmentation result map;

[0114] If it is a model evaluation task, the intersection of the true value and the predicted value is calculated based on the pixel-by-pixel classification result and the read true classification. The model evaluation index of this round is compared with the previous highest evaluation index. If it is greater than the highest index, the weight is updated to the model parameter of this round. Otherwise, it enters the next round of supervised training.

[0115] This embodiment uses a validation set as input to the remote sensing image segmentation model to evaluate the model's performance on unseen data, thereby verifying the model's generalization ability. The accuracy of pixel-by-pixel classification results is an important metric for evaluating model performance and helps understand the model's ability to process details. For model inference tasks, the pixel-by-pixel classification results are presented as segmentation plots, allowing users to visually visualize the model's segmentation performance and better understand how the model works and its performance across different categories. By calculating the intersection over union (IoU) between the true and predicted values, the model's accuracy on the classification task can be quantified. Comparing the current round's evaluation metric with the previous highest evaluation metric can determine whether the model has improved. If the current round's evaluation metric outperforms the previous highest metric, the model parameters are updated, which helps maintain or further improve the model's performance in subsequent training. If the current round's evaluation metric does not exceed the previous highest metric, the next round of supervised training can be performed to find more optimal model parameters. The performance on the validation set can guide subsequent training. If the model performs poorly on the validation set, adjustments to the training strategy may be necessary, such as increasing the number of training rounds, adjusting the learning rate, or using a more complex network architecture. By continuously monitoring the performance on the validation set, you can ensure that the model does not overfit or underfit during training.

[0116] Example 2

[0117] See also Figure 2 , a remote sensing image segmentation system based on local consistency and pseudo-label filtering, including:

[0118] Dataset acquisition module, used to acquire optical remote sensing datasets;

[0119] A segmentation model construction module is used to construct a remote sensing image segmentation model, wherein the remote sensing image segmentation model includes a local consistency loss calculation module and a pseudo label filtering module;

[0120] The segmentation result obtaining module is used to input the optical remote sensing data set into the remote sensing image segmentation model to obtain the remote sensing image segmentation result map.

[0121] By introducing a local consistency loss calculation module, this embodiment enables the system to consider the relationship between adjacent pixels in the image, thereby maintaining spatial consistency during the segmentation process, reducing noise and isolated points in the segmentation results, and improving the smoothness and accuracy of the segmentation results. The pseudo-label filtering module can identify and filter out low-quality pseudo-labels, ensuring that the labeled data used for model training has high accuracy, reducing noise interference during model training, and improving the model's segmentation accuracy. The remote sensing image segmentation system based on local consistency and pseudo-label filtering can handle complex and variable remote sensing image scenes, including those with different lighting conditions, resolutions, and landform types. The system has greater adaptability and robustness in practical applications. By calculating local consistency loss and using pseudo-label filtering, the system can effectively reduce the impact of noise on segmentation results, maintaining high segmentation accuracy and stability when processing noisy remote sensing images. Pseudo-labels can be generated from unlabeled remote sensing image data, reducing reliance on large amounts of labeled data, reducing the cost and time of data annotation, and improving data utilization efficiency. Through continuous iteration and optimization of model parameters, the system can gradually improve the quality of pseudo-labels, further improving segmentation accuracy, and achieving continuous performance improvement with limited labeled data.

[0122] Example 3

[0123] See also Figure 3 , an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the remote sensing image segmentation method based on local consistency and pseudo-label filtering described in the first embodiment is implemented.

[0124] Example 4

[0125] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the remote sensing image segmentation method based on local consistency and pseudo-label filtering described in embodiment 1.

[0126] Example 5

[0127] The remote sensing image segmentation method based on local consistency and pseudo-label filtering in this embodiment is a semi-supervised remote sensing image segmentation method. This method uses unlabeled remote sensing data to classify each pixel in the remote sensing image into a specific category label to produce a final segmentation result. This method aims to ensure accurate semantic segmentation while maintaining clear segmentation edges in remote sensing images with cluttered backgrounds. Figure 4 The remote sensing image segmentation method based on local consistency and pseudo-label filtering in this embodiment includes the following steps:

[0128] Step S1: Construct a dataset. The acquisition of remote sensing high-resolution optical images usually relies on remote sensing platforms such as remote sensing satellites and drones. Remote sensing high-resolution optical images are an application of remote sensing technology, using optical sensors to capture reflected or emitted light from surface objects to generate high-resolution images. The number of pixels in the image is large, and the ground area corresponding to a single pixel is small, so the detailed features of surface objects can be clearly displayed. The resolution of remote sensing high-resolution optical images mainly refers to their spatial resolution, that is, the ground size corresponding to the unit pixel (Pixel) on the image, usually expressed in meters (m) or smaller units. Currently, high-resolution images at the meter level or even sub-meter level can be provided.

[0129] This example uses the Gaofen-1 satellite to collect high-resolution optical remote sensing images and professionally annotates each image at the pixel level. Furthermore, a large amount of unlabeled data is collected to expand the coverage and diversity of the dataset. This unlabeled data is used in semi-supervised learning methods to improve the model's generalization ability in new scenarios. By combining labeled and unlabeled data, a comprehensive, high-quality optical remote sensing dataset is constructed.

[0130] Step S2: Model training: The acquired optical remote sensing segmentation dataset is segmented using the remote sensing image segmentation model with local consistency and pseudo-label filtering designed in the embodiment.

[0131] Furthermore, the model in step S2 is obtained by the following steps:

[0132] Step S21: Sampling and dividing the data set.

[0133] When constructing a dataset, since labeling data requires significant manpower, physical resources, and financial resources, unlabeled data far outweighs labeled data. Therefore, the unlabeled data should be sampled to a similar size as the labeled data. Furthermore, the labeled data should be divided into training, validation, and test sets in a 7:2:1 ratio. The training set is used for model optimization and convergence, while the validation set is used for optimal model selection. The test set is used to evaluate the final model performance after model training and parameter adjustment. If using a public dataset, the data division of the public dataset should be maintained.

[0134] Step S22: Build a model. Use the training set to train a multimodal visual language model.

[0135] Furthermore, the model in step S22 is obtained by the following steps:

[0136] Step S221: Construct a segmentation network. Use Swin Transformer Base as the encoder and UperNetHead as the decoder to construct a remote sensing segmentation network, and initialize the encoder with ImageNet weights.

[0137] Swin Transformer is a hierarchical visual model based on Transformer. It uses the self-attention mechanism within the window to reduce the amount of computation and realizes cross-window connections by moving the window. Swin TransformerBase is its basic version, which usually contains fewer layers and parameters, but is able to capture the key features in the image. UperNetHead is a decoder structure commonly used in image segmentation tasks. It usually contains multiple convolutional layers, upsampling layers, and feature fusion layers to convert the feature maps extracted by the encoder into the final segmentation results. The design of UperNetHead enables it to effectively utilize the multi-scale features extracted by the encoder and generate high-resolution segmentation maps.

[0138] Step S23: Generate pseudo labels.

[0139] Furthermore, the pseudo labels in step S23 are obtained by the following steps:

[0140] Step S231: Send the unlabeled data into the constructed segmentation network to generate rough pseudo labels.

[0141] Step S232: See Figure 5 ,Through the pseudo label filtering module, the parts that are inconsistent between clustering and boundary filtering are ,filtered to generate the final pseudo label.,Clustering is to group pixels of similar color and brightness together, while boundary ,filtering separates them based on edges.

[0142] The first step is to obtain the feature map after passing through Swin Transformer Base and cluster it.

[0143] In the second step, the clustered image is added to the rough pseudo-label generated in step S231.

[0144] The third step is to calculate the entropy value in each cluster block. The calculation formula is as follows:

[0145]

[0146] in, It is a segmentation network For the first The softmax probability value generated by an unlabeled image at pixel (h, w), It is Victoria , The higher the value, the higher the uncertainty of the model prediction, that is, the entropy map is sorted pixel by pixel in ascending order.

[0147] The fourth step is to sort according to the entropy value and remove the 5% cluster blocks with higher entropy values.

[0148] Step 5: Use the Canny operator to get the image boundary. The category within the boundary is the maximum value of the accumulated pixel categories.

[0149] The Canny operator is a classic multi-step edge detection algorithm proposed by John F. Canny in 1986. It uses Gaussian filtering, gradient calculation, non-maximum suppression, double thresholding, and edge connection to extract edge information with high accuracy and low error. The core concept of the Canny operator is to use multi-stage edge detection to find the optimal edge detection solution, namely, the location with the strongest grayscale intensity change in the image. This significantly reduces the image data size while preserving the original image properties. The Canny operator accurately detects true edges in an image while minimizing interference from non-edge components. During the edge detection process, the Canny operator minimizes false edge detections, where noise and details are mistakenly identified as edges. For distinct edges, the Canny operator produces a single-pixel-wide edge response.

[0150] In the sixth step, the pseudo labels obtained in the fourth and fifth steps are combined to obtain the final pseudo labels.

[0151] Step S24: Process the labeled data.

[0152] Furthermore, in step S24, the tag data is obtained by the following steps:

[0153] Step S241: random scaling (the scale is between 0.5 and 2, 0.5 means reducing to 50% of the original size, and 2 means enlarging to 200% of the original size), random cropping, and random horizontal flipping.

[0154] Step S242: Perform strong enhancement—color conversion on the data.

[0155] Step S3: Supervised training. Use unlabeled data with pseudo labels and labeled data to train a multimodal visual language model.

[0156] The Multimodal Vision-Language Model (VLM) is an artificial intelligence model that can simultaneously process and understand both visual (image) and language (text) modal information. It combines computer vision (CV) and natural language processing (NLP), giving machines the ability to understand and interpret the complex relationships between images and text.

[0157] Furthermore, the supervised training in step S3 is completed by the following process:

[0158] Step S31: Integrate unlabeled data and labeled data. The unlabeled data with pseudo labels generated in step S23 and the labeled data in step S24 are fed into the constructed segmentation network.

[0159] Step S32: Calculate the loss.

[0160] The first step is to calculate the cross entropy loss.

[0161]

[0162] in, is the category predicted by the pseudo label, is the category of the true label;

[0163] In the second step, the local consistency contrast loss is calculated.

[0164]

[0165] in, represents the local consistency contrast loss, represents mutual information, represents the Shannon entropy,) is given hour The conditional entropy of is the number of ROIs in a patch, There is a corresponding ROI in the label enhancement image, is the corresponding ROI of the labeled image; mutual information The exact value of is difficult to calculate, so an estimate based on the MINE algorithm is used, which is implemented by backpropagation in a two-layer fully connected network.

[0166] Step S33: Gradient optimization. The network weights are optimized and updated according to the loss. The AdamW optimizer and Step learning rate strategy are used. The learning rate for the first 60 epochs is set to , the learning rate for the next 30 epochs is set to .

[0167] An epoch refers to the number of times the entire training dataset is forward-propagated and backward-propagated through a neural network. After an epoch, the model updates its weights using the backpropagation algorithm. Epoch is a crucial parameter in neural network training, influencing the model's training effectiveness and convergence speed. Through multiple epochs of training, the model gradually learns the characteristics of the dataset and optimizes its weights and biases to minimize the loss function.

[0168] The AdamW optimizer is an improvement on the Adam optimizer. The core improvement of the AdamW algorithm is to decouple weight decay from gradient updates, thereby applying weight decay more effectively in each iteration, improving the convergence speed and generalization ability of the model, and retaining the advantages of momentum and adaptive learning rate of the Adam algorithm.

[0169] The step learning rate strategy is a piecewise decay strategy that reduces the learning rate by a fixed factor every certain training cycles or steps. During training, as the model approaches the optimal solution, the learning rate needs to be gradually reduced to avoid excessive fluctuations near the optimal solution. The step learning rate strategy achieves this goal by periodically reducing the learning rate, which allows for gradual convergence. It is suitable for relatively stable optimization tasks and is simple to implement, understand, and apply. The AdamW optimizer and the step learning rate strategy can be used together to improve model training results and performance.

[0170] Step S34: Determine whether the current training round number Epoch has reached the set round number 90. If so, save the model parameters and end the training. If not, enter the verification phase and set Epoch=Epoch+1.

[0171] Step S4: Validate the model. Use the validation set to evaluate the generated remote sensing image segmentation model based on local consistency and pseudo-label filtering.

[0172] Furthermore, step S4 includes the following steps:

[0173] Step S41: Input a remote sensing image in the visible light mode, normalize the input image, and do no other data enhancement.

[0174] Step S42: Send the image to the segmentation model for calculation to obtain the final pixel-by-pixel classification result.

[0175] Step S43: Select the result presentation method according to the task type. If it is a model inference task, assign different colors to each category according to the pixel-by-pixel classification results to obtain a segmentation result map.

[0176] For model evaluation tasks, metrics are calculated based on the pixel-by-pixel classification results and the actual classifications. The Intersection over Union (IoU) ratio between the actual and predicted values ​​is used. The current round's model evaluation metric is compared with the previous highest metric. If it is greater than the highest metric, the weights are updated to the model parameters for that round. Otherwise, training proceeds to the next round.

[0177] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0178] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0179] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0180] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the specific implementations of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A remote sensing image segmentation method based on local consistency and pseudo-label filtering, characterized in that: The following steps are involved: Acquiring an optical remote sensing data set includes the following steps: acquiring a remote sensing image, annotating the remote sensing image to obtain labeled data and unlabeled data, and constructing an optical remote sensing data set using the labeled data and the unlabeled data; the remote sensing image is a visible light modality remote sensing image; Constructing a remote sensing image segmentation model, the remote sensing image segmentation model includes a local consistency loss calculation module and a pseudo-label filtering module; the unlabeled data is subjected to a remote sensing segmentation network and the pseudo-label filtering module to generate pseudo-labels to obtain unlabeled data with pseudo-labels, including the following steps: Input unlabeled data into the remote sensing segmentation network to generate rough pseudo labels; The coarse pseudo labels are filtered by the pseudo label filtering module to obtain pseudo labels; The coarse pseudo labels are filtered by the pseudo label filtering module to obtain pseudo labels, specifically: Input the training set into Swin Transformer Base to obtain feature maps, and cluster the feature maps to obtain clustered images; Add the clustered images and the rough pseudo-labels to get the cluster blocks, and calculate the entropy value of each cluster block; The entropy value of each cluster block is calculated using the following formula: in, It is a segmentation network For the first The softmax probability value generated by an unlabeled image at pixel (h, w), It is Victoria , The higher the value, the higher the uncertainty of the model prediction, which means sorting the entropy map pixel by pixel in ascending order; All cluster blocks are sorted according to the entropy value, and the top 5% of cluster blocks are deleted to obtain the remaining cluster blocks after removing the high entropy value cluster blocks; The boundary information of the image is obtained through the Canny operator. The category of each area within the boundary is the maximum value of the accumulated pixel point category, and the area category determined by boundary detection is obtained; The pseudo labels are obtained by combining the remaining cluster blocks after removing the high entropy cluster blocks and the region categories determined by boundary detection; The training set is enhanced to obtain the enhanced labeled data, including the following steps: The training set is randomly scaled, randomly cropped, and randomly flipped horizontally before being subjected to color conversion and strong enhancement to obtain enhanced labeled data. The scale of the random scaling is between 0.5 and 2, where 0.5 represents reduction to 50% of the original size and 2 represents enlargement to 200% of the original size. The calculation formula of local consistency contrast loss is as follows: in, represents the local consistency contrast loss, represents mutual information, represents the Shannon entropy, is given hour The conditional entropy of is the number of ROIs in a patch, There is a corresponding ROI in the label enhancement image, It is the corresponding ROI of the label map; The optical remote sensing dataset is input into the remote sensing image segmentation model to obtain the remote sensing image segmentation result map.

2. The remote sensing image segmentation method based on local consistency and pseudo-label filtering according to claim 1, characterized in that: The step of inputting the optical remote sensing data set into the remote sensing image segmentation model to obtain the remote sensing image segmentation result graph comprises the following steps: Divide the labeled data into training set, validation set and test set; The unlabeled data is processed through the remote sensing segmentation network and the pseudo-label filtering module to generate pseudo-labels to obtain unlabeled data with pseudo-labels; Enhance the training set to obtain enhanced labeled data; Use pseudo-labeled unlabeled data and enhanced labeled data to perform supervised training on the remote sensing segmentation network to generate a remote sensing image segmentation model; The validation set is input into the remote sensing image segmentation model for model validation. The task types of model validation include model evaluation tasks and model reasoning tasks. The remote sensing image segmentation result map is obtained through the model reasoning task.

3. The remote sensing image segmentation method based on local consistency and pseudo-label filtering according to claim 2, characterized in that: The remote sensing segmentation network uses Swin Transformer Base as the encoder and UperNetHead as the decoder, and the encoder is initialized using ImageNet weights.

4. The remote sensing image segmentation method based on local consistency and pseudo-label filtering according to claim 2, characterized in that: The supervised training is specifically as follows: Calculate the loss according to the cross entropy loss calculation formula and the local consistency contrast loss calculation formula; The calculation formula of the cross entropy loss is as follows: in, is the category predicted by the pseudo label, is the category of the true label; The network weights are optimized and updated according to the loss, using the AdamW optimizer and the Step learning rate strategy. The learning rate for the first two-thirds of the training rounds is , the learning rate for the last one-third of the training rounds is ; Determine whether the number of Epochs of the current supervised training has reached the preset number of epochs. If so, save the model parameters and end the training. If not, enter the verification phase and set Epoch=Epoch+1.

5. The remote sensing image segmentation method based on local consistency and pseudo-label filtering according to claim 2, characterized in that: The specific steps of inputting the validation set into the remote sensing image segmentation model for model validation are as follows: After normalizing the remote sensing image, it is fed into the remote sensing image segmentation model for calculation to obtain the pixel-by-pixel classification results. The pixel-by-pixel classification result presentation method is selected according to the task type of model verification. If it is a model inference task, a different color is assigned to each category based on the pixel-by-pixel classification results to obtain a segmentation result map; If it is a model evaluation task, the intersection of the true value and the predicted value is calculated based on the pixel-by-pixel classification result and the read true classification. The model evaluation index of this round is compared with the previous highest evaluation index. If it is greater than the highest index, the weight is updated to the model parameter of this round. Otherwise, it enters the next round of supervised training.

6. A remote sensing image segmentation system based on local consistency and pseudo-label filtering, characterized in that: include: The data set acquisition module is used to acquire an optical remote sensing data set, comprising the following steps: acquiring a remote sensing image, annotating the remote sensing image to obtain labeled data and unlabeled data, and constructing an optical remote sensing data set using the labeled data and the unlabeled data; the remote sensing image is a visible light modality remote sensing image; The segmentation model construction module is used to construct a remote sensing image segmentation model, wherein the remote sensing image segmentation model includes a local consistency loss calculation module and a pseudo-label filtering module; the unlabeled data is generated into pseudo-labeled unlabeled data by the remote sensing segmentation network and the pseudo-label filtering module, and the steps include: Input unlabeled data into the remote sensing segmentation network to generate rough pseudo labels; The coarse pseudo labels are filtered by the pseudo label filtering module to obtain pseudo labels; The coarse pseudo labels are filtered by the pseudo label filtering module to obtain pseudo labels, specifically: Input the training set into Swin Transformer Base to obtain feature maps, and cluster the feature maps to obtain clustered images; Add the clustered images and the rough pseudo-labels to get the cluster blocks, and calculate the entropy value of each cluster block; The entropy value of each cluster block is calculated using the following formula: in, It is a segmentation network For the first The softmax probability value generated by an unlabeled image at pixel (h, w), It is Victoria , The higher the value, the higher the uncertainty of the model prediction, which means sorting the entropy map pixel by pixel in ascending order; All cluster blocks are sorted according to the entropy value, and the top 5% of cluster blocks are deleted to obtain the remaining cluster blocks after removing the high entropy value cluster blocks; The boundary information of the image is obtained through the Canny operator. The category of each area within the boundary is the maximum value of the accumulated pixel point category, and the area category determined by boundary detection is obtained; The pseudo labels are obtained by combining the remaining cluster blocks after removing the high entropy cluster blocks and the region categories determined by boundary detection; The training set is enhanced to obtain the enhanced labeled data, including the following steps: The training set is randomly scaled, randomly cropped, and randomly flipped horizontally before being subjected to color conversion and strong enhancement to obtain enhanced labeled data. The scale of the random scaling is between 0.5 and 2, where 0.5 represents reduction to 50% of the original size and 2 represents enlargement to 200% of the original size. The calculation formula of local consistency contrast loss is as follows: in, represents the local consistency contrast loss, represents mutual information, represents the Shannon entropy, is given hour The conditional entropy of is the number of ROIs in a patch, There is a corresponding ROI in the label enhancement image, It is the corresponding ROI of the label map; The segmentation result obtaining module is used to input the optical remote sensing data set into the remote sensing image segmentation model to obtain the remote sensing image segmentation result map.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the remote sensing image segmentation method based on local consistency and pseudo-label filtering as described in any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the remote sensing image segmentation method based on local consistency and pseudo-label filtering described in any one of claims 1 to 5 is implemented.

Images