Remote sensing image segmentation method based on local consistency and pseudo label filtering

Abstract

The invention discloses a remote sensing image segmentation method based on local consistency and pseudo label filtering. The method comprises the following steps: acquiring an optical remote sensing data set; constructing a remote sensing image segmentation model, wherein the remote sensing image segmentation model comprises a local consistency loss calculation module and a pseudo label filtering module; and inputting the optical remote sensing data set into the remote sensing image segmentation model to obtain a remote sensing image segmentation result graph. A local consistency loss and pseudo label filtering strategy is provided, local consistency features are effectively learned, pseudo label noise is removed, a good remote sensing image segmentation effect is obtained, more label-free remote sensing images are utilized to improve remote sensing image segmentation precision, remote sensing images acquired by satellites can be segmented, and remote sensing image segmentation efficiency is improved. The problems that in a remote sensing image, objects are dense, the background difference is large, the size change range is large, the color and texture difference is large, local consistency features are difficult to obtain, and a large amount of noise inevitably exists in a pseudo tag are solved.

Description

Technical Field

[0001] The invention belongs to the technical field of image segmentation, and in particular relates to a remote sensing image segmentation method based on local consistency and pseudo-label filtering. Background Art

[0002] Remote sensing image segmentation is the process and technology of pixel-level classification of remote sensing images, which classifies each pixel of the remote sensing image into a specific category label. Image segmentation models based on supervised learning have made remarkable achievements in the field of natural image and remote sensing image interpretation by utilizing sufficient amounts of accurate manually labeled data. However, in some practical application scenarios, it is difficult to obtain high-quality labeled data, while it is easy to obtain a large amount of raw data, especially for high-resolution remote sensing images. It will consume a lot of resources because it requires high professionalism of the labelers and the labor cost is also high. To alleviate this problem, a natural idea is to use a large amount of raw data to assist model learning when only a small amount of labeled data is available, which is called semi-supervised learning. Most of the existing SSL methods focus on image classification tasks and can be divided into two mainstream strategies, namely self-training (ST) and consistency regularization (CR). ST improves the model by expanding the training set and retraining with high-quality pseudo labels of unlabeled data under a predefined confidence threshold. CR improves the performance of SSL by forcing the model prediction to remain consistent under different perturbations, including input, feature or network perturbations. Another popular SSL family adopts strong data augmentation (SDA) to enhance the robustness and generalization of the model, thus achieving better performance.

[0003] Although semi-supervised image segmentation has made significant progress in the field of natural image interpretation, there are few studies on semi-supervised RS image segmentation. Among them, the paper Simple and Efficient: ASemisupervised Learning Framework for Remote Sensing Image Semantic Segmentation, published in the TGARS journal in 2022, provides a strong baseline for the field of semi-supervised image segmentation and proposes a two-stage teacher-student network LSST with sample-level category-related adaptive thresholds. The paper Self Pseudo Entropy Knowledge Distillation for Semi-supervised Semantic Segmentation published in the TGARS journal in 2023 proposed an effective framework based on consistency learning, designed a novel unsupervised data enhancement strategy EntropyMix, and proposed self-pseudo entropy knowledge distillation (SPEED) to optimize the linear combination of feature and logit-level distillation losses, and learn global pixel relationships from multi-view and cross-view perturbations, thereby improving model performance without the need for additional auxiliary segmentation heads or complex pre-trained teacher models.

[0004] However, the above methods do not design corresponding modules for the problem that objects in remote sensing images are dense, the background is different, the size range is large, the color and texture are different, and it is difficult to obtain local consistency features. The design of noise filtering in pseudo-labels is only based on categories or overall features, and does not comprehensively consider the edge information and color morphology information of unlabeled data. Summary of the invention

[0005] The purpose of the present invention is to overcome the problem that it is difficult to obtain local consistency features in remote sensing images due to dense objects and large background differences, and there is a lot of noise in pseudo-labels. A remote sensing image segmentation method based on local consistency and pseudo-label filtering is proposed.

[0006] In order to achieve the above object, the present invention adopts the following technical scheme: In a first aspect, the remote sensing image segmentation method based on local consistency and pseudo-label filtering provided by the present invention comprises the following steps: Acquisition of optical remote sensing datasets; Constructing a remote sensing image segmentation model, wherein the remote sensing image segmentation model includes a local consistency loss calculation module and a pseudo label filtering module; The optical remote sensing data set is input into the remote sensing image segmentation model to obtain the remote sensing image segmentation result map.

[0007] Furthermore, the obtaining of the optical remote sensing data set comprises the following steps: obtaining a remote sensing image, annotating the remote sensing image to obtain labeled data and unlabeled data, and constructing an optical remote sensing data set using the labeled data and the unlabeled data; the remote sensing image is a visible light modality remote sensing image.

[0008] Furthermore, the step of inputting the optical remote sensing data set into the remote sensing image segmentation model to obtain the remote sensing image segmentation result map comprises the following steps: Divide the labeled data into training set, validation set and test set; The unlabeled data is processed through the remote sensing segmentation network and the pseudo-label filtering module to generate pseudo-labels to obtain unlabeled data with pseudo-labels. Enhance the training set to obtain enhanced labeled data; Use pseudo-labeled unlabeled data and enhanced labeled data to conduct supervised training on the remote sensing segmentation network to generate a remote sensing image segmentation model; The validation set is input into the remote sensing image segmentation model for model validation. The task types of model validation include model evaluation tasks and model reasoning tasks. The remote sensing image segmentation result map is obtained through the model reasoning task.

[0009] Furthermore, the remote sensing segmentation network uses Swin Transformer Base as encoder, UperNetHead as decoder, and uses ImageNet weights to initialize the encoder.

[0010] Furthermore, the unlabeled data generates pseudo labels through a remote sensing segmentation network and a pseudo label filtering module to obtain unlabeled data with pseudo labels, including the following steps: Input unlabeled data into the remote sensing segmentation network to generate rough pseudo labels; The coarse pseudo-label is filtered by the pseudo-label filtering module to obtain the pseudo-label; The coarse pseudo-label is filtered by the pseudo-label filtering module to obtain the pseudo-label, which is specifically: Input the training set into Swin Transformer Base to obtain feature maps, and cluster the feature maps to obtain clustered images; Add the clustered images and the rough pseudo-labels to get the cluster blocks, and calculate the entropy value of each cluster block; The entropy value of each cluster block is calculated using the following formula:

[0011] in, It is a segmentation network For The softmax probability value generated by the unlabeled image at pixel (h, w), It is Victoria , The higher the value, the higher the uncertainty of the model prediction, which means sorting the entropy map pixel by pixel in ascending order; All cluster blocks are sorted according to the entropy value, and the top 5% of cluster blocks in the entropy value sorting are deleted to obtain the remaining cluster blocks after removing the high entropy value cluster blocks; The boundary information of the image is obtained through the Canny operator, and the category of each area within the boundary is the maximum value of the accumulated pixel point categories to obtain the area category determined by boundary detection; The remaining cluster blocks after removing the high entropy cluster blocks and the regional categories determined by boundary detection are combined to obtain pseudo labels; The step of enhancing the training set to obtain enhanced labeled data includes the following steps: The training set is subjected to color conversion strong enhancement after random scaling, random cropping, and random horizontal flipping to obtain enhanced labeled data. The scale of the random scaling is between 0.5 and 2, where 0.5 represents reduction to 50% of the original size and 2 represents enlargement to 200% of the original size.

[0012] Furthermore, the supervised training is specifically as follows: Calculate the loss according to the cross entropy loss calculation formula and the local consistency contrast loss calculation formula; The calculation formula of the cross entropy loss is as follows:

[0013] in, is the category predicted by the pseudo label, is the category of the true label; The calculation formula of the local consistency contrast loss is as follows:

[0014] in, represents the local consistency contrast loss, represents mutual information, represents Shannon entropy,) is given hour The conditional entropy of is the number of ROIs in a patch, It is the corresponding ROI in the label enhancement image. It is the corresponding ROI of the labeled image; The network weights are optimized and updated according to the loss, using the AdamW optimizer and the Step learning rate strategy. The learning rate for the first two-thirds of the training rounds is , the learning rate for the last one-third of the training rounds is ; Determine whether the number of Epochs of the current supervised training has reached the preset number of Epochs. If so, save the model parameters and end the training. If not, enter the verification phase and set Epoch=Epoch+1.

[0015] Furthermore, the step of inputting the verification set into the remote sensing image segmentation model for model verification is specifically as follows: After normalizing the remote sensing image, send it to the remote sensing image segmentation model for calculation to obtain the pixel-by-pixel classification result. The pixel-by-pixel classification result presentation method is selected according to the task type of model verification; If it is a model inference task, different colors are assigned to each category according to the pixel-by-pixel classification results to obtain the segmentation result map; If it is a model evaluation task, the intersection and union ratio of the true value and the predicted value is calculated based on the pixel-by-pixel classification result and the real classification read, and the model evaluation index of this round is compared with the previous highest evaluation index. If it is greater than the highest index, the weight is updated to the model parameter of this round, otherwise it enters the next round of supervised training.

[0016] In a second aspect, the remote sensing image segmentation system based on local consistency and pseudo-label filtering provided by the present invention comprises: A data set acquisition module, used to acquire optical remote sensing data sets; A segmentation model construction module, used to construct a remote sensing image segmentation model, wherein the remote sensing image segmentation model includes a local consistency loss calculation module and a pseudo label filtering module; The segmentation result obtaining module is used to input the optical remote sensing data set into the remote sensing image segmentation model to obtain the remote sensing image segmentation result map.

[0017] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the remote sensing image segmentation method based on local consistency and pseudo-label filtering when executing the computer program.

[0018] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the remote sensing image segmentation method based on local consistency and pseudo-label filtering is implemented.

[0019] Compared with the prior art, the present invention has the following beneficial technical effects: The remote sensing image segmentation method based on local consistency and pseudo-label filtering proposed in the present invention proposes a local consistency loss and pseudo-label filtering strategy, effectively learns local consistency features, removes the noise of pseudo-labels, and obtains good remote sensing image segmentation effect. More unlabeled remote sensing images are used to improve the remote sensing image segmentation accuracy. The remote sensing images acquired by satellites can be segmented to obtain segmentation maps with specific object category labels, which solves the problems of dense objects in remote sensing images, large background differences, large size variation range, large color and texture differences, difficulty in obtaining local consistency features, and inevitable presence of a large amount of noise in pseudo-labels.

[0020] Furthermore, in order to solve the problems of dense objects and large background differences in RS images, the present invention adopts a semi-supervised segmentation method based on local consistency and pseudo-label filtering in the field of remote sensing single-modal image segmentation, and uses local consistency loss to extract local semantic details in remote sensing images; in order to solve the problem of a large amount of noise in pseudo-labels, a pseudo-label filtering module is proposed, which clusters feature maps and sorts them by entropy values, and can adaptively remove noise with inconsistent color and morphology in labels; and uses the Canny operator to extract image boundaries and remove boundary noise, thereby effectively removing noise in pseudo-labels, thereby better obtaining segmentation result maps. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are for explanation purposes only and are not intended to limit the scope of the present invention in any way. In addition, the shapes and proportional dimensions of the components in the drawings are only for illustration purposes to help understand the present invention, and are not intended to specifically limit the shapes and proportional dimensions of the components of the present invention. In the drawings: Figure 1 The present invention is a flowchart of a remote sensing image segmentation method based on local consistency and pseudo-label filtering.

[0022] Figure 2 This is a structural diagram of the remote sensing image segmentation system based on local consistency and pseudo-label filtering of the present invention.

[0023] Figure 3 The present invention is a flowchart of a remote sensing image segmentation method based on local consistency and pseudo-label filtering.

[0024] Figure 4 The present invention provides a flowchart of a remote sensing image segmentation method based on local consistency and pseudo-label filtering in an embodiment of the present invention.

[0025] Figure 5 It is a pseudo-label filtering module in an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0027] Embodiment 1 See also Figure 1 , a remote sensing image segmentation method based on local consistency and pseudo-label filtering includes the following steps: Acquisition of optical remote sensing datasets; Constructing a remote sensing image segmentation model, wherein the remote sensing image segmentation model includes a local consistency loss calculation module and a pseudo label filtering module; The optical remote sensing data set is input into the remote sensing image segmentation model to obtain the remote sensing image segmentation result map.

[0028] The remote sensing image segmentation method based on local consistency and pseudo-label filtering provided in this embodiment improves the automation level of remote sensing image segmentation by introducing technical means such as local consistency and pseudo-label filtering; local consistency loss calculation helps the model to adjust local features more flexibly during the training process, reduce overfitting, and improve the generalization ability of the model. Remote sensing images often contain complex surface coverage and various types of objects. The combination of local consistency and pseudo-label filtering helps the model to better adapt to these complex scenes. Through local consistency loss calculation, the model can learn the characteristic relationship between local areas in the image, so as to more accurately identify different types of objects. Pseudo-label technology is to use the model trained by labeled data to predict on unlabeled data, and screen samples according to the prediction results. The pseudo-label filtering module can identify and remove pseudo-labels whose prediction results are unreliable or noisy, thereby avoiding the negative impact of these erroneous labels on model training. By filtering out pseudo-labels of poor quality, the model can be trained based on more accurate data, further improving the segmentation accuracy. Pseudo-label technology enables the model to be trained using a large amount of unlabeled data, thereby expanding the scale of the training data set. Through the pseudo-label filtering module, the model can filter out valuable samples from these unlabeled data for training, improving training efficiency. The combination of the local consistency loss calculation module and the pseudo-label filtering module helps the model converge to the optimal solution faster. By optimizing local features and removing noise labels, the model can achieve higher segmentation accuracy in fewer iterations.

[0029] Acquiring an optical remote sensing data set includes the following steps: acquiring a remote sensing image, annotating the remote sensing image to obtain labeled data and unlabeled data, and constructing an optical remote sensing data set using the labeled data and the unlabeled data; the remote sensing image is a visible light modality remote sensing image.

[0030] The visible light remote sensing images of this embodiment can directly reflect the shape, color, texture and other features of the surface objects. These features are clarified and strengthened in the annotation process, thereby improving the accuracy of the data. The annotation process helps to highlight the key information in the remote sensing image, making the data more intuitive and easy to understand. By acquiring visible light remote sensing images at different times, locations and conditions, a data set containing rich diversity and variability can be constructed, which is crucial to improving the generalization ability of the model.

[0031] Inputting the optical remote sensing data set into the remote sensing image segmentation model to obtain the remote sensing image segmentation result map includes the following steps: Divide the labeled data into training set, validation set and test set; The unlabeled data is processed through the remote sensing segmentation network and the pseudo-label filtering module to generate pseudo-labels to obtain unlabeled data with pseudo-labels. Enhance the training set to obtain enhanced labeled data; Use pseudo-labeled unlabeled data and enhanced labeled data to conduct supervised training on the remote sensing segmentation network to generate a remote sensing image segmentation model; The validation set is input into the remote sensing image segmentation model for model validation. The task types of model validation include model evaluation tasks and model reasoning tasks. The remote sensing image segmentation result map is obtained through the model reasoning task.

[0032] By dividing the labeled data into training set, validation set and test set, we can ensure that the model is fully learned during the training process, and the performance of the model can be accurately evaluated during the validation and testing phases. This division helps to avoid overfitting and improve the generalization ability of the model.

[0033] This embodiment enhances the training set to increase the diversity and richness of the data, so that the model can learn more features and patterns. This helps to improve the recognition ability of the model in complex scenes and enhance the robustness of the model. Through the remote sensing segmentation network and the pseudo-label filtering module, the unlabeled data can be assigned pseudo-labels, thereby converting them into unlabeled data with pseudo-labels. Although these pseudo-labels are not as accurate as manual annotations, they can still provide valuable information for the training of the model. Using unlabeled data with pseudo-labels and enhanced labeled data to conduct supervised training on the remote sensing segmentation network can make full use of the information in the unlabeled data and further improve the performance of the model. This semi-supervised learning method is particularly effective when label data is scarce. In the model verification stage, the performance of the model can be accurately evaluated through the model evaluation task, which helps to understand the performance of the model in different scenarios and provide a basis for the optimization and improvement of the model. Through the model reasoning task, the remote sensing image segmentation result map can be obtained. These result maps can intuitively show the segmentation effect of the model and provide important basic data for subsequent remote sensing applications. At the same time, the model reasoning task also helps to discover the problems and deficiencies that may exist in the model during the segmentation process, and provides a direction for further optimization of the model.

[0034] The remote sensing segmentation network uses Swin Transformer Base as the encoder, UperNetHead as the decoder, and uses ImageNet weights to initialize the encoder.

[0035] Swin Transformer Base is a deep learning model based on the Transformer architecture, which is particularly suitable for processing high-resolution remote sensing images. The spatial self-attention mechanism can adaptively obtain information in remote sensing images, and can effectively integrate and model spatial information. It not only retains the advantage of Transformer for long-range dependencies, but also avoids memory limitations when calculating large images. Swin TransformerBase can capture different levels of details of the image through windows and feature fusion of different scales. This multi-scale feature representation capability makes the model more flexible and accurate when processing complex remote sensing images. Swin Transformer Base has high computational efficiency and can reduce memory and computing resource requirements while ensuring accuracy, which is crucial for processing large-scale remote sensing image data.

[0036] UperNetHead is usually used for image segmentation tasks. Through its symmetrical encoder-decoder structure and jump connections, it can achieve accurate segmentation of remote sensing images. Jump connections help capture multi-scale information, thereby improving the accuracy of image segmentation. UperNetHead can capture contextual information in the image, which is crucial for remote sensing image segmentation tasks. Contextual information helps the model understand the image content more accurately, thereby improving the accuracy of the segmentation results.

[0037] By initializing the encoder with ImageNet weights, you can use the model weights pre-trained on the ImageNet dataset to accelerate the model training process, which helps the model converge to the optimal solution faster. The ImageNet dataset contains a large number of natural images, which have certain similarities with remote sensing images in structure and texture. Therefore, using ImageNet weights for initialization can make the encoder have better generalization ability when processing remote sensing images, thereby improving the performance of the model. Using pre-trained weights for initialization can also reduce the risk of overfitting of the model during training. This is because the pre-trained weights have learned a large number of image features and patterns, which are also useful for new remote sensing image segmentation tasks.

[0038] The unlabeled data is segmented by the remote sensing network and the pseudo-label filtering module to generate pseudo-labels to obtain unlabeled data with pseudo-labels, including the following steps: Input unlabeled data into the remote sensing segmentation network to generate rough pseudo labels; The coarse pseudo-label is filtered by the pseudo-label filtering module to obtain the pseudo-label; The coarse pseudo-labels are filtered by the pseudo-label filtering module to obtain pseudo-labels, which are as follows: Input the training set into Swin Transformer Base to obtain feature maps, and cluster the feature maps to obtain clustered images; Add the clustered images and the rough pseudo-labels to get the cluster blocks, and calculate the entropy value of each cluster block; The entropy value of each cluster block is calculated using the following formula:

[0039] in, It is a segmentation network For The softmax probability value generated by the unlabeled image at pixel (h, w), It is Victoria , The higher the value, the higher the uncertainty of the model prediction, which means sorting the entropy map pixel by pixel in ascending order; All cluster blocks are sorted according to the entropy value, and the top 5% of cluster blocks in the entropy value sorting are deleted to obtain the remaining cluster blocks after removing the high entropy value cluster blocks; The boundary information of the image is obtained through the Canny operator, and the category of each area within the boundary is the maximum value of the accumulated pixel point categories to obtain the area category determined by boundary detection; The remaining cluster blocks after removing the high entropy cluster blocks and the regional categories determined by boundary detection are combined to obtain pseudo labels; The training set is enhanced to obtain the enhanced labeled data, including the following steps: The training set is randomly scaled, randomly cropped, and randomly flipped horizontally before being strongly enhanced with color conversion to obtain the enhanced labeled data. The scale of random scaling is between 0.5 and 2, where 0.5 means reducing it to 50% of the original size and 2 means enlarging it to 200% of the original size.

[0040] By generating pseudo labels for unlabeled data, this embodiment can make full use of these unlabeled data resources, expand the scale of the training set, and thus improve the generalization ability of the model. In the process of generating pseudo labels, through clustering, entropy calculation and boundary detection, the areas with high uncertainty in model prediction can be filtered out to improve the quality of pseudo labels. Using high-quality pseudo labels for training can further improve the performance of the remote sensing segmentation network, making it more accurate and efficient when processing remote sensing images. Enhancing the training set, such as random scaling, random cropping, random horizontal flipping and color conversion, can increase the diversity of data and improve the adaptability of the model to different image transformations. This helps to improve the robustness of the model when processing complex and variable remote sensing images. Traditional supervised learning methods require a large amount of labeled data, and labeled data usually requires a lot of manpower and time. By generating pseudo labels, the dependence on labeled data can be reduced to a certain extent, thereby reducing the cost of data annotation. This process combines the elements of unsupervised learning (such as clustering) and supervised learning (such as training with pseudo labels), reflecting the ideas of semi-supervised learning and self-supervised learning.

[0041] Supervised training is as follows: Calculate the loss according to the cross entropy loss calculation formula and the local consistency contrast loss calculation formula; The calculation formula of cross entropy loss is as follows:

[0042] in, is the category predicted by the pseudo label, is the category of the true label; The calculation formula of local consistency contrast loss is as follows:

[0043] in, represents the local consistency contrast loss, represents mutual information, represents Shannon entropy,) is given hour The conditional entropy of is the number of ROIs in a patch, It is the corresponding ROI in the label enhancement image. It is the corresponding ROI of the labeled image; The network weights are optimized and updated according to the loss, using the AdamW optimizer and the Step learning rate strategy. The learning rate for the first two-thirds of the training rounds is , the learning rate for the last one-third of the training rounds is ; Determine whether the number of Epochs of the current supervised training has reached the preset number of Epochs. If so, save the model parameters and end the training. If not, enter the verification phase and set Epoch=Epoch+1.

[0044] The cross entropy loss function is a common method to measure the difference between two probability distributions, and is particularly suitable for processing labels in the form of one-hot encoding. By minimizing the cross entropy between the predicted distribution and the actual distribution, the classification performance of the model can be optimized, allowing the model to learn the differences between categories more accurately.

[0045] Local consistency contrast loss focuses on the similarities and differences between data samples, which helps the model learn more discriminative feature representations, thereby improving the accuracy of the model.

[0046] The AdamW optimizer of this embodiment combines the advantages of the momentum method and the RMSProp algorithm, achieves the goal of dynamically adjusting the learning rate, and adaptively adjusts the learning rate for each parameter by calculating the first-order moment estimate and the second-order moment estimate of the gradient, so as to perform well on sparse gradients or noise-containing problems. This helps to speed up the training speed of the model and improve the training efficiency. The Step learning rate strategy dynamically adjusts the learning rate according to the number of training rounds, which helps the model to converge quickly in the early stage of training, while maintaining stability in the later stage of training to avoid overfitting. Through supervised training, the model can learn more data patterns and features, thereby enhancing the generalization ability of the model, and the model can better predict and classify when facing new and unseen data. Local consistency contrast loss also helps the model understand and learn abstract concepts, mine deep representations in the data, and further improve the generalization ability of the model. The loss value calculated according to the cross entropy loss and local consistency contrast loss can optimize and update the network weights. This helps the model to continuously adjust parameters during training to better fit the data distribution and characteristics. The preset number of training epochs can be used to clearly define the end conditions of the training process. When the preset number of epochs is reached, the model parameters are saved and the training ends, which helps avoid overfitting problems caused by excessive training.

[0047] Input the validation set into the remote sensing image segmentation model for model validation as follows: After normalizing the remote sensing image, send it to the remote sensing image segmentation model for calculation to obtain the pixel-by-pixel classification result. The pixel-by-pixel classification result presentation method is selected according to the task type of model verification; If it is a model inference task, different colors are assigned to each category according to the pixel-by-pixel classification results to obtain the segmentation result map; If it is a model evaluation task, the intersection and union ratio of the true value and the predicted value is calculated based on the pixel-by-pixel classification result and the real classification read, and the model evaluation index of this round is compared with the previous highest evaluation index. If it is greater than the highest index, the weight is updated to the model parameter of this round, otherwise it enters the next round of supervised training.

[0048] In this embodiment, by inputting the validation set into the remote sensing image segmentation model, the performance of the model on unseen data can be evaluated, thereby verifying the generalization ability of the model. The accuracy of the pixel-by-pixel classification results is an important indicator for evaluating model performance, which can help understand the model's ability to process details. For model reasoning tasks, the pixel-by-pixel classification results are presented in the form of a segmentation result map, so that the segmentation effect of the model can be intuitively seen, and the working method of the model and the performance of the model on different categories can be more intuitively understood. By calculating the intersection over union (IoU) of the true value and the predicted value, the accuracy of the model on the classification task can be quantified. By comparing the evaluation index of this round with the previous highest evaluation index, it can be determined whether the model has been improved. If the evaluation index of the current round is better than the previous highest index, the model parameters are updated, which helps the model maintain or further improve performance in subsequent training. If the evaluation index of the current round does not exceed the previous highest index, the next round of supervised training can be continued to find better model parameters. The performance of the validation set can guide the subsequent training process. If the model performs poorly on the validation set, it may be necessary to adjust the training strategy, such as increasing the number of training rounds, adjusting the learning rate, using a more complex network structure, etc. By continuously monitoring the performance on the validation set, you can ensure that the model does not overfit or underfit during training.

[0049] Embodiment 2 See also Figure 2 , a remote sensing image segmentation system based on local consistency and pseudo-label filtering, including: A data set acquisition module, used to acquire optical remote sensing data sets; A segmentation model construction module, used to construct a remote sensing image segmentation model, wherein the remote sensing image segmentation model includes a local consistency loss calculation module and a pseudo label filtering module; The segmentation result obtaining module is used to input the optical remote sensing data set into the remote sensing image segmentation model to obtain the remote sensing image segmentation result map.

[0050] By introducing a local consistency loss calculation module, this embodiment can consider the relationship between adjacent pixels in the image, thereby maintaining the spatial consistency of the image during the segmentation process, reducing noise and isolated points in the segmentation results, and improving the smoothness and accuracy of the segmentation results. The pseudo-label filtering module can identify and filter out pseudo-labels with low quality, thereby ensuring that the label data used for model training has high accuracy, reducing noise interference during model training, and improving the segmentation accuracy of the model. The remote sensing image segmentation system based on local consistency and pseudo-label filtering can handle complex and changeable remote sensing image scenes, including different lighting conditions, different resolutions, and different types of objects, and the system has stronger adaptability and robustness in practical applications. Through local consistency loss calculation and pseudo-label filtering, the system can effectively reduce the impact of noise on the segmentation results, and can maintain high segmentation accuracy and stability when processing noisy remote sensing images. Pseudo-labels can be generated using unlabeled remote sensing image data, thereby reducing dependence on a large amount of labeled data, reducing the cost and time of data labeling, and improving data utilization efficiency. Through continuous iteration and optimization of model parameters, the system can gradually improve the quality of pseudo-labels, thereby further improving segmentation accuracy, and can achieve continuous performance improvement under limited labeled data.

[0051] Embodiment 3 See also Figure 3 An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the remote sensing image segmentation method based on local consistency and pseudo-label filtering described in Embodiment 1 is implemented.

[0052] Embodiment 4 A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the remote sensing image segmentation method based on local consistency and pseudo-label filtering described in embodiment 1 is implemented.

[0053] Embodiment 5 The remote sensing image segmentation method based on local consistency and pseudo-label filtering in this embodiment is a semi-supervised remote sensing image segmentation method, which means that when using unlabeled remote sensing data, each pixel of the remote sensing image is classified into a specific category label to obtain the final segmentation result map. It is necessary to ensure the correct semantic segmentation result while ensuring clear segmentation edges when the background of the remote sensing image is cluttered. Figure 4 The remote sensing image segmentation method based on local consistency and pseudo-label filtering in this embodiment includes the following steps: Step S1: Construct a data set. The acquisition of remote sensing high-resolution optical images usually relies on remote sensing platforms such as remote sensing satellites and drones. Remote sensing high-resolution optical images are an application of remote sensing technology, which uses optical sensors to capture the reflected light or emitted light of surface objects to generate high-resolution images. There are many pixels in the image, and the ground area corresponding to a single pixel is small, so the detailed features of the surface objects can be clearly displayed. The resolution of remote sensing high-resolution optical images mainly refers to their spatial resolution, that is, the ground size corresponding to the unit pixel (Pixel) on the image, usually expressed in meters (m) or smaller units. At present, it is possible to provide meter-level or even sub-meter-level high-resolution images.

[0054] This embodiment uses the Gaofen-1 satellite to collect remote sensing high-resolution optical images and professionally annotates each image at the pixel level. In addition, a large amount of unlabeled data will be collected to expand the coverage and diversity of the data set. These unlabeled data will be used in semi-supervised learning methods to improve the generalization ability of the model in new scenarios. By comprehensively utilizing labeled and unlabeled data, a comprehensive, high-quality optical remote sensing data set is constructed.

[0055] Step S2: Model training: The acquired optical remote sensing segmentation data set is segmented by using the remote sensing image segmentation model with local consistency and pseudo-label filtering designed in the embodiment.

[0056] Furthermore, the model in step S2 is obtained by the following steps: Step S21: Sampling and dividing the data set.

[0057] When constructing a data set, since labeled data requires a lot of manpower, physical and financial resources, there are far more unlabeled data than labeled data, and the unlabeled data needs to be sampled to a similar number as the labeled data. At the same time, the labeled data is divided into training set, validation set and test set in a ratio of 7:2:1. The training set is used for model optimization and convergence, and the validation set is used for optimal model selection. The test set is used to evaluate the final performance of the model after model training and parameter adjustment are completed. If a public data set is used, the data division of the public data set is maintained.

[0058] Step S22: Building a model. Using the training set to train a multimodal visual language model; Furthermore, the model in step S22 is obtained by the following steps: Step S221: Construct a segmentation network. Use Swin Transformer Base as the encoder and UperNetHead as the decoder to construct a remote sensing segmentation network, and use ImageNet weights to initialize the encoder.

[0059] Swin Transformer is a hierarchical visual model based on Transformer. It uses the self-attention mechanism within the window to reduce the amount of computation and realizes cross-window connections by moving the window. Swin TransformerBase is its basic version, which usually contains fewer layers and parameters, but can already capture the key features in the image. UperNetHead is a decoder structure commonly used in image segmentation tasks. It usually contains multiple convolutional layers, upsampling layers, and feature fusion layers to convert the feature maps extracted by the encoder into the final segmentation results. The design of UperNetHead enables it to effectively utilize the multi-scale features extracted by the encoder and generate high-resolution segmentation maps.

[0060] Step S23: Generate pseudo labels.

[0061] Furthermore, the pseudo-label in step S23 is obtained by the following steps: Step S231: Send the unlabeled data into the constructed segmentation network to generate rough pseudo labels.

[0062] Step S232: See Figure 5 ,Through the pseudo-label filtering module, the inconsistent parts of clustering and boundary filtering are ,filtered to generate the final pseudo-label.,Clustering is to group pixels of similar color and brightness together, and ,boundary filtering is to separate them according to edges.

[0063] The first step is to obtain the feature map after passing through Swin Transformer Base and cluster it.

[0064] In the second step, the clustered image is added to the rough pseudo-label generated in step S231.

[0065] The third step is to calculate the entropy value in each cluster block. The calculation formula is as follows:

[0066] in, It is a segmentation network For The softmax probability value generated by the unlabeled image at pixel (h, w), It is Victoria , The higher the value, the higher the uncertainty of the model prediction, that is, the entropy map is sorted pixel by pixel in ascending order. The fourth step is to sort according to the entropy value and remove the 5% cluster blocks with higher entropy values.

[0067] Step 5: Use the Canny operator to get the boundary of the image. The category within the boundary is the maximum value of the accumulated pixel category.

[0068] The Canny operator is a classic multi-step edge detection algorithm proposed by John F. Canny in 1986. It extracts edge information with high accuracy and low error rate through steps such as Gaussian filtering, gradient calculation, non-maximum suppression, double threshold processing and edge connection. The core idea of ​​the Canny operator is to find an optimal edge detection solution through multi-level edge detection, that is, the position where the grayscale intensity changes the most in the image. While retaining the original properties of the image, the amount of image data is significantly reduced. The Canny operator can accurately detect the real edges in the image and try to eliminate the interference of non-edge parts; the Canny operator can minimize erroneous edge detection during edge detection, that is, the situation where noise and details are misjudged as edges. For obvious edges, the Canny operator can produce edge responses with a single pixel width.

[0069] In the sixth step, the pseudo labels obtained in the fourth and fifth steps are combined to obtain the final pseudo labels.

[0070] Step S24: Processing labeled data.

[0071] Further, in step S24, the tag data is obtained by the following steps: Step S241: random scaling (the scale is between 0.5 and 2, 0.5 means reducing to 50% of the original size, and 2 means enlarging to 200% of the original size), random cropping, and random horizontal flipping.

[0072] Step S242: Perform strong enhancement—color conversion on the data.

[0073] Step S3: Supervised training. Use unlabeled data with pseudo labels and labeled data to train a multimodal visual language model; Multimodal Visual Language Model (VLM) Multimodal Visual Language Model is an artificial intelligence model that can simultaneously process and understand both visual (image) and language (text) modal information. It combines computer vision (CV) and natural language processing (NLP) to give machines the ability to understand and interpret the complex relationships between images and text.

[0074] Furthermore, the supervised training in step S3 is completed by the following process: Step S31: Integrate unlabeled data and labeled data. The unlabeled data with pseudo labels generated in step S23 and the labeled data in step S24 are fed into the constructed segmentation network.

[0075] Step S32: Calculate the loss.

[0076] The first step is to calculate the cross entropy loss.

[0077]

[0078] in, is the category predicted by the pseudo label, is the category of the true label; In the second step, the local consistency contrast loss is calculated.

[0079]

[0080] in, represents the local consistency contrast loss, represents mutual information, represents Shannon entropy,) is given hour The conditional entropy of is the number of ROIs in a patch, It is the corresponding ROI in the label enhancement image. is the corresponding ROI of the labeled image; mutual information The exact value of is difficult to calculate, so an estimate based on the MINE algorithm is used, which is implemented by backpropagation in a two-layer fully connected network.

[0081] Step S33: Gradient optimization. The network weights are optimized and updated according to the loss. The AdamW optimizer and Step learning rate strategy are used. The learning rate for the first 60 epochs is set to , the learning rate for the next 30 epochs is set to .

[0082] Epoch refers to the number of times the entire training data set is forward propagated and back propagated in the neural network. After an epoch is completed, the model will update the weights according to the back propagation algorithm. Epoch is an important parameter in neural network training, which affects the training effect and convergence speed of the model. Through multiple epoch training, the model can gradually learn the characteristics of the data set and optimize its weights and biases to minimize the loss function.

[0083] The AdamW optimizer is an improvement on the Adam optimizer. The core improvement of the AdamW algorithm is to decouple weight decay from gradient updates, thereby applying weight decay more effectively in each iteration, improving the convergence speed and generalization ability of the model, and retaining the advantages of momentum and adaptive learning rate of the Adam algorithm.

[0084] The Step learning rate strategy is a piecewise decay strategy. Every certain training cycle or step, the learning rate will be reduced by a fixed factor. During the training process, as the model gradually approaches the optimal solution, the learning rate needs to be gradually reduced to avoid excessive fluctuations near the optimal solution. The Step learning rate strategy achieves this goal by periodically reducing the learning rate, which can gradually converge and is suitable for relatively stable optimization tasks. It is simple to implement and easy to understand and apply. The AdamW optimizer and the Step learning rate strategy can be used in combination to improve the training effect and performance of the model.

[0085] Step S34: Determine whether the current training round number Epoch reaches the set round number 90. If so, save the model parameters and end the training. If not, enter the verification phase and set Epoch=Epoch+1.

[0086] Step S4: Validate the model. Use the validation set to evaluate the generated remote sensing image segmentation model based on local consistency and pseudo-label filtering; Further, step S4 includes the following steps: Step S41: input a remote sensing image in the visible light mode, normalize the input image, and do no other data enhancement.

[0087] Step S42: Send to the segmentation model for calculation to obtain the final pixel-by-pixel classification result.

[0088] Step S43: Select the result presentation method according to the task type. If it is a model reasoning task, assign different colors to each category according to the pixel-by-pixel classification results to obtain a segmentation result map.

[0089] If it is a model evaluation task, the indicator is calculated based on the pixel-by-pixel classification result and the real classification read. The intersection over union (IoU) of the real value and the predicted value is used. The current round of model evaluation indicator is compared with the previous highest evaluation indicator. If it is greater than the highest indicator, the weight is updated to the model parameter of this round, otherwise it enters the next round of training.

[0090] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0091] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0092] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0093] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention is described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation of the present invention can still be modified or replaced by equivalents, and any modification or equivalent replacement that does not deviate from the spirit and scope of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A remote sensing image segmentation method based on local consistency and pseudo-label filtering, characterized in that: The following steps are involved: Acquisition of optical remote sensing datasets; Constructing a remote sensing image segmentation model, wherein the remote sensing image segmentation model includes a local consistency loss calculation module and a pseudo label filtering module; The optical remote sensing data set is input into the remote sensing image segmentation model to obtain the remote sensing image segmentation result map.

2. The remote sensing image segmentation method based on local consistency and pseudo-label filtering according to claim 1, characterized in that: The acquisition of the optical remote sensing data set comprises the following steps: acquiring a remote sensing image, annotating the remote sensing image to obtain labeled data and unlabeled data, and constructing an optical remote sensing data set using the labeled data and the unlabeled data; the remote sensing image is a visible light modality remote sensing image.

3. The remote sensing image segmentation method based on local consistency and pseudo-label filtering according to claim 2, characterized in that: The step of inputting the optical remote sensing data set into the remote sensing image segmentation model to obtain the remote sensing image segmentation result graph comprises the following steps: Divide the labeled data into training set, validation set and test set; The unlabeled data is processed through the remote sensing segmentation network and the pseudo-label filtering module to generate pseudo-labels to obtain unlabeled data with pseudo-labels. Enhance the training set to obtain enhanced labeled data; Use pseudo-labeled unlabeled data and enhanced labeled data to conduct supervised training on the remote sensing segmentation network to generate a remote sensing image segmentation model; The validation set is input into the remote sensing image segmentation model for model validation. The task types of model validation include model evaluation tasks and model reasoning tasks. The remote sensing image segmentation result map is obtained through the model reasoning task.

4. The remote sensing image segmentation method based on local consistency and pseudo-label filtering according to claim 3, characterized in that: The remote sensing segmentation network uses Swin Transformer Base as encoder, UperNetHead as decoder, and uses ImageNet weights to initialize the encoder.

5. The remote sensing image segmentation method based on local consistency and pseudo-label filtering according to claim 4, characterized in that: The unlabeled data is subjected to a remote sensing segmentation network and a pseudo-label filtering module to generate pseudo-labels to obtain unlabeled data with pseudo-labels, including the following steps: Input unlabeled data into the remote sensing segmentation network to generate rough pseudo labels; The coarse pseudo-label is filtered by the pseudo-label filtering module to obtain the pseudo-label; The coarse pseudo-label is filtered by the pseudo-label filtering module to obtain the pseudo-label, which is specifically: Input the training set into Swin Transformer Base to obtain feature maps, and cluster the feature maps to obtain clustered images; Add the clustered images and the rough pseudo-labels to get the cluster blocks, and calculate the entropy value of each cluster block; The entropy value of each cluster block is calculated using the following formula: in, It is a segmentation network For The softmax probability value generated by the unlabeled image at pixel (h, w), It is Victoria , The higher the value, the higher the uncertainty of the model prediction, which means sorting the entropy map pixel by pixel in ascending order; All cluster blocks are sorted according to the entropy value, and the top 5% of cluster blocks in the entropy value sorting are deleted to obtain the remaining cluster blocks after removing the high entropy value cluster blocks; The boundary information of the image is obtained through the Canny operator, and the category of each area within the boundary is the maximum value of the accumulated pixel point categories to obtain the area category determined by boundary detection; The remaining cluster blocks after removing the high entropy cluster blocks and the regional categories determined by boundary detection are combined to obtain pseudo labels; The step of enhancing the training set to obtain enhanced labeled data includes the following steps: The training set is subjected to color conversion strong enhancement after random scaling, random cropping, and random horizontal flipping to obtain enhanced labeled data. The scale of the random scaling is between 0.5 and 2, where 0.5 represents reduction to 50% of the original size and 2 represents enlargement to 200% of the original size.

6. The remote sensing image segmentation method based on local consistency and pseudo-label filtering according to claim 5, characterized in that: The supervised training is specifically as follows: Calculate the loss according to the cross entropy loss calculation formula and the local consistency contrast loss calculation formula; The calculation formula of the cross entropy loss is as follows: in, is the category predicted by the pseudo label, is the category of the true label; The calculation formula of the local consistency contrast loss is as follows: in, represents the local consistency contrast loss, represents mutual information, represents Shannon entropy,) is given hour The conditional entropy of is the number of ROIs in a patch, It is the corresponding ROI in the label enhancement image. It is the corresponding ROI of the labeled image; The network weights are optimized and updated according to the loss, using the AdamW optimizer and the Step learning rate strategy. The learning rate for the first two-thirds of the training rounds is , the learning rate for the last one-third of the training rounds is ; Determine whether the number of Epochs of the current supervised training has reached the preset number of Epochs. If so, save the model parameters and end the training. If not, enter the verification phase and set Epoch=Epoch+1.

7. The remote sensing image segmentation method based on local consistency and pseudo-label filtering according to claim 6, characterized in that: The specific steps of inputting the validation set into the remote sensing image segmentation model for model validation are as follows: After normalizing the remote sensing image, send it to the remote sensing image segmentation model for calculation to obtain the pixel-by-pixel classification result. The pixel-by-pixel classification result presentation method is selected according to the task type of model verification; If it is a model inference task, different colors are assigned to each category according to the pixel-by-pixel classification results to obtain the segmentation result map; If it is a model evaluation task, the intersection and union ratio of the true value and the predicted value is calculated based on the pixel-by-pixel classification result and the real classification read, and the model evaluation index of this round is compared with the previous highest evaluation index. If it is greater than the highest index, the weight is updated to the model parameter of this round, otherwise it enters the next round of supervised training.

8. A remote sensing image segmentation system based on local consistency and pseudo-label filtering, characterized in that: include: A data set acquisition module, used to acquire optical remote sensing data sets; A segmentation model construction module, used to construct a remote sensing image segmentation model, wherein the remote sensing image segmentation model includes a local consistency loss calculation module and a pseudo label filtering module; The segmentation result obtaining module is used to input the optical remote sensing data set into the remote sensing image segmentation model to obtain the remote sensing image segmentation result map.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the remote sensing image segmentation method based on local consistency and pseudo-label filtering as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the remote sensing image segmentation method based on local consistency and pseudo-label filtering as described in any one of claims 1 to 7.

Images