Label refinement method and training method of label refinement model

Abstract

The invention relates to a label refinement method and a label refinement model training method, and the method comprises the steps: obtaining a remote sensing image and a noise label, and inputting the remote sensing image and the noise label to a trained label refinement model for label refinement processing; wherein the noise labels comprise potential wrong and missing labels; the label refinement process comprises the following steps: respectively carrying out feature extraction on a remote sensing image and a noise label, and fusing the extracted features under a triple attention mechanism to obtain a first fusion feature and a label error probability graph; according to a label error probability graph, performing fusion calculation on the first fusion feature and a label feature corresponding to the noise label to obtain a second fusion feature and an updated label; and correcting the updated tag according to the second fusion feature and a residual correction strategy to obtain a target tag. The accuracy and reliability of the target label can be improved, and training and performance improvement of a remote sensing image intelligent interpretation large model are facilitated.

Description

Technical Field

[0001] The present application relates to the technical field of remote sensing image processing, and specifically to a label refinement method for a large model of remote sensing image intelligent interpretation, a training method for the label refinement model, a computing device and a storage medium. Background Art

[0002] In the rapid development of artificial intelligence technology, data, algorithms and computing power are considered to be the three cornerstones. Especially for large models of intelligent interpretation, the quality of sample data is crucial to the training and performance of the model. As an important means of obtaining surface information, remote sensing technology provides a large amount of data for various applications such as urban planning, land cover and land use classification. Sample refinement is the process of identifying and correcting noisy labels in remote sensing image semantic segmentation tasks, thereby improving the quality of training data and the performance of the model.

[0003] In the field of remote sensing image semantic segmentation, label noise mainly comes from multiple sources, including human errors in the manual annotation process, inaccuracies in automatic annotation tools, and the inherent complexity of remote sensing images, which makes interpretation more complicated. Label noise significantly weakens the learning efficiency and generalization ability of large models, resulting in poor performance of the model on unseen data and misclassification of geographic features due to misleading noisy labels.

[0004] At present, there are many methods for dealing with noisy labels. For noisy label learning of natural images, the main methods include robust architecture and regularization, robust loss function and adjustment, and sample selection-based methods. The process of refining the labels of remote sensing image samples may involve manual verification or automated procedures. The method based on manual verification improves label accuracy through multiple rounds of quality assessment, but this increases the requirements for the ability of the annotator. To achieve automatic boundary refinement, a coarse-to-fine boundary refinement technique is used to accurately extract the outline of buildings in remote sensing images to ensure entity continuity and boundary accuracy. In the change detection task, an efficient weakly supervised label refinement strategy combining low-resolution and high-resolution training data significantly improves detection accuracy. Although feature-level label refinement effectively improves the performance of object detection in complex scenes, it increases the complexity and inference time of large models. Obviously, the research on noisy label learning in the field of remote sensing is still in its infancy. Although existing methods increase the resistance of large models to noise, they have not yet completely solved the problem of noisy label omissions. Summary of the invention

[0005] Based on the above statements, the present application provides a label refinement method and a label refinement model training method to solve the problem of poor training and performance of large models due to mislabeling or missing noise labels during the training process of large models for intelligent interpretation of remote sensing images.

[0006] In a first aspect, the present application provides a label refinement method, comprising:

[0007] Acquire remote sensing images and noise labels, and input them into the trained label refinement model for label refinement processing; wherein the noise labels include potential erroneous labels; the label refinement process includes:

[0008] Extracting features from the remote sensing image and the noise label respectively, and fusing the extracted features under a triple attention mechanism to obtain a first fused feature and a label error probability map;

[0009] According to the label error probability map, the first fused feature and the label feature corresponding to the noise label are fused and calculated to obtain a second fused feature and an updated label;

[0010] According to the second fusion feature and the residual correction strategy, the updated label is corrected to obtain a target label.

[0011] In one or more embodiments, the noise label is used to characterize the object category of the pixel point corresponding to the remote sensing image, and the input form of the noise label is a binary image.

[0012] In one or more embodiments, the trained label refinement model includes two feature extractors; the remote sensing image and the noise label are respectively subjected to feature extraction, and the extracted features are fused under a triple attention mechanism to obtain a first fused feature and a label error probability map, including:

[0013] Using the two feature extractors, extracting features from the remote sensing image and the noise label respectively to obtain image features and label features with consistent dimensions;

[0014] Aligning and subtracting the image features and the label features to obtain initial fusion features;

[0015] Perform multi-scale feature fusion on the initial fused features under the triplet attention mechanism to obtain a first fused feature;

[0016] The first fusion feature is convolved and nonlinearly transformed to obtain a heat map, and the heat map is used as the label error probability map.

[0017] In one or more embodiments, performing multi-scale feature fusion on the initial fused features under a triplet attention mechanism to obtain a first fused feature includes:

[0018] Downsampling the initial fusion feature to obtain sub-fusion features of multiple scales;

[0019] The sub-fusion features of each scale are fused under the triplet attention mechanism, and then the learned sub-fusion features of multiple scales are upsampled and concatenated to obtain the first fusion feature.

[0020] In one or more embodiments, the step of fusing the sub-fusion features of each scale under the triple attention mechanism includes:

[0021] The sub-fusion feature is rotated horizontally and vertically respectively to obtain a first input feature and a second input feature, and the sub-fusion feature itself is used as a third input feature;

[0022] The first input feature, the second input feature and the third input feature are sequentially subjected to maximum pooling, convolution, normalization and nonlinear transformation to obtain corresponding first output features, second output features and third output features;

[0023] The first output feature, the second output feature and the third output feature are averaged to obtain a sub-fusion feature after feature fusion.

[0024] In one or more embodiments, the step of performing a fusion calculation on the first fusion feature and the label feature corresponding to the noise label according to the label error probability map to obtain a second fusion feature and an updated label includes:

[0025] Adjusting the dimension of the label feature and the dimension of the label error probability map so that the dimensions of the label feature, the label error probability map and the first fusion feature are consistent;

[0026] Aligning the label error probability map with the same dimension with the first fusion feature and adding them to obtain an intermediate fusion feature;

[0027] Performing a dot product operation on the feature value of the intermediate fusion feature and the error probability value of the label error probability map, performing a dot product operation on the feature value of the label feature after dimension adjustment and the correct probability value of the label error probability map, and adding the results of the two dot product operations to obtain a second fusion feature;

[0028] The second fused feature is convolved, nonlinearly transformed, and binarized to obtain an updated label.

[0029] In one or more embodiments, the step of correcting the updated label according to the second fusion feature and the residual correction strategy to obtain the target label includes:

[0030] Adjusting the dimension of the update label, and aligning and adding the update label and the second fusion feature with the same dimension to obtain a correction matrix;

[0031] Aligning and adding the correction matrix and the second fusion feature to obtain a third fusion feature;

[0032] The third fusion feature is convolved, nonlinearly transformed, and binarized to obtain a target label.

[0033] In a second aspect, the present application provides a method for training a label refinement model, comprising:

[0034] Acquire a sample data set, wherein the sample data set includes a sample remote sensing image, a sample noise label, and a sample true value label; the sample noise label includes an error or omission label;

[0035] Constructing a pre-trained label refinement model, inputting the sample remote sensing image and the sample noise label into the pre-trained label refinement model for training, and obtaining a predicted refined label;

[0036] Calculate the loss value between the predicted refined label and the sample true value label according to a preset loss function;

[0037] According to the loss value, the pre-trained label refinement model is back-propagated to adjust the model weight parameters until the loss value is less than a preset threshold.

[0038] In a third aspect, the present application provides a computing device including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described in any one of the above embodiments when executing the computer program.

[0039] In a fourth aspect, the present application provides a storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method described in any one of the above embodiments of the claims.

[0040] Compared with the prior art, the technical solution of this application has the following beneficial technical effects:

[0041] The embodiment of the present application uses remote sensing images and noise labels as input data, and uses remote sensing images to assist noise labels in error identification and updating; first, the remote sensing images and noise labels are input into a trained label refinement model for feature extraction, and then the extracted features are fused under a triple attention mechanism, so as to better express the semantic information of label errors and omissions, and after fusion, a first fused feature and a label error probability map are obtained; according to the label error probability map, the first fused feature and the label feature corresponding to the noise label are fused and calculated to obtain a second fused feature and an updated label, that is, the noise label is adjusted, and pixels with a larger error probability are updated; according to the second fused feature and the residual correction strategy, the updated label is corrected to solve the problems of some small holes and small noise points in the updated label, and improve the accuracy and reliability of the target label. In this way, refined sample data is obtained, which is beneficial to the training and performance improvement of the large model of remote sensing image intelligent interpretation, thereby contributing to the accuracy of the large model in identifying complex surface features. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A flowchart of a label refinement method provided in an embodiment of the present application;

[0043] Figure 2 for Figure 1 Schematic diagram of the label refinement process;

[0044] Figure 3 for Figure 2 Schematic diagram of the process of multi-scale feature fusion of the initial fusion features under the triple attention mechanism;

[0045] Figure 4 for Figure 3 Schematic diagram of the feature fusion process of sub-fusion features of a single scale in the triple attention mechanism;

[0046] Figure 5 A flowchart of a method for training a label refinement model provided in an embodiment of the present application;

[0047] Figure 6 A schematic diagram of the principle of a label refinement device provided in an embodiment of the present application;

[0048] Figure 7 A schematic diagram of the principles of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to facilitate understanding of the present application, the present application will be described more fully below with reference to the relevant drawings. Embodiments of the present application are provided in the drawings. However, the present application can be implemented in many different forms and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application.

[0051] It will be appreciated that spatial relationship terms such as "under," "beneath," "below," "under," "above," "above," etc., may be used herein to describe the relationship of an element or feature shown in the figures to other elements or features. It will be appreciated that, in addition to the orientations shown in the figures, spatial relationship terms also include different orientations of the device in use and operation. For example, if the device in the accompanying drawings is flipped, an element or feature described as "under other elements" or "under it" or "under it" will be oriented as being "above" the other elements or features. Thus, the exemplary terms "under" and "under" may include both upper and lower orientations. In addition, the device may also include additional orientations (e.g., rotated 90 degrees or other orientations), and the spatial descriptors used herein are interpreted accordingly.

[0052] It should be noted that when an element is considered to be "connected" to another element, it can be directly connected to the other element, or connected to the other element through an intermediate element. The "connection" in the following embodiments should be understood as "electrical connection", "communication connection", etc. if the connected circuits, units, units, etc. have electrical signals or data transmission between each other.

[0053] When used herein, the singular forms "a", "an", and "said / the" may also include plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "include / comprise" or "have" etc. specify the presence of stated features, wholes, steps, operations, components, parts or combinations thereof, but do not exclude the possibility of the presence or addition of one or more other features, wholes, steps, operations, components, parts or combinations thereof.

[0054] In the semantic segmentation of remote sensing images, the main challenges faced by the research on noisy labels include the difficulty in obtaining accurately annotated large-scale datasets, which are essential for the effective training of large models for intelligent interpretation - deep neural networks (DNNs). Relying on non-expert sources for annotation can lead to a large number of unreliable labels, with reported label breakage rates ranging from 8% to 38.5%. Since DNNs are able to fit complex functions, they are particularly prone to overfitting to noisy labels, which weakens their generalization ability. Although traditional regularization techniques help, they do not completely solve the overfitting problem caused by noisy labels. Achieving good generalization ability in the presence of noisy labels remains a major challenge. This highlights a gap in research and indicates that innovative methods are needed to mitigate the impact of label noise, improve label accuracy, and enhance the robustness of models in noisy environments. On the other hand, the huge differences in the spatial and spectral characteristics of remote sensing data (affected by weather conditions, seasonal changes, and the type and settings of sensors when acquired), as well as the high-dimensional nature of the data and the large number of spectral bands included, pose significant challenges to the accuracy and quality of data annotation. The complexity of the different surface feature information provided by each spectral band requires that the labeling work not only requires precise expertise, but also must take into account the changing external conditions. These factors together increase the difficulty of obtaining high-quality labeled data.

[0055] Based on this, the embodiments of the present application provide a label refinement method and a label refinement model training method to solve the problem of poor training and performance of the large model due to omissions of noise labels during the training process of the large model for intelligent interpretation of remote sensing images.

[0056] See also Figure 1 , the present application embodiment provides a label refinement method, comprising the following steps:

[0057] S10, obtaining remote sensing images and noise labels, and inputting them into the trained label refinement model for label refinement processing; wherein the noise labels include potential erroneous labels.

[0058] In this embodiment, remote sensing images can be obtained through remote sensing platforms such as satellites, airplanes, drones or ground sensors, which are equipped with various types of sensors to capture images and data on the earth's surface. The remote sensing image can be an initial remote sensing image of a target scene such as a building, farmland, school, hospital or factory. The remote sensing image contains rich high spatial resolution and spectral information. The noise label is used to characterize the object category corresponding to the pixel point of the remote sensing image, and the input form of the noise label is a binary image. In theory, the pixels of the target object are white and the pixels of the non-target object are black. The noise label can be obtained by manual labeling or machine labeling. It should be noted that the noise label contains potential error labels, specifically including mislabeled labels and missing labels. For example, the target object is "factory", and the pixel points of the object category "road" are marked as "factory" in the noise label. This situation belongs to mislabeled labels; the pixel points of the object category "factory" are marked as "road", which belongs to missing labels. In this embodiment, the noise label can be denoised by the label refinement model to obtain a clean label.

[0059] In this embodiment, the label refinement process includes:

[0060] S11. Extract features of the remote sensing image and the noise label respectively, and fuse the extracted features under the triple attention mechanism to obtain the first fused feature and the label error probability map.

[0061] See also Figure 2 In this embodiment, two feature extractors are used to extract features of remote sensing images and noise labels respectively to obtain image features and label features with consistent dimensions.

[0062] Specifically, the feature extractor includes but is not limited to a U-Net model, an HRNet model, and a Transformer-based Vision Transformer model. For example, the U-Net model can be used to extract features from remote sensing images to obtain image features. It should be noted that the U-Net model is an image segmentation model based on a convolutional neural network, which is designed to identify and segment surface features of remote sensing images. Among them, the U-Net model involves convolution, normalization, activation, and pooling operations. The convolution operation is used to extract local features in the image and capture spatial context information. The normalization operation is used to standardize feature data to accelerate the training process and improve the stability of the model. The activation operation is to introduce nonlinear capabilities through nonlinear activation functions (such as ReLU functions) so that the model can learn more complex features. The pooling operation is to downsample the feature map through maximum pooling or average pooling operations, reduce feature dimensions, retain important features, and improve computational efficiency. In addition, for noisy labels, there is no need to use a complex U-Net model. Simple convolution, normalization, activation, and pooling operations can be used to extract features and obtain label features.

[0063] In this embodiment, the image features include spatial features, spectral features and semantic features. Label features include semantic features. Spatial features are used to characterize detailed information such as terrain, boundaries and texture of objects, and spectral features are used to characterize spectral reflectance characteristics of different objects. Semantic features are used to characterize image foreground features and background features. If necessary, the label features need to be expanded so that the dimensions of the label features are the same as those of the image features, such as unified as (8,128,512,512), which means that 8 images can be processed at a time, with a total of 128 dimensions, and the image size is 512×512.

[0064] Furthermore, the image features and label features are aligned and subtracted to obtain the initial fused features. Specifically, the image features and label features with the same dimension are aligned, and then the aligned image features and label features are subtracted to obtain the initial fused features. In this way, the semantic information of mislabeled or missing labels can be easily captured.

[0065] For further information, see Figure 2 , the initial fusion features are fused with multi-scale features under the triple attention mechanism to obtain the first fusion feature. Figure 3 In this embodiment, the initial fusion feature is downsampled to obtain sub-fusion features of multiple scales, and then the sub-fusion features of each scale are fused under the triple attention mechanism. For details, see Figure 4, each sub-fusion feature is rotated horizontally (Permutation-W) and vertically (Permutation-H) to obtain the first input feature and the second input feature, and the sub-fusion feature itself is used as the third input feature; the first input feature, the second input feature and the third input feature are all subjected to maximum pooling, convolution, normalization and nonlinear transformation (through the Sigmoid activation function) in turn to obtain the corresponding first output feature, the second output feature and the third output feature; the first output feature, the second output feature and the third output feature are averaged to obtain the sub-fusion feature after feature fusion; the sub-fusion features of multiple scales after learning are upsampled and spliced ​​to obtain the first fusion feature, which retains the initial fusion feature. In this way, the initial fusion feature can better express the deep features of the implicit label error through feature fusion under the triple attention mechanism. In addition, the multi-scale approach can also learn more detailed features, thereby improving the accuracy of label error recognition.

[0066] For further information, see Figure 2 , convolve and nonlinearly transform the first fusion feature to obtain a heat map, and use the heat map as a label error probability map. Specifically, convolve and nonlinearly transform the high-dimensional first fusion feature to achieve dimensionality reduction and obtain a one-dimensional heat map, which reflects the probability of incorrect labeling of each pixel. The range of error probability is (-1,1), where the area with larger absolute values ​​clearly indicates the false positive and false negative information in the noise label. False positives and false negatives are common types of errors in classification models and are used to evaluate the performance of the model. False Positive (FP): refers to the model mistakenly predicting a sample that is actually a negative class as a positive class, which is the mislabeled information in the noise label. False Negative (FN): refers to the model mistakenly predicting a sample that is actually a positive class as a negative class, which is the missing label information in the noise label. The label error probability map can be expressed by the following formula:

[0067] I=F i (X,Y)

[0068] Among them, I represents the label error probability map; X represents the remote sensing image; Y represents the noise label; F i It means that the features of the remote sensing image X and the noise label Y are extracted respectively, and the extracted features are fused under the triple attention mechanism to obtain the label error probability map I.

[0069] S12. According to the label error probability map, the first fused feature is fused with the label feature corresponding to the noise label to obtain a second fused feature and an updated label.

[0070] Specifically, first, the dimension of the label feature and the dimension of the label error probability map are adjusted to make the dimensions of the label feature, the label error probability map and the first fusion feature consistent. In this embodiment, a convolution operation can be used to expand the dimension of the label feature and the dimension of the label error probability map to make the dimensions of the label feature, the label error probability map and the first fusion feature consistent, which is convenient for subsequent fusion calculations.

[0071] Furthermore, the label error probability map with consistent dimensions is aligned with the first fusion feature and added to obtain an intermediate fusion feature. Furthermore, a dot multiplication operation is performed on the eigenvalue of the intermediate fusion feature and the error probability value of the label error probability map, and a dot multiplication operation is performed on the eigenvalue of the label feature after dimension adjustment and the correct probability value of the label error probability map, and the results of the two dot multiplication operations are added to obtain a second fusion feature. Furthermore, the second fusion feature is convolved, nonlinearly transformed, and binarized to obtain an updated label. Specifically, the high error probability appearing in the error probability map suggests that the noise label may have errors, and the label of the corresponding pixel point needs to be updated. The low error probability appearing in the error probability map suggests that the noise label may be correct, and the label of the corresponding pixel point will basically be retained. Specifically, the updated label is calculated by the following formula:

[0072] U=I×F u (X,Y,I)+(1-I)×Y

[0073] Among them, U represents the update label; F u Indicates that the first fusion feature and the label error probability map are fused to obtain the intermediate fusion feature, where the first fusion feature includes the image feature corresponding to the remote sensing image X and the label feature corresponding to the noise label Y; I×F u (X,Y,I) represents the dot product operation between the feature value of the intermediate fusion feature and the error probability value of the label error probability map, and (1-I)×Y represents the dot product operation between the feature value of the label feature corresponding to the noise label and the correct probability value of the label error probability map.

[0074] For example, in the noise label, the initial label of a certain pixel is "road". If the label error probability map shows that the error probability of the pixel is 0.8, after calculation by the above formula, the updated label U may adjust the label of the pixel, for example, correct the label of the pixel to a ground feature category that is more in line with the actual situation, that is, adjust the color value of the pixel accordingly; if the label error probability map shows that the error probability of the pixel is 0.2, it means that the label of the pixel is relatively accurate. After calculation by the above formula, the updated label U basically retains the initial label of the pixel. In this way, by combining the label error probability map and the first fusion feature, it is determined whether to update or retain the initial label of each pixel in the noise label, thereby improving the overall accuracy of the updated label.

[0075] S13. According to the second fusion feature and the residual correction strategy, the update label is corrected to obtain the target label.

[0076] In this embodiment, first, the dimension of the update label can be adjusted through a convolution operation, and the update label with the same dimension and the second fusion feature are fused, that is, aligned and added to obtain a correction matrix. The correction matrix and the second fusion feature are aligned and added to obtain a third fusion feature. The third fusion feature is decoded, that is, convolution, nonlinear transformation and binarization operations are performed to obtain the target label. Specifically, the update label can be corrected using the following formula:

[0077] Y′=U+F r (X,Y,I,U)

[0078] Among them, Y' represents the target label; F u Indicates that the second fusion feature and the updated label are fused to obtain a correction matrix, where the second fusion feature is the image feature corresponding to the remote sensing image X, the label feature corresponding to the noise label Y, and the fusion feature of the label error probability map. In this way, the updated label U is fine-tuned through the correction matrix to obtain the refined target label Y'. After fine-tuning, the target label can more accurately reflect the actual situation, correct the small holes and small noise points in the noise label, significantly improve the accuracy and reliability of the label, and thus help improve the overall performance of the large model in the semantic segmentation task.

[0079] It should be noted that, please continue to refer to Figure 2 , the third fusion feature can also be decoded by a decoding operation to obtain an error label. By observing the error label, the error area in the noise label can be intuitively known, which is convenient for manual direct adjustment of the noise label or taking the next step.

[0080] In this embodiment, through the above method steps, the fine identification, updating and refinement of noise labels are achieved, the sample data quality is improved, and it is conducive to the training and performance improvement of the large model of remote sensing image intelligent interpretation. The experimental results show that the label refinement method is significantly better than the existing technology in terms of accuracy and efficiency of label refinement, and has broad application prospects.

[0081] In one embodiment, the present application also provides a method for training a label refinement model. Figure 5 , the training method comprises the following steps:

[0082] S50, obtaining a sample data set, the sample data set including a sample remote sensing image, a sample noise label and a sample true value label; the sample noise label includes an error or omission label;

[0083] Specifically, the training of the label refinement model is supervised by the sample true value labels. In addition, the sample data set can be divided into a training set, a test set, and a validation set. The training set is used to train the label refinement model, and the test set is used to adjust the label refinement model.

[0084] S51, construct a pre-trained label refinement model, input the sample remote sensing image and the sample noise label into the pre-trained label refinement model for training, and obtain a predicted refined label.

[0085] Specifically, the pre-trained label refinement model includes an error recognition module, a label update module and a label correction module. The error recognition module includes two feature extractors, hidden layers and a multi-scale triple attention mechanism module. The error recognition module is used to extract features from remote sensing images and noise labels respectively, and fuse the extracted features under the triple attention mechanism to obtain a first fused feature and a label error probability map. The label update module includes several hidden layers. The label update module is used to fuse and calculate the label features corresponding to the first fused feature and the noise label according to the label error probability map to obtain a second fused feature and an updated label. The label correction module includes several hidden layers. The label correction module is used to correct the updated label according to the second fused feature and the residual correction strategy to obtain a predicted refined label. It should be noted that the hidden layers all contain convolution kernels and nonlinear activation functions to realize the functions of the corresponding modules.

[0086] S52. Calculate the loss value between the predicted refined label and the sample true value label according to a preset loss function. In this embodiment, the preset loss function is shown as follows:

[0087] L total =λ identify L identify +λ update L update +λ refine L refine

[0088] Among them, L total represents the overall loss of the label refinement model, L identify represents the loss of the error recognition module, L update represents the loss of the label update module, L refine represents the loss of the label correction module. identify , update , refine represents the weight coefficient of each loss term. It should be noted that L identify , L update , L refine The three loss terms all combine cross entropy loss and Dice loss to adapt to the feature distribution of the image.

[0089] S53, according to the loss value, back-propagate the pre-trained label refinement model to adjust the model weight coefficient until the loss value is less than a preset threshold. Specifically, according to the loss value, back-propagate the parameter gradient of the label refinement model to adjust the model weight coefficient. Specifically, in this embodiment, L is finally determined by the model performance in the cross-validation process. identify , L update , L refine The ratio is 2:1:1.

[0090] In one embodiment, the present application provides a label refinement device, which corresponds one-to-one to a label refinement method in the above embodiment. Figure 6 As shown, the label refinement device 600 includes a data acquisition unit 610, an error identification unit 620, a label update unit 630 and a label correction unit 640. The functional units are described in detail as follows:

[0091] The data acquisition unit 610 is used to acquire remote sensing images and noise labels, and input them into the trained label refinement model for label refinement processing.

[0092] The error identification unit 620 is used to extract features from the remote sensing image and the noise label respectively, and fuse the extracted features under the triple attention mechanism to obtain a first fused feature and a label error probability map.

[0093] The label updating unit 630 is used to perform a fusion calculation on the first fusion feature and the label feature corresponding to the noise label according to the label error probability map to obtain a second fusion feature and an updated label.

[0094] The label correction unit 640 is used to correct the updated label according to the second fusion feature and the residual correction strategy to obtain the target label.

[0095] In one embodiment, a computing device is provided. The computing device may be a server, and its internal structure diagram may be as follows: Figure 7 As shown. The computing device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computing device is used to provide computing and control capabilities. The memory of the computing device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computing device is used to store data such as remote sensing images, noise labels, label refinement models and target labels. The network interface of the computing device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps of a label refinement method in the above embodiment are implemented.

[0096] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of a label refinement method in the above embodiment are implemented.

[0097] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A label refinement method, characterized in that: include: Acquire remote sensing images and noise labels, and input them into the trained label refinement model for label refinement processing; wherein the noise labels include potential erroneous labels; the label refinement process includes: Extracting features from the remote sensing image and the noise label respectively, and fusing the extracted features under a triple attention mechanism to obtain a first fused feature and a label error probability map; According to the label error probability map, the first fused feature and the label feature corresponding to the noise label are fused and calculated to obtain a second fused feature and an updated label; According to the second fusion feature and the residual correction strategy, the updated label is corrected to obtain a target label.

2. The label refinement method according to claim 1, characterized in that: The noise label is used to characterize the ground object category of the pixel point corresponding to the remote sensing image, and the input form of the noise label is a binary image.

3. The label refinement method according to claim 1, characterized in that: The trained label refinement model includes two feature extractors; the remote sensing image and the noise label are respectively subjected to feature extraction, and the extracted features are fused under a triple attention mechanism to obtain a first fused feature and a label error probability map, including: Using the two feature extractors, extracting features from the remote sensing image and the noise label respectively to obtain image features and label features with consistent dimensions; Aligning and subtracting the image features and the label features to obtain initial fusion features; Perform multi-scale feature fusion on the initial fused features under the triplet attention mechanism to obtain a first fused feature; The first fusion feature is convolved and nonlinearly transformed to obtain a heat map, and the heat map is used as the label error probability map.

4. The label refinement method according to claim 3, characterized in that: The initial fusion feature is subjected to multi-scale feature fusion under the triple attention mechanism to obtain a first fusion feature, including: Downsampling the initial fusion feature to obtain sub-fusion features of multiple scales; The sub-fusion features of each scale are fused under the triplet attention mechanism, and the sub-fusion features of multiple scales after learning are upsampled and spliced ​​to obtain the first fusion feature.

5. The label refinement method according to claim 4, characterized in that: The step of fusing the sub-fusion features of each scale under the triplet attention mechanism includes: The sub-fusion feature is rotated horizontally and vertically respectively to obtain a first input feature and a second input feature, and the sub-fusion feature itself is used as a third input feature; The first input feature, the second input feature and the third input feature are sequentially subjected to maximum pooling, convolution, normalization and nonlinear transformation to obtain corresponding first output features, second output features and third output features; The first output feature, the second output feature and the third output feature are averaged to obtain a sub-fusion feature after feature fusion.

6. The label refinement method according to any one of claims 1 to 5, characterized in that: The step of fusing the first fused feature and the label feature corresponding to the noise label according to the label error probability map to obtain a second fused feature and an updated label includes: Adjusting the dimension of the label feature and the dimension of the label error probability map so that the dimensions of the label feature, the label error probability map and the first fusion feature are consistent; Aligning the label error probability map with the same dimension with the first fusion feature and adding them to obtain an intermediate fusion feature; Performing a dot product operation on the feature value of the intermediate fusion feature and the error probability value of the label error probability map, performing a dot product operation on the feature value of the label feature after dimension adjustment and the correct probability value of the label error probability map, and adding the results of the two dot product operations to obtain a second fusion feature; The second fused feature is convolved, nonlinearly transformed, and binarized to obtain an updated label.

7. The label refinement method according to any one of claims 1 to 5, characterized in that: The step of correcting the updated label according to the second fusion feature and the residual correction strategy to obtain the target label includes: Adjusting the dimension of the update label, and aligning and adding the update label and the second fusion feature with the same dimension to obtain a correction matrix; Aligning and adding the correction matrix and the second fusion feature to obtain a third fusion feature; The third fusion feature is convolved, nonlinearly transformed, and binarized to obtain a target label.

8. A method for training a label refinement model, characterized in that: include: Acquire a sample data set, wherein the sample data set includes a sample remote sensing image, a sample noise label, and a sample true value label; The sample noise labels include erroneous or missing labels; Constructing a pre-trained label refinement model, inputting the sample remote sensing image and the sample noise label into the pre-trained label refinement model for training, and obtaining a predicted refined label; Calculate the loss value between the predicted refined label and the sample true value label according to a preset loss function; According to the loss value, the pre-trained label refinement model is back-propagated to adjust the model weight parameters until the loss value is less than a preset threshold.

9. A computing device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images