Noisy label robust learning method, system and device for remote sensing change detection

Abstract

The present invention discloses a noise label robust learning method, system and device for remote sensing change detection, proposes an edge-guided cross-level difference enhancement module, improves the edges of low-level difference features through low-level-high-level feature interaction, and uses the difference features after edge improvement to guide high-level dual-phase features for difference enhancement, fully mines the edge information and change information of dual-phase images, and a noise label joint detection strategy based on uncertainty analysis combines the uncertainty value and prediction probability of samples to identify potential noise labels and correct them, thereby reducing the risk of identifying difficult samples as noise samples; a noise robust loss function for adaptive enhancement of difficult samples is designed, and loss weights are adaptively assigned to samples based on the uncertainty value of samples. The larger the uncertainty value, the more difficult the sample learning and the larger the loss weight. In this way, the network is forced to learn more complex and difficult feature representations, thereby enhancing the robustness of the model to noise labels.

Description

Technical Field

[0001] The present invention belongs to the technical field of image change detection, and in particular relates to a noise label robust learning method, system and device for remote sensing change detection. Background Art

[0002] Changes in land cover types caused by natural and human activities have attracted widespread attention. Accurately and quickly extracting information on land changes from remote sensing images is of great value in guiding the efficient use of land resources, urban planning, disaster risk assessment, resource management monitoring, and ecological environment monitoring. Remote sensing image change detection is a technology that uses mathematical analysis models to identify and quantify the process of land change by analyzing remote sensing images of the same area at different times. With the significant improvement of remote sensing satellite technology in space, spectrum and time, change detection technology has ushered in unprecedented development opportunities. These technological advances have put forward higher standards for data processing, feature extraction and change analysis, and promoted the rapid transformation of change detection technology to automation and intelligence.

[0003] Traditional remote sensing image change detection methods are mainly divided into three categories: methods based on algebraic calculations, methods based on image transformation, and methods based on post-classification comparison. However, these methods are usually based on manually designed feature representations, with limited ability to capture change information and unable to cope with complex change scenarios. In recent years, thanks to the powerful semantic extraction and representation capabilities of neural networks, a large number of remote sensing image change detection algorithms based on deep learning have been proposed for detecting surface changes. For example, some scholars proposed a dual-branch multi-level cross-temporal network, combined self-attention and cross-attention to design a cross-temporal joint attention module to fully mine the global feature information of the input image, promote the coupling of feature information at the same level, and suppress interference irrelevant to the task. Other scholars have improved the structural detail information of the changed area through temporal feature interaction and guided refinement, captured temporal difference information at different feature levels, and aggregated low-level and high-level temporal difference information by repeatedly performing guided refinement operations, emphasizing the high-level feature position information while reducing irrelevant background interference of low-level features. In order to further enhance the interaction between difference information and phase information, some scholars designed a difference-guided multi-scale aggregation attention network, established a multi-scale difference fusion module to fuse dual-phase features, and then used two difference aggregation modules to aggregate the difference features at different levels to achieve multi-level feature interaction. Although these methods have shown excellent performance in change detection tasks, they ignore the importance of edge information. In order to improve the accuracy of change edge extraction, some scholars introduced an edge improvement module in the network to perceive and improve the edges of the change area. The results show that it performs better in edge detection. Some scholars proposed an edge-guided strategy to inject edge information when fusing features at all levels, which effectively improves the edges of the change area.

[0004] Although remote sensing image change detection algorithms based on deep learning have achieved great success, the performance of deep learning networks is highly dependent on massive amounts of high-quality labeled data. In actual scenarios, due to the lack of prior information, inconsistent data processing, and subjectivity in manual labeling, sample data often contains an indefinite proportion of noise labels (noise labels refer to labels that are incorrectly labeled and inconsistent with the actual situation). If such erroneous label information is directly used to supervise the network, it will affect the training optimization of the model and reduce its performance. Therefore, how to minimize the impact of noise labels and achieve optimal model training in the presence of noise labels has become one of the urgent problems to be solved in the field of change detection. Summary of the invention

[0005] In view of the above technical problems, the present invention provides a noise label robust learning method, system and device for remote sensing change detection.

[0006] The technical solution adopted by the present invention to solve the technical problem is:

[0007] A noisy label robust learning method for remote sensing change detection, the method comprising the following steps:

[0008] S100: Build a remote sensing image change detection model, including a backbone network, an edge-guided cross-level difference enhancement module, a progressive multi-level feature fusion module, and a noise label joint detection module;

[0009] S200: Obtain dual-temporal remote sensing images and input them into the backbone network to extract multi-level dual-temporal features from the dual-temporal remote sensing images , , obtain the original difference features of the first 1 to N-1 levels by difference ;

[0010] S300: The original difference features of the first 1 to N-1 levels Biphasic characteristics with 2 to N levels Feed it to the edge-guided cross-level difference enhancement module, which improves the original difference features of the first 1 to N-1 levels through multi-scale dilated convolution The edge of the edge is improved to obtain the difference feature , and use the difference features after the edge improvement of the current level Leading to the next level of bi-phase features , Perform difference enhancement to obtain enhanced difference features between the two levels and ;

[0011] S400: Enhanced difference between the two levels and They are input into the progressive feature fusion module respectively, and the intermediate features of each layer in the fusion process are taken out and added to obtain the final change features of the N-1 layer, which are sent to the classifier to obtain the prediction probability of each category, and the category with the largest prediction probability is selected as the change detection result to obtain the corresponding N-1 change detection results;

[0012] S500: The noise label joint detection strategy based on uncertainty analysis performs uncertainty analysis on N-1 change detection results to obtain the uncertainty value of the sample, screens the samples based on the uncertainty value and the predicted probability, and compares the screened samples with the original labels to obtain noise samples and correct samples;

[0013] S600: Design a noise robust loss function for adaptive enhancement of difficult samples, including a change supervision loss function and an edge supervision loss function, to train a remote sensing image change detection model. When a preset training end condition is reached, a trained remote sensing image change detection model is obtained, which is used to perform change detection on dual-phase remote sensing images acquired in real time.

[0014] Preferably, in S300, the edges of the original difference features of the first N-1 levels are improved by multi-scale dilated convolution to obtain features after edge improvement, including:

[0015] S310: Use parallel multi-scale dilated convolution to improve the edge of the original difference feature to capture the difference features with different scale receptive fields;

[0016] S320: After the convolution operation is completed, batch normalization and nonlinear activation processing are performed, and the multi-scale dilated convolution results are aggregated in the channel dimension;

[0017] S330: Perform edge refinement and extraction through 1×1 convolution, use Sigmoid function to obtain edge weights, and use edge weights to assign weights to original difference features to obtain edge features , and finally add it to the original difference feature to get the difference feature after edge improvement .

[0018] Preferably, in S300, the edge of the original difference feature of the first N-1 levels is improved by multi-scale dilated convolution to obtain the feature after edge improvement, specifically:

[0019] ;

[0020] ;

[0021] ;

[0022] in represents a 3×3 convolution kernel with a dilation rate and a filling rate of r, and Respectively represent BatchNorm batch normalization and ReLU activation function, represents splicing along the channel dimension, is the Sigmoid activation function, represents the original difference feature, represents edge features, represents the difference feature after edge improvement, Represents the results of multi-scale dilated convolution.

[0023] Preferably, in S300, the difference feature after edge improvement at the current level is used Leading to the next level of bi-phase features , Perform difference enhancement to obtain enhanced difference features between the two levels and ,include:

[0024] S340: Difference features after edge improvement and bi-phase features from the next level , Feature embedding is obtained through convolution and reshape operations ;

[0025] S350: Calculate separately using dot multiplication and , The similarity between them enables the dual-phase features to focus on richer edge information of changes;

[0026] S360: The weight matrix is ​​obtained through the softmax operation, and the weight matrix is ​​used to Weighted, and finally two enhanced difference features are obtained by splicing and difference and .

[0027] Preferably, S350 and S360 are specifically:

[0028] ;

[0029] ;

[0030] Where T represents the matrix transpose; and Indicates the next level of bi-phase characteristics; Indicates absolute value calculation.

[0031] Preferably, S500 includes:

[0032] S510: using KL divergence to measure the difference between the N-1 change detection results, and using the obtained difference result as the uncertainty value of the sample;

[0033] S520: sorting the uncertainty values ​​of all samples from low to high, selecting a preset number of samples before sorting, further screening the samples using the predicted probability and probability threshold, retaining samples with a predicted probability greater than the probability threshold, and obtaining low uncertainty and high predicted probability samples;

[0034] S530: Compare the prediction result of the low uncertainty and high prediction probability sample with the original label. If they are consistent, the original label is the correct sample, otherwise the original label is a noise sample. Correct the original label of the noise sample according to the prediction result, and use the corrected label for the next round of training.

[0035] Preferably, S500 is specifically:

[0036] ;

[0037] ;

[0038] ;

[0039] in, and Represent the predicted output and average output of the network respectively; represents the KL divergence calculation, It is calculated by KL divergence and is the uncertainty value of the sample; Represents the filtered sub-dataset, express The mth image sample in , represents the pixel at position (h, w) in the mth image sample, and They represent the predicted category and label category of the pixel position (h, w) in the mth image sample, respectively.

[0040] Preferably, S600 specifically includes:

[0041] ;

[0042] ;

[0043] ;

[0044] ;

[0045] ;

[0046] in represents the prediction result of the pixel position (h, w), is the change label corresponding to the pixel position, is the cross entropy loss corresponding to the pixel position, is the uncertain value corresponding to the pixel position, H and W represent the height and width of the image respectively. represents the edge label, and The horizontal and vertical convolution kernels, represents the L1 norm loss function, represents the edge detection result, represents the change supervision loss, represents the marginal supervision loss.

[0047] A noisy label robust learning system for remote sensing change detection, including a remote sensing image change detection model building module, a backbone network, an edge-guided cross-level difference enhancement module, a progressive multi-level feature fusion module, a noisy label joint detection module, a model training and a noisy label detection module;

[0048] Remote sensing image change detection model building module, which is used to build a remote sensing image change detection model, including a backbone network, an edge-guided cross-level difference enhancement module, a progressive multi-level feature fusion module, and a noise label joint detection module;

[0049] Backbone network, used to obtain dual-phase remote sensing images and extract multi-level dual-phase features from dual-phase remote sensing images , , obtain the original difference features of the first 1 to N-1 levels by difference ;

[0050] Edge-guided cross-level difference enhancement for improving the original difference features of the first 1 to N-1 levels through multi-scale dilated convolutions The edge of the edge is improved to obtain the difference feature , and use the difference features after the edge improvement of the current level Leading to the next level of bi-phase features , Perform difference enhancement to obtain enhanced difference features between the two levels and ;

[0051] A progressive feature fusion module is used to receive the enhanced difference features between the two levels. and , take out the intermediate features of each layer in the fusion process and add them to get the final change features of the N-1 layer, send them to the classifier to get the prediction probability of each category, select the category with the largest prediction probability as the change detection result, and get the corresponding N-1 change detection results;

[0052] The noise label joint detection module is used to perform uncertainty analysis on N-1 change detection results based on the noise label joint detection strategy of uncertainty analysis, screen samples based on uncertainty values ​​and prediction probabilities, and compare the screened samples with the original labels to obtain noise samples and correct samples;

[0053] The model training and noise label detection module is used to design a noise robust loss function for adaptive enhancement of difficult samples, including a change supervision loss function and an edge supervision loss function, to train the remote sensing image change detection model. When the preset training end conditions are reached, a trained remote sensing image change detection model is obtained, which is used to detect noise labels for dual-phase remote sensing images acquired in real time.

[0054] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned noise label robust learning method for remote sensing change detection when executing the computer program.

[0055] The above-mentioned noise label robust learning method, system and device for remote sensing change detection proposes an edge-guided cross-level difference enhancement module, which improves the edges of low-level difference features through low-level-high-level feature interaction, and uses the difference features after edge improvement to guide high-level dual-phase features for difference enhancement, fully mining the edge information and change information of dual-phase images, and proposes a noise label joint detection strategy based on uncertainty analysis, which combines the uncertainty value and prediction probability of the sample to identify potential noise labels and correct them, thereby reducing the risk of identifying difficult samples (difficult samples refer to samples with correct labels but difficult model learning and easy to classify incorrectly) as noise samples; a noise robust loss function for adaptive enhancement of difficult samples is designed, which adaptively assigns loss weights to samples based on the uncertainty value of the sample. The larger the uncertainty value, the more difficult the sample learning and the larger the loss weight. In this way, the network is forced to learn more complex and difficult feature representations, thereby enhancing the robustness of the model to noise labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 is a flow chart of a noise label robust learning method for remote sensing change detection in one embodiment of the present invention;

[0057] Figure 2 is a flow chart of a noise label robust learning method for remote sensing change detection in another embodiment of the present invention;

[0058] Figure 3 Schematic diagram of the principle of an edge-guided cross-level difference enhancement module in one embodiment of the present invention;

[0059] Figure 4 Schematic diagram of the principle of a progressive multi-level feature fusion module in one embodiment of the present invention;

[0060] Figure 5 The figure is a flow chart of a noise label joint detection strategy based on uncertainty analysis in one embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings.

[0062] In one embodiment, Figure 1 and Figure 2 As shown, a noisy label robust learning method for remote sensing change detection comprises the following steps:

[0063] S100: Build a remote sensing image change detection model, including a backbone network, an edge-guided cross-level difference enhancement module, a progressive multi-level feature fusion module, and a noise label joint detection module;

[0064] S200: Obtain dual-temporal remote sensing images and input them into the backbone network to extract multi-level dual-temporal features from the dual-temporal remote sensing images , , obtain the original difference features of the first 1 to N-1 levels by difference ;

[0065] S300: The original difference features of the first 1 to N-1 levels Biphasic characteristics with 2 to N levels Feed it to the edge-guided cross-level difference enhancement module, which improves the original difference features of the first 1 to N-1 levels through multi-scale dilated convolution The edge of the edge is improved to obtain the difference feature , and use the difference features after the edge improvement of the current level Leading to the next level of bi-phase features , Perform difference enhancement to obtain enhanced difference features between the two levels and ;

[0066] S400: Enhanced difference between the two levels and They are input into the progressive feature fusion module respectively, and the intermediate features of each layer in the fusion process are taken out and added to obtain the final change features of the N-1 layer, which are sent to the classifier to obtain the prediction probability of each category, and the category with the largest prediction probability is selected as the change detection result to obtain the corresponding N-1 change detection results;

[0067] S500: The noise label joint detection strategy based on uncertainty analysis performs uncertainty analysis on N-1 change detection results to obtain the uncertainty value of the sample, screens the samples based on the uncertainty value and the predicted probability, and compares the screened samples with the original labels to obtain noise samples and correct samples;

[0068] S600: Design a noise robust loss function for adaptive enhancement of difficult samples, including a change supervision loss function and an edge supervision loss function, to train a remote sensing image change detection model. When a preset training end condition is reached, a trained remote sensing image change detection model is obtained, which is used to perform change detection on dual-phase remote sensing images acquired in real time.

[0069] Specifically, in the task of remote sensing image change detection, due to the influence of external environmental factors such as lighting conditions, cloud occlusion or seasonal changes, the acquired dual-phase images may have significant differences even for the same type of ground objects. Especially in the edge area, it is often difficult to define the change edge in the subsequent annotation process, which is easy to be incorrectly labeled and introduce noise labels. Therefore, in order to fully explore the change information and edge information of the dual-phase images and improve the model's ability to detect change edges. The present invention proposes an edge-guided cross-level difference enhancement module, which uses low-level features of edge improvement to guide high-level dual-phase features for difference enhancement. At the same time, in order to reduce the risk of identifying difficult samples as noise samples and improve the detection effect of noise samples, a noise label joint detection strategy based on uncertainty analysis is proposed. In addition, in order to further enhance the robustness of the network model to noisy labels, a noise-robust loss function with adaptive enhancement of difficult samples is designed. The uncertainty value of the sample is introduced as a constraint term into the variation supervision loss, and the uncertainty value is gradually reduced through back propagation optimization, that is, the confidence of the prediction result is improved; at the same time, adaptive enhancement of difficult samples is achieved based on the uncertainty value of the sample. The larger the uncertainty value, the more difficult the sample learning is and the larger the loss weight is, forcing the network model to learn more complex and difficult feature representations, thereby enhancing its noise robustness.

[0070] Furthermore, in this embodiment, N=4 is used as an example to illustrate that first, the backbone network is used to extract multi-level dual-phase features from the dual-phase remote sensing image. , . Obtain the original difference features by difference , the original difference features of the first three levels (i.e., i=1,2,3) and the bi-temporal features of the last three levels (i.e., i=2,3,4) are fed into the proposed edge-guided cross-level difference enhancement module.

[0071] In one embodiment, Figure 3 As shown, in S300, the edges of the original difference features of the first N-1 levels are improved by multi-scale dilated convolution to obtain edge-improved features, including:

[0072] S310: Use parallel multi-scale dilated convolution to improve the edge of the original difference feature to capture the difference features with different scale receptive fields;

[0073] S320: After the convolution operation is completed, batch normalization and nonlinear activation processing are performed, and the multi-scale dilated convolution results are aggregated in the channel dimension;

[0074] S330: Perform edge refinement and extraction through 1×1 convolution, use Sigmoid function to obtain edge weights, and use edge weights to assign weights to original difference features to obtain edge features , and finally add it to the original difference feature to get the difference feature after edge improvement .

[0075] In one embodiment, in S300, the edge of the original difference feature of the first N-1 levels is improved by multi-scale dilated convolution to obtain the feature after edge improvement, specifically:

[0076] ;

[0077] ;

[0078] ;

[0079] in represents a 3×3 convolution kernel with a dilation rate and a filling rate of r, and Respectively represent BatchNorm batch normalization and ReLU activation function, represents splicing along the channel dimension, is the Sigmoid activation function, represents the original difference feature, represents edge features, represents the difference feature after edge improvement, Represents the results of multi-scale dilated convolution.

[0080] Specifically, parallel multi-scale dilated convolution is used to transform the original difference features Edge improvements are performed to capture differential features with receptive fields of different scales. At the same time, in order to reduce the risk of gradient explosion and ensure stable operation of the model, batch normalization and nonlinear activation processing are required after each convolution operation. Then, the multi-scale dilated convolution results are aggregated in the channel dimension to achieve more compact spatial structure information extraction. Edges are refined and extracted through 1×1 convolution, and the edge weights are obtained using the Sigmoid function. The edge weights are used to weight the original difference features to obtain edge features. , and finally add it to the original difference feature to get the edge-improved difference feature.

[0081] In one embodiment, Figure 3 As shown, the difference feature after the edge improvement of the current level is used in S300 Leading to the next level of bi-phase features , Perform difference enhancement to obtain enhanced difference features between the two levels and ,include:

[0082] S340: Difference features after edge improvement and bi-phase features from the next level , Feature embedding is obtained through convolution and reshape operations ;

[0083] S350: Calculate separately using dot multiplication and , The similarity between them enables the dual-phase features to focus on richer edge information of changes;

[0084] S360: The weight matrix is ​​obtained through the softmax operation, and the weight matrix is ​​used to Weighted, and finally two enhanced difference features are obtained by splicing and difference and .

[0085] In one embodiment, S350 and S360 are specifically:

[0086] ;

[0087] ;

[0088] Where T represents the matrix transpose; and Indicates the next level of bi-phase characteristics; Indicates absolute value calculation.

[0089] Specifically, considering that the low-level features extracted by the neural network have richer edge information, but the location of the change area is inaccurate, while the high-level features have accurate location of the change area, but lack edge information. Therefore, we further enhance the interaction between low-level and high-level features, and use the difference features after edge improvement to guide the next-level dual-phase features for difference enhancement. First, the difference features after edge improvement and the dual-phase features from the next level are embedded through convolution and reshape operations. , using dot multiplication to calculate and , The similarity between them enables the dual-phase feature to focus on richer edge information of changes, and then obtains the weight matrix through the softmax operation, and uses the weight matrix to Finally, two enhanced difference features are obtained by splicing and difference. and .

[0090] Furthermore, if Figure 4 As shown, and They are input into the feature fusion module of the current level respectively, and the intermediate features are added to obtain the change features of the current level. The intermediate features of this level are respectively and The intermediate features are concatenated and input to the feature fusion module of the next level, and the intermediate features are added to obtain the change features of the current level. The intermediate features of this level are respectively and The results are spliced ​​and input into the feature fusion module of the next level, and the output features are added to obtain the change features of this level. The change detection results of each level are input into the classifier to obtain the corresponding three change detection results.

[0091] In one embodiment, S500 includes:

[0092] S510: using KL divergence to measure the difference between the N-1 change detection results, and using the obtained difference result as the uncertainty value of the sample;

[0093] S520: sorting the uncertainty values ​​of all samples from low to high, selecting a preset number of samples before sorting, further screening the samples using the predicted probability and probability threshold, retaining samples with a predicted probability greater than the probability threshold, and obtaining low uncertainty and high predicted probability samples;

[0094] S530: Compare the prediction result of the low uncertainty and high prediction probability sample with the original label. If they are consistent, the original label is the correct sample, otherwise the original label is a noise sample. Correct the original label of the noise sample according to the prediction result, and use the corrected label for the next round of training.

[0095] Specifically, in order to reduce the risk of identifying difficult samples as noise samples and improve the detection effect of noise samples, a noise label joint detection strategy based on uncertainty analysis is proposed. Specifically, the KL divergence is used to measure the difference between the three change detection results, and the obtained difference results are used as the uncertainty value of the sample. The greater the difference, the greater the uncertainty and the lower the prediction confidence; conversely, the smaller the difference, the smaller the uncertainty and the higher the prediction confidence. By first selecting samples with low uncertainty, it can be ensured as much as possible that the selected samples are predicted correctly, and then the predicted probability is used to further screen the samples to select samples with high confidence to the greatest extent, avoiding the problem of incorrect detection due to network prediction errors. After the sample screening is completed, the prediction results of the selected samples are compared with the original labels. If they are consistent, they are correct samples, otherwise they are noise samples. Finally, the noise samples are corrected according to the prediction results, and then the corrected labels are used for the next round of training. By gradually correcting and updating the labels, the number of noise samples is reduced, and their impact on the optimization of network model training is minimized. The entire detection process is as follows. Figure 5 shown.

[0096] In one embodiment, S500 specifically includes:

[0097] ;

[0098] ;

[0099] ;

[0100] in, and Represent the predicted output and average output of the network respectively; represents the KL divergence calculation, It is calculated by KL divergence and is the uncertainty value of the sample; Represents the filtered sub-dataset, express The mth image sample in , represents the pixel at position (h, w) in the mth image sample, and They represent the predicted category and label category of the pixel position (h, w) in the mth image sample, respectively.

[0101] In one embodiment, S600 specifically includes:

[0102] ;

[0103] Taking one of the change detection results as an example, the change supervision loss is as follows:

[0104] ;

[0105] ;

[0106] In order to further enhance the network model's ability to detect changing edges, the L1 norm loss is used to supervise the intermediate edge features generated by the edge-guided cross-level difference enhancement module. The Sobel operator is used to extract the edge of the changing label image as a label. For example, the optimization objectives are as follows:

[0107] ;

[0108] in represents the prediction result of the pixel position (h, w), is the change label corresponding to the pixel position, is the cross entropy loss corresponding to the pixel position, is the uncertain value corresponding to the pixel position, H and W represent the height and width of the image respectively. represents the edge label, and The horizontal and vertical convolution kernels, represents the L1 norm loss function, represents the edge detection result, represents the change supervision loss, represents the marginal supervision loss.

[0109] In one embodiment, a noise label robust learning system for remote sensing change detection includes a remote sensing image change detection model building module, a backbone network, an edge-guided cross-level difference enhancement module, a progressive multi-level feature fusion module, a noise label joint detection module, a model training and a noise label detection module;

[0110] Remote sensing image change detection model building module, which is used to build a remote sensing image change detection model, including a backbone network, an edge-guided cross-level difference enhancement module, a progressive multi-level feature fusion module, and a noise label joint detection module;

[0111] Backbone network, used to obtain dual-phase remote sensing images and extract multi-level dual-phase features from dual-phase remote sensing images , , obtain the original difference features of the first 1 to N-1 levels by difference ;

[0112] Edge-guided cross-level difference enhancement for improving the original difference features of the first 1 to N-1 levels through multi-scale dilated convolutions The edge of the edge is improved to obtain the difference feature , and use the difference features after the edge improvement of the current level Leading to the next level of bi-phase features , Perform difference enhancement to obtain enhanced difference features between the two levels and ;

[0113] A progressive feature fusion module is used to receive the enhanced difference features between the two levels. and , take out the intermediate features of each layer in the fusion process and add them to get the final change features of the N-1 layer, send them to the classifier to get the prediction probability of each category, select the category with the largest prediction probability as the change detection result, and get the corresponding N-1 change detection results;

[0114] The noise label joint detection module is used to perform uncertainty analysis on N-1 change detection results based on the noise label joint detection strategy of uncertainty analysis, screen samples based on uncertainty values ​​and prediction probabilities, and compare the screened samples with the original labels to obtain noise samples and correct samples;

[0115] The model training and noise label detection module is used to design a noise robust loss function for adaptive enhancement of difficult samples, including a change supervision loss function and an edge supervision loss function, to train the remote sensing image change detection model. When the preset training end conditions are reached, a trained remote sensing image change detection model is obtained, which is used to detect noise labels for dual-phase remote sensing images acquired in real time.

[0116] For the specific definition of the noisy label robust learning system for remote sensing change detection, please refer to the definition of the noisy label robust learning method for remote sensing change detection in the above text, which will not be repeated here. Each module in the above-mentioned noisy label robust learning system for remote sensing change detection can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0117] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned noise label robust learning method for remote sensing change detection when executing the computer program.

[0118] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0119] The performance of deep learning networks is highly dependent on massive amounts of high-quality labeled data. However, in real scenarios, sample data often contains varying proportions of noise labels, which results in reduced algorithm performance. The present invention proposes a noise label robust learning method, system, and device for remote sensing change detection, which mainly have the following advantages:

[0120] A new noise label detection method is proposed to make up for the defect that the previous small loss strategy is easy to misclassify difficult samples into noise samples. Combined with the uncertainty value and prediction probability of the sample, the effective recognition and detection of noise labels is realized; in actual scenes, edge areas are more likely to introduce noise labels, but most existing methods are difficult to achieve effective coordination of change information and edge information. This invention proposes an edge-guided cross-level difference enhancement module, which fully mines the edge information and change information of dual-phase remote sensing images. The network has good robustness to noise labels, and a noise robust loss function for adaptive enhancement of difficult samples is designed. Based on the uncertainty value of the sample, the loss weight is adaptively assigned to the sample. The larger the uncertainty value, the more difficult the sample learning is, and the larger the loss weight is. In this way, the network is forced to learn more complex and more difficult feature representations, enhancing the robustness of the model to noise labels.

[0121] The above is a detailed introduction to the noise label robust learning method, system and device for remote sensing change detection provided by the present invention. This article uses specific examples to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the core idea of ​​the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A noisy label robust learning method for remote sensing change detection, characterized in that The method comprises the following steps: S100: Build a remote sensing image change detection model, including a backbone network, an edge-guided cross-level difference enhancement module, a progressive multi-level feature fusion module, and a noise label joint detection module; S200: Obtain dual-temporal remote sensing images and input them into the backbone network to extract multi-level dual-temporal features from the dual-temporal remote sensing images , , obtain the original difference features of the first 1 to N-1 levels by difference ; S300: The original difference features of the first 1 to N-1 levels Biphasic characteristics with 2 to N levels Feed it to the edge-guided cross-level difference enhancement module, which improves the original difference features of the first 1 to N-1 levels through multi-scale dilated convolution The edge of the edge is improved to obtain the difference feature , and use the difference features after the edge improvement of the current level Leading to the next level of bi-phase features , Perform difference enhancement to obtain enhanced difference features between the two levels and ; S400: Enhanced difference between the two levels and They are input into the progressive feature fusion module respectively, and the intermediate features of each layer in the fusion process are taken out and added to obtain the final change features of the N-1 layer, which are sent to the classifier to obtain the prediction probability of each category, and the category with the largest prediction probability is selected as the change detection result to obtain the corresponding N-1 change detection results; S500: The noise label joint detection strategy based on uncertainty analysis performs uncertainty analysis on N-1 change detection results to obtain the uncertainty value of the sample, screens the samples based on the uncertainty value and the predicted probability, and compares the screened samples with the original labels to obtain noise samples and correct samples; S600: Design a noise robust loss function for adaptive enhancement of difficult samples, including a change supervision loss function and an edge supervision loss function, to train a remote sensing image change detection model. When a preset training end condition is reached, a trained remote sensing image change detection model is obtained, which is used to perform change detection on dual-phase remote sensing images acquired in real time.

2. The method according to claim 1, characterized in that In S300, the edges of the original difference features of the first N-1 levels are improved through multi-scale dilated convolution to obtain edge-improved features, including: S310: Use parallel multi-scale dilated convolution to improve the edge of the original difference feature to capture the difference features with different scale receptive fields; S320: After the convolution operation is completed, batch normalization and nonlinear activation processing are performed, and the multi-scale dilated convolution results are aggregated in the channel dimension; S330: Perform edge refinement and extraction through 1×1 convolution, use Sigmoid function to obtain edge weights, and use edge weights to assign weights to original difference features to obtain edge features , and finally add it to the original difference feature to get the difference feature after edge improvement .

3. The method according to claim 2, characterized in that In S300, the edges of the original difference features of the first N-1 levels are improved by multi-scale dilated convolution to obtain the features after edge improvement, specifically: ； ； ； in represents a 3×3 convolution kernel with a dilation rate and a filling rate of r, and Respectively represent BatchNorm batch normalization and ReLU activation function, represents splicing along the channel dimension, is the Sigmoid activation function, represents the original difference feature, represents edge features, represents the difference feature after edge improvement, Represents the results of multi-scale dilated convolution.

4. The method according to claim 3, characterized in that S300 uses the difference feature after edge improvement at the current level Leading to the next level of bi-phase features , Perform difference enhancement to obtain enhanced difference features between the two levels and ,include: S340: Difference features after edge improvement and bi-phase features from the next level , Feature embedding is obtained through convolution and reshape operations ; S350: Calculate separately using dot multiplication and , The similarity between them enables the dual-phase features to focus on richer edge information of changes; S360: The weight matrix is ​​obtained through the softmax operation, and the weight matrix is ​​used to Weighted, and finally two enhanced difference features are obtained by splicing and difference and .

5. The method according to claim 4, characterized in that S350 and S360 are as follows: ； ； Where T represents the matrix transpose; and Indicates the next level of bi-phase characteristics; Indicates absolute value calculation.

6. The method according to claim 5, characterized in that S500 includes: S510: using KL divergence to measure the difference between the N-1 change detection results, and using the obtained difference result as the uncertainty value of the sample; S520: sorting the uncertainty values ​​of all samples from low to high, selecting a preset number of samples before sorting, further screening the samples using the predicted probability and probability threshold, retaining samples with a predicted probability greater than the probability threshold, and obtaining low uncertainty and high predicted probability samples; S530: Compare the prediction result of the low uncertainty and high prediction probability sample with the original label. If they are consistent, the original label is the correct sample, otherwise the original label is a noise sample. Correct the original label of the noise sample according to the prediction result, and use the corrected label for the next round of training.

7. The method according to claim 6, characterized in that S500 is specifically: ； ； ； in, and Represent the predicted output and average output of the network respectively; represents the KL divergence calculation, It is calculated by KL divergence and is the uncertainty value of the sample; Represents the filtered sub-dataset, express The mth image sample in , represents the pixel at position (h, w) in the mth image sample, and They represent the predicted category and label category of the pixel position (h, w) in the mth image sample, respectively.

8. The method according to claim 7, characterized in that S600 is specifically: ； ； ； ； ； in represents the prediction result of the pixel position (h, w), is the change label corresponding to the pixel position, is the cross entropy loss corresponding to the pixel position, is the uncertain value corresponding to the pixel position, H and W represent the height and width of the image respectively. represents the edge label, and The horizontal and vertical convolution kernels, represents the L1 norm loss function, represents the edge detection result, represents the change supervision loss, represents the marginal supervision loss.

9. A noisy label robust learning system for remote sensing change detection, characterized in that It includes remote sensing image change detection model building module, backbone network, edge-guided cross-level difference enhancement module, progressive multi-level feature fusion module, noise label joint detection module, model training and noise label detection module; Remote sensing image change detection model building module, which is used to build a remote sensing image change detection model, including a backbone network, an edge-guided cross-level difference enhancement module, a progressive multi-level feature fusion module, and a noise label joint detection module; Backbone network, used to obtain dual-phase remote sensing images and extract multi-level dual-phase features from dual-phase remote sensing images , , obtain the original difference features of the first 1 to N-1 levels by difference ; Edge-guided cross-level difference enhancement for improving the original difference features of the first 1 to N-1 levels through multi-scale dilated convolutions The edge of the edge is improved to obtain the difference feature , and use the difference features after the edge improvement of the current level Leading to the next level of bi-phase features , Perform difference enhancement to obtain enhanced difference features between the two levels and ; A progressive feature fusion module is used to receive the enhanced difference features between the two levels. and , take out the intermediate features of each layer in the fusion process and add them to get the final change features of the N-1 layer, send them to the classifier to get the prediction probability of each category, select the category with the largest prediction probability as the change detection result, and get the corresponding N-1 change detection results; The noise label joint detection module is used to perform uncertainty analysis on N-1 change detection results based on the noise label joint detection strategy of uncertainty analysis, screen samples based on uncertainty values ​​and prediction probabilities, and compare the screened samples with the original labels to obtain noise samples and correct samples; The model training and noise label detection module is used to design a noise robust loss function for adaptive enhancement of difficult samples, including a change supervision loss function and an edge supervision loss function, to train the remote sensing image change detection model. When the preset training end conditions are reached, a trained remote sensing image change detection model is obtained, which is used to detect noise labels for dual-phase remote sensing images acquired in real time.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

Images