A label noise removal and spectral distribution modeling joint optimization method

Abstract

The application provides a label noise removal and spectrum distribution modeling combined optimization method, and belongs to the field of hyperspectrum. The method extracts reliable samples of labels through non-uniform spatial sampling, utilizes multi-center spectrum prototypes to constrain classification features, and realizes stable training of a model under a label noise condition through joint optimization of a pixel-by-pixel overlap loss, a class cross-entropy loss and a spectrum prototype contrast loss. The method can dynamically remove noise labels, and does not have an over-cleaning problem. Joint optimization of noise removal and spectrum modeling guarantees low noise of input data, the model has a good regularization effect, and has high practical value.

Description

Technical Field

[0001] This invention belongs to the field of hyperspectral imaging, specifically relating to a joint optimization method for tag noise removal and spectral distribution modeling. Background Technology

[0002] Hyperspectral images offer the advantage of combining image and spectral data, playing a significant role in fine-grained land cover classification tasks. However, current hyperspectral image visualization results struggle to simultaneously represent spatial and spectral information, and human visual interpretation introduces errors, leading to noise in hyperspectral data labels. Noisy labels can be categorized into two types based on their cause: instance-related noise labels and instance-independent noise labels. Instance-related noise labels refer to errors caused by factors related to instance features; for example, if land cover A and land cover B have similar spectral features and their geographical distributions are not significantly different prior, then land cover A and land cover B may be incorrectly labeled. Instance-independent noise labels refer to errors caused by factors unrelated to instance features, manifesting as random errors such as unclear outlines of land covers at class boundaries or random errors during labeling. This impacts supervised training of hyperspectral image classification models, severely hindering robust hyperspectral training. Currently, to address the training problem of hyperspectral image classification models with label noise, existing methods fall into two main categories: one is to remove label noise before training the model, and the other is to regularize the model to improve its noise resistance.

[0003] Label noise removal primarily refers to filtering label noise by utilizing prior knowledge of training samples. For example, it can employ a constrained energy minimization strategy to process extracted highly separable spatial-spectral features to detect and correct noisy labels, or propose using density peak clustering algorithms based on local density decision strategies to identify incorrectly labeled training samples, thus eliminating mislabeled training samples. Alternatively, it can construct a spectral-spatial probability transfer matrix that simultaneously considers spectral similarity and spatial information based on superpixels, and use multiple voting methods to eliminate interference from noisy labels. However, label noise removal methods suffer from over-cleaning, meaning they can easily delete accurately labeled samples.

[0004] Model regularization is a technique used to prevent overfitting of models, specifically preventing them from overfitting to noisy labels during training. In deep learning, due to the large number of model parameters, models are prone to overfitting to noisy labeled data during training, leading to a decline in model performance. Regularization penalizes the complexity of the model, encouraging it to learn simpler patterns, thereby preventing overfitting to noisy labels and improving its generalization ability on test data. In noisy label robust training tasks, label smoothing reduces the probability of the true label and distributes the remaining probability to other classes, decreasing the degree of label supervision on the model; confidence regularization encourages the model to be cautious about the output during training, reducing the model's overconfidence in noisy labels, for example, by introducing an additional loss term through KL divergence to constrain the output features. However, model regularization is more effective when the proportion of noisy labels is low; if the number of noisy labels is too large, the robustness of model regularization will be significantly reduced. Summary of the Invention

[0005] To address the aforementioned technical issues, this invention proposes a joint optimization method for label noise removal and spectral distribution modeling. This method extracts reliable samples with labels through non-uniform spatial sampling, constrains classification features using multi-center spectral prototypes, and achieves stable model training under noisy label conditions through joint optimization of pixel-wise overlap loss, class cross-entropy loss, and spectral prototype contrast loss. This method can dynamically remove noisy labels without over-cleaning; furthermore, the joint optimization of noise removal and spectral modeling ensures low noise in the input data, and the model regularization effect is good, demonstrating high practical value.

[0006] To achieve the above objectives, the present invention proposes the following technical solution:

[0007] A joint optimization method for tag noise removal and spectral distribution modeling, the method comprising:

[0008] Step 1: Remove noise labels: Randomly sample from the prediction mask and the ground truth mask, obtain the confidence matrix of the sampling points through the confidence calculation formula, sort the confidence of the sampling points, and take the top points as the clean samples with labels.

[0009] Step 2: Regularize the classification model: Introduce a multicenter spectral prototype into the classification model, measure the similarity between the vector to be classified and the multicenter spectral prototype, and determine the category assignment based on the measurement result, thereby weakening the label supervision role.

[0010] Step 3: Design and optimize the loss function to achieve dynamic interactive iterative optimization of noise label removal and classification model regularization.

[0011] The beneficial effects of this invention are as follows:

[0012] It is highly dynamic, with the label denoising results changing dynamically with each training round of the model, without hindering model training; it has good versatility, with spectral distribution modeling including multi-center prototypes, which can describe the hyperspectral characteristics of various land features; it is highly robust, with the joint optimization framework ensuring that the hyperspectral classification model can be trained under different label noise ratios. Attached Figure Description

[0013] Figure 1 This is a schematic diagram illustrating the training principle of the hyperspectral classification model under noisy labels in this invention.

[0014] Figure 2 This is a flowchart of the joint optimization method for tag noise removal and spectral distribution modeling of the present invention;

[0015] Figure 3 The figure shows the experimental results of the joint optimization method for tag noise removal and spectral distribution modeling of the present invention. Detailed Implementation

[0016] The present invention will be further described in conjunction with the accompanying drawings and embodiments.

[0017] As attached Figure 1 As shown, the proposed joint optimization method for label noise removal and spectral distribution modeling consists of three steps: noise label removal, model regularization, and joint optimization. These three steps are repeated during the training of the hyperspectral classification model, enabling the training of a stable classification model on noisy hyperspectral data. Specifically, hyperspectral images contain numerous noisy labels, such as unclear boundaries between wetlands and water bodies, and some pixels being mislabeled. Directly using this dataset for training leads to overfitting of the classification model to the noisy labels. Even with only segmented processing through noise label removal and model regularization, the ideal and actual classification surfaces still differ significantly. Therefore, joint optimization is needed to achieve robust training of the classification model through iterative iteration.

[0018] like Figure 2 The flowchart of the joint optimization method for tag noise removal and spectral distribution modeling of the present invention is shown, and the specific steps include:

[0019] 1) Noise tag removal

[0020] The hyperspectral image input to the classification network is denoted as... ,in This represents the number of hyperspectral bands. Image height, R is the image width, and R is the real number field. For the real number field. Multiply the hyperspectral image by the hyperspectral image. After encoding and decoding, the output foreground prediction mask is denoted as... The truth mask is denoted as ,in, The number of categories is given, and the foreground prediction mask and ground truth mask can be decomposed into:

[0021] (1)

[0022] (2)

[0023] Indicates the first The first mask The probability that a pixel is predicted to be a foreground pixel. Indicates the first The first mask The true value of each pixel. ∈ Then the confidence matrix Each element It can be represented as:

[0024] (3)

[0025] Represents the modulo value, where the dynamic confidence objective function is... It is expressed as follows:

[0026] (4)

[0027] in, Indicates the current iteration step The set threshold for the number of iteration steps. The confidence index is the natural index. The confidence matrix represents the confidence level of the sampled points; the larger the value, the higher the confidence ranking. The values ​​of the top-ranked confidence points are selected. Each point is used as a sampling point and participates in the subsequent calculation of features and loss function.

[0028] 2) Model Regularization

[0029] After noise label removal, the noise content of the sample labels input to the classification network is significantly reduced, but some label noise may still be present. Therefore, we design a multi-center spectral prototype learning method to regularize the model and prevent the classification model from overfitting to noisy labels. Specifically, let the category features be... Let the multicenter spectral prototype be denoted as ,in, For category feature dimension, For the first The feature vector corresponding to each prediction mask For the first The first spectral prototype There are 1 center, with a total of A central prototype, And the feature scale of each central prototype is Then when the first The feature vectors corresponding to each prediction mask are classified into a certain category. At that time, the eigenvector and the multicenter spectral prototype distance The following relationship must be satisfied:

[0030] (5)

[0031] in, For the first The feature vector corresponding to the nth mask is segmented after the nth... The segment, where the distance metric is the vector cosine distance, can be represented as: The formula is as follows:

[0032] (6)

[0033] Represents the vector dot product, which determines the category assignment based on the distance between the feature vector slice and the multicenter spectral prototype, where, when the... The feature vector corresponding to each prediction mask Belongs to a specific When the spectral prototype corresponds to the category, It can be derived from the following formula:

[0034] (7)

[0035] in, , These represent the category index and center index of the multicenter spectral prototype corresponding to the minimum feature distance metric, respectively. During training, the multicenter spectral prototype uses momentum updates, with the [missing information] as the [missing information]. The first spectral prototype individual centers For example, the formula is as follows:

[0036] (8)

[0037] in, Update weights for momentum. For the first All of the following categories The center of the classification feature vector of the segment. This method enables the modeling of classification features and improves the robustness of the model.

[0038] 3) Joint optimization

[0039] To ensure dynamic and iterative optimization of noise label removal and model regularization during the training of the hyperspectral classification model, a joint optimization loss function was designed. It consists of three parts: pixel-by-pixel overlap loss (spatial nonuniform sampling), category cross-entropy loss (cross entropy) and spectral prototype contrast loss (Spectral prototype contrastive), the formula is expressed as follows:

[0040] (9)

[0041] in, , , To correspond to the pixel-by-pixel overlap loss respectively Category cross-entropy loss Loss compared to the spectral prototype The loss coefficient, pixel-wise overlap loss It can be represented as:

[0042] (10)

[0043] Category cross-entropy loss It can be represented as:

[0044] (11)

[0045] Spectral prototype contrast loss It can be represented as:

[0046] (12)

[0047] By using the joint gradient descent of this loss function, joint optimization of label noise removal and spectral distribution modeling can be achieved, and the experimental results are as follows. Figure 3 As shown, some areas in the ground truth map contain noisy labels that do not match the land cover types in the input map. The proposed joint optimization method can train the model well and still predict the correct land cover category results even with label noise.

[0048] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A joint optimization method for tag noise removal and spectral distribution modeling, characterized in that, The method includes: Step 1: Remove noise labels: Randomly sample from the prediction mask and the ground truth mask, obtain the confidence matrix of the sampling points through the confidence calculation formula, sort the confidence of the sampling points, and take the top points as the clean samples with labels. Step 2: Regularize the classification model; introduce a multicenter spectral prototype into the classification model, measure the similarity between the vector to be classified and the multicenter spectral prototype, determine the category assignment based on the measurement result, and update the multicenter spectral prototype with momentum; including: Let the category feature be Let the multicenter spectral prototype be denoted as ,in, For category feature dimension, For the first The feature vector corresponding to each prediction mask ∈ , For the number of categories, For the real number field, For the first The first multicenter spectral prototype One center, , Let be the total number of multicenter spectral prototypes, and let be the characteristic scale of each multicenter spectral prototype. When the first The feature vectors corresponding to each prediction mask are classified into categories. At that time, the eigenvector and the multicenter spectral prototype distance The following relationship must be satisfied: ， in, For the first The feature vector corresponding to the nth mask is segmented into the nth... The segment, where the distance metric is the vector cosine distance, is represented as... The formula is as follows: ， Represents the vector dot product, which determines the category assignment based on the distance between the feature vector slice and the multicenter spectral prototype. When the... The feature vector corresponding to each foreground prediction mask belong When the category corresponds to a multicenter spectral prototype, Derived from the following formula: ， in, , These are the category index and center index of the multicenter spectral prototype corresponding to the minimum feature distance metric, respectively. During training, the multicenter spectral prototype adopts momentum update, and the momentum update formula is as follows: ， in, Update weights for momentum. For the first All of the following categories The center of the segment's classification feature vector; Step 3: Design and optimize the loss function to achieve dynamic interactive iterative optimization of noise label removal and classification model regularization.

2. The joint optimization method for tag noise removal and spectral distribution modeling according to claim 1, characterized in that, Step 1 includes, The hyperspectral image input to the classification network is denoted as... ,in This represents the number of hyperspectral bands. Image height, Image width, For the real number domain, the hyperspectral image After encoding and decoding, the output foreground prediction mask is denoted as... The truth mask is denoted as ,in, Number of categories: ， ， In the formula, Indicates the first The first mask The probability that a pixel is predicted to be a foreground pixel. Indicates the first The first mask The true value of each pixel. ∈ Confidence matrix element It is calculated by the following formula: ， Represents the modulo value, where the dynamic confidence objective function is... It is expressed as follows: ， in, Indicates the current iteration step. The set threshold for the number of iteration steps. It is the natural index.

3. The joint optimization method for tag noise removal and spectral distribution modeling according to claim 2, characterized in that, Step 3 includes jointly optimizing the loss function. Including pixel-wise overlap loss Category cross-entropy loss Loss compared to the spectral prototype The formula is expressed as follows: ， in, , , To correspond to the pixel-by-pixel overlap loss respectively Category cross-entropy loss Loss compared to the spectral prototype The loss coefficient, pixel-wise overlap loss Represented as: ， Category cross-entropy loss Represented as: ， Spectral prototype contrast loss Represented as: ， In the formula, N represents the total number of pixels after filtering.

Images