Damaged image processing method, apparatus and equipment based on negative label smoothing

Abstract

This application relates to a method, apparatus, and device for processing damaged images based on negative label smoothing. The method includes: constructing a noise label training model. A set of label vectors, obtained by label smoothing regularization of the original training dataset containing damaged images, is input into the noise label training model. The noise label training model evaluates the noise data differences between classification category samples using negative label smoothing, and divides the original training dataset into a first training set and a pseudo-label training set. An entropy minimization loss function is constructed based on the first training set and the pseudo-label training set. The noise label training model is trained using the entropy minimization loss function to obtain a trained noise label training model, which is then used to process damaged images. This method helps reduce overfitting of the model to noisy data, improves generalization ability, and increases the accuracy of damaged image restoration.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus and device for processing damaged images based on negative label smoothing. Background Technology

[0002] Deep learning models have made tremendous progress in many fields, continuously having a disruptive impact on various areas of technology and science. Based on complex neural network layers, these models are capable of learning and interpreting complex patterns in data, playing a crucial role in tasks ranging from image recognition to speech recognition and market trend prediction. However, they still face a significant challenge: the inconsistency between the distribution of training data and actual test data. These models are typically fine-tuned on clean and ordered datasets. However, when applied in the real world, they often encounter noisy, irregular, or corrupted data, severely weakening their performance in practical applications and becoming a major obstacle to the practical implementation of deep learning solutions.

[0003] To address this key challenge, modern researchers and practitioners are increasingly focusing on real-time model tuning strategies during the testing phase. This innovative approach, known as Temporal Test Adaptation (TTA), involves dynamically adjusting the model as it encounters new data during runtime. This method enables the model to continuously learn and adapt, ensuring accuracy and effectiveness even when faced with data drastically different from the training set. Furthermore, TTA not only enhances the adaptability and flexibility of deep learning models but also significantly improves their robustness to the unpredictability and variability of real-world environments. As a major breakthrough in deep learning, TTA bridges the gap between theoretical model performance and practical applications, ensuring that deep learning systems remain reliable and effective in a wide range of real-world applications. In addition, Label Smoothing (LS), which adjusts hard labels to a softer distribution, prevents overconfidence in predictions and enhances model generalization, especially in the presence of noisy labels.

[0004] However, its effectiveness decreases during test-time adaptation (TTA) of unlabeled noisy data, and is limited by the increased entropy of the noisy data, where LS may over-smooth the posterior, resulting in a large bias in the restoration of the damaged image. Summary of the Invention

[0005] Therefore, it is necessary to provide a method, apparatus, and device for processing damaged images based on negative label smoothing that can improve the accuracy of damaged image restoration, addressing the aforementioned technical problems.

[0006] A method for processing damaged images based on negative label smoothing, the method comprising:

[0007] Construct a noise label training model.

[0008] The set of label vectors obtained by label smoothing regularization of the original training data set containing damaged images is input into the noisy label training model.

[0009] The noise label training model evaluates the differences in noise data between classification category samples through negative label smoothing and divides the original training data set into a first training set and a pseudo-label training set.

[0010] Based on the first training set and the pseudo-label training set, an entropy minimization loss function is constructed. The noise label training model is then trained using the entropy minimization loss function to obtain a trained noise label training model. This trained noise label training model is then used to process damaged images.

[0011] In one embodiment, the method further includes: optimizing the original dataset containing corrupted images using a softmax function over a period of time.

[0012]

[0013] Where, x t For the corrupted image data that is adaptively accessed in the test time domain of the target domain, y t Let θ be the label of the corrupted image data adaptively accessed in the test domain of the target domain, θ be the network parameters of the noise label training model, C be the image classification category of the original training dataset, and z be the image classification category of the original training dataset. C Let z be the c-th vector of logits output by the neural network. i Let be the i-th vector of logits output by the neural network. After label smoothing regularization, the optimized original dataset yields a set of label vectors:

[0014]

[0015] Among them, y i Let be the set of label vectors for the corrupted image data, γ be the label smoothing rate, LS be the label smoothing regularization, and C be the image classification category of the original training dataset. The classification category samples are then input into the noisy label training model.

[0016] In one embodiment, the method further includes: a noise-trained model evaluating the noise data differences of classification category samples using a negative label smoothing method to obtain a set of negative label vectors for the corrupted image data.

[0017]

[0018] in, Let γ be the set of negative label vectors in the corrupted image data, γ be the negative label smoothing rate, NLS be the negative label smoothing, C be the image classification category in the original training dataset, and y be the negative label vectors in the corrupted image data. i This is the set of label vectors for corrupted image data. Based on the set of negative label vectors, the original training dataset is divided into a first training set and a pseudo-label training set.

[0019] In one embodiment, it further includes:

[0020]

[0021] Among them, L TEHT (f θ (x t ),y t Let x be the entropy minimization loss function. t For the corrupted image data that is adaptively accessed in the test time domain of the target domain, y t Let z be the label of the corrupted image data adaptively accessed in the test domain of the target domain, θ be the network parameters of the noise label training model, and z be the target domain. C Let z be the c-th vector of logits output by the neural network. i Let f be the i-th vector of logits output by the neural network, C be the image classification category of the original training dataset, and f be the vector of logits output by the neural network. θ The model is trained using noise labels, and N is the number of matching batches between the first training set and the pseudo-label training set.

[0022] In one embodiment, the method further includes: fitting and optimizing the entropy minimization loss function by replacing the negative label vector set with a pseudo-label training set, thereby obtaining an optimized entropy minimization loss function.

[0023]

[0024] Among them, L ours Let z be the optimized entropy minimization loss function, γ be the negative label smoothing rate, and z be the negative label smoothing rate. C Let z be the c-th vector of logits output by the neural network. i Let be the i-th vector of logits output by the neural network, C be the image classification category of the original training dataset, and N be the number of matching batches between the first training set and the pseudo-label training set. The network parameters of the noise label training model are updated according to the optimized entropy minimization loss function to converge the noise label training model, resulting in a well-trained noise label training model. This well-trained noise label training model is then used to process the damaged image.

[0025] A damaged image processing apparatus based on negative label smoothing, the apparatus comprising:

[0026] The model building module is used to build a noise label training model.

[0027] The label smoothing module is used to input the set of label vectors obtained by label smoothing regularization of the original training data set with damaged images into the noisy label training model.

[0028] The negative label smoothing module is used to evaluate the difference in noise data between the classification category samples by using negative label smoothing to train the model, and to divide the original training data set into the first training set and the pseudo-label training set.

[0029] The damaged image processing module is used to construct an entropy minimization loss function based on the first training set and the pseudo-label training set, train the noise label training model using the entropy minimization loss function to obtain the trained noise label training model, and use the trained noise label training model to process the damaged image.

[0030] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program performing the following steps:

[0031] Construct a noise label training model.

[0032] The set of label vectors obtained by label smoothing regularization of the original training data set containing damaged images is input into the noisy label training model.

[0033] The noise label training model evaluates the differences in noise data between classification category samples through negative label smoothing and divides the original training data set into a first training set and a pseudo-label training set.

[0034] Based on the first training set and the pseudo-label training set, an entropy minimization loss function is constructed. The noise label training model is then trained using the entropy minimization loss function to obtain a trained noise label training model. This trained noise label training model is then used to process damaged images.

[0035] A computer-readable storage medium having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0036] Construct a noise label training model.

[0037] The set of label vectors obtained by label smoothing regularization of the original training data set containing damaged images is input into the noisy label training model.

[0038] The noise label training model evaluates the differences in noise data between classification category samples through negative label smoothing and divides the original training data set into a first training set and a pseudo-label training set.

[0039] Based on the first training set and the pseudo-label training set, an entropy minimization loss function is constructed. The noise label training model is then trained using the entropy minimization loss function to obtain a trained noise label training model. This trained noise label training model is then used to process damaged images.

[0040] The aforementioned method, apparatus, and device for processing damaged images based on negative label smoothing perform label smoothing regularization on damaged images in the original training dataset to obtain a smoothed set of label vectors. This ensures the smoothness of the label data, reducing the impact of noise on model training. Furthermore, the model evaluates the noise data differences between classification category samples using negative label smoothing and divides the original training dataset into a first training set and a pseudo-label training set. This enables the model to better handle various types of noise, damaged, or incomplete data, improving data robustness. Finally, an entropy minimization loss function is constructed using the first training set and the pseudo-label training set to train the noisy label training model. Training with the entropy minimization loss function helps reduce overfitting of the model to noisy data, improves generalization ability, effectively utilizes the information of damaged images in the original dataset, and increases the usability of training data. Additionally, it helps reduce reliance on manual image restoration, improving the efficiency and accuracy of automated processing. Through model processing, damaged images can be processed or restored, improving image quality. Attached Figure Description

[0041] Figure 1 This is a flowchart illustrating a damaged image processing method based on negative label smoothing in one embodiment;

[0042] Figure 2 The figure shows a comparison of the proposed method with the source model, batch normalization, online TENT, and online-CoTTA on WideResNet-28-10 at degradation level 5 on the CIFAR10-C dataset in one embodiment.

[0043] Figure 3 The figure shows a comparison of the proposed method with the source model, batch normalization, online TENT, and online-CoTTA on WideResNet-40-2 at degradation level 5 on the CIFAR10-C dataset in one embodiment.

[0044] Figure 4 This is a comparison of the proposed method with the source model, batch normalization, online TENT, and Online-CoTTA methods on PreActResNet-18 at degradation level 5 on the CIFAR10-C dataset in one embodiment.

[0045] Figure 5This is a graph showing the comparison results of our method with the source model, batch normalization, online TENT, and online Cot2x4d on ResNext293x4d at corruption level 5 on the CIFAR100-C dataset in one embodiment.

[0046] Figure 6 The figure shows the comparison results of this method with the source model, batch normalization, online TENT and online-CoTTA on WideResNet-40-2 at corruption level 5 on the CIFAR100-C dataset in one embodiment.

[0047] Figure 7 The image shows a comparison of this method with the source model, batch normalization, online TENT, and Online-CoTTA on ResNet-50 of the ImageNet-C dataset in one embodiment.

[0048] Figure 8 This is a structural block diagram of a damaged image processing device based on negative label smoothing in one embodiment;

[0049] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] In one embodiment, such as Figure 1 As shown, a method for processing damaged images based on negative label smoothing is provided, including the following steps:

[0052] Step 102: Construct a noise label training model.

[0053] Step 104: Input the set of label vectors obtained by label smoothing regularization processing of the original training data set with damaged images into the noisy label training model.

[0054] Step 106: The noise label training model evaluates the noise data difference of the classification category samples through negative label smoothing and divides the original training data set into the first training set and the pseudo-label training set.

[0055] Step 108: Construct an entropy minimization loss function based on the first training set and the pseudo-label training set. Train the noise label training model using the entropy minimization loss function to obtain a trained noise label training model. Use the trained noise label training model to process the damaged image.

[0056] In the aforementioned method for processing damaged images based on negative label smoothing, the damaged images in the original training dataset undergo label smoothing regularization to obtain a smoothed set of label vectors. This ensures the smoothness of the label data, reducing the impact of noise on model training. Furthermore, the model evaluates the noise data differences between feature category samples using negative label smoothing and divides the original training dataset into a first training set and a pseudo-label training set. This enables the model to better handle various types of noise, damaged, or incomplete data, improving data robustness. Finally, an entropy minimization loss function is constructed using the first training set and the pseudo-label training set to train the noisy label training model. Training with the entropy minimization loss function helps reduce overfitting of the model to noisy data, improves generalization ability, effectively utilizes the information from damaged images in the original dataset, and increases the usability of the training data. Additionally, it helps reduce reliance on manual image restoration, improving the efficiency and accuracy of automated processing. Through model processing, damaged images can be processed or restored, improving image quality.

[0057] In one embodiment, the original dataset containing corrupted images is optimized using a softmax function over a period of time:

[0058]

[0059] Where, x t For the corrupted image data that is adaptively accessed in the test time domain of the target domain, y t Let Z be the label of the corrupted image data adaptively accessed in the test domain of the target domain, θ be the network parameters of the noise label training model, C be the image classification category of the original training dataset, representing a C-classified data, and Z be the set of neural network output logits, where Z contains C logits for a C-classified data. i Let x be the i-th vector of logits output by the neural network. t The logit z output by the neural network c Let c be the c-th vector of logits output by the neural network. These are calculated according to the softmax formula, with the denominator being class C (from 1 to C). The softmax function, through cumulative summation, normalizes the neural network output logit to a value range of 0-1. The optimized original dataset, after label smoothing regularization, yields a set of label vectors.

[0060]

[0061] Among them, y iLet be the set of label vectors for the corrupted image data, γ be the label smoothing rate, LS be the label smoothing regularization, and C be the image classification category of the original training dataset. The classification category samples are then input into the noisy label training model.

[0062] In one embodiment, the noise-trained model evaluates the noise data differences of the classification category samples through a negative label smoothing method, resulting in a set of negative label vectors for the corrupted image data.

[0063]

[0064] in, Let γ be the set of negative label vectors in the corrupted image data, γ be the negative label smoothing rate, NLS be the negative label smoothing, C be the image classification category in the original training dataset, and y be the negative label vectors in the corrupted image data. i The set of label vectors for corrupted image data represents the values ​​of the neural network output after passing through the softmax function; 1 here refers to one-hot vectors. The original training dataset is divided into a first training set and a pseudo-label training set based on the set of negative label vectors.

[0065] In one embodiment,

[0066]

[0067] Among them, L TEHT (f θ (x t ),y t Let x be the entropy minimization loss function. t For the corrupted image data that is adaptively accessed in the test time domain of the target domain, y t Let z be the label of the corrupted image data adaptively accessed in the test domain of the target domain, θ be the network parameters of the noise label training model, and z be the target domain. C Let z be the c-th vector of logits output by the neural network. i Let be the i-th vector of logits output by the neural network. These are calculated according to the softmax formula, with the denominator being class C (from 1 to C). The softmax function, which sums cumulatively, normalizes the neural network output logit to a value range of 0-1. C represents the image classification categories in the original training dataset, and f... θ For the noise-labeled training model, N is the number of matching batches between the first training set and the pseudo-labeled training set.

[0068] In one embodiment, the entropy minimization loss function is optimized by replacing the negative label vector set with a pseudo-label training set, resulting in the optimized entropy minimization loss function:

[0069]

[0070] Among them, L ours Let z be the optimized entropy minimization loss function, γ be the negative label smoothing rate, and z be the negative label smoothing rate. C Let z be the c-th vector of logits output by the neural network. i Let be the i-th vector of logits output by the neural network. These are calculated according to the softmax formula, with the denominator being class C (from 1 to C). The cumulative summation and softmax function normalize the neural network output logit to a value range of 0-1. C represents the image classification category of the original training dataset, and N represents the number of matching batches between the first training set and the pseudo-label training set. The network parameters of the noise label training model are updated according to the optimized entropy minimization loss function to converge the noise label training model, resulting in a well-trained noise label training model. This well-trained noise label training model is then used to process the damaged image.

[0071] In one embodiment, such as Figure 2 As shown, at degradation level 5 on the CIFAR10-C dataset, this method is compared with the source model, batch normalization, online TENT, and online-CoTTA on WideResNet-28-10: classification error rate (%), with the best results highlighted in bold. Dataset: classification error rate (%), with the best results highlighted in bold. To comprehensively evaluate the effectiveness of this method, we employed different network architectures, each with its own unique characteristics and advantages.

[0072] WideResNet-28-10: This network architecture, consisting of 28 deep layers and 10 extended width layers, is renowned for improving capacity and performance in complex image classification tasks. The increased width in this configuration allows for more substantial feature extraction capabilities, making it an ideal candidate for robust image processing challenges.

[0073] PreActResNet-18: As an 18-layer deep residual network, its key feature is its pre-activated residual units. This architecture offers a balance between depth and computational efficiency, making it a common choice for various image recognition applications.

[0074] WideResNet-40-2: This network has a depth of 40 layers and a width of 2 layers, striking a balance between the depth and increased width of the traditional ResNet architecture. This design aims to improve the network's ability to learn more complex features without significantly increasing computational requirements.

[0075] ResNext29-32x4d: As a variant of the ResNet architecture, ResNext29 32x4d has 29 layers, including 32 parallel paths, each with a width of 4. This structure takes advantage of the increased depth and width, while also introducing cardinality as a fundamental factor, further diversifying the model's ability to handle a range of image processing tasks.

[0076] ResNet50: As the standard backbone architecture for the ImageNet dataset, it is a 50-layer deep network renowned for its effectiveness in large-scale image classification. Its widespread application in various image recognition challenges makes it a key benchmark for evaluating the performance of new methods.

[0077] Specifically, experiments on these different network architectures fully demonstrate the generalization and adaptability of our proposed method. To ensure fair comparisons, pre-trained models from a robust platform source were used. The Adam optimizer was employed in all considered models. The learning rate was set to 1e-3, and the β parameter was configured to 0.9. Weight decay was not applied in the experiments, as it was set to 0, and a consistent batch size of 200 was maintained across all tests. This method builds upon the online-tent approach, aiming to enhance the model's adaptability and robustness when handling corrupted images. It leverages the advantages of the Online-Tent framework while introducing new improvements to address its limitations.

[0078] Furthermore, different approaches were compared on the WideResNet-28-10 architecture under CIFAR10-C degradation 5 on the CIFAR10-C dataset. This comparison shows the classification error rate (%) for various types of corruption, as well as the mean error rate. The source model, serving as the baseline, presents an initial challenge with an mean error rate of 43.5% on the dataset. Batch normalization significantly improved the performance of the normalization layer by tuning the model, reducing the error rate to 20.4%. This highlights the effectiveness of this technique in adapting to image degradation. Online-TENT further reduced the error rate to 18.6% by optimizing the batch normalization statistics and minimizing entropy, achieving test-temporal adaptation (TTA), which emphasizes the effectiveness of TTA in dynamically adapting to different types of corruption. Online-CoTTA updates the entire network parameters, utilizing data augmentation, with a mean error rate of 18.5%. Notably, it performs well in handling impulse noise, with a low error rate of 28.0%, demonstrating its strength in specific degradation scenarios.

[0079] In one embodiment, the method is compared with the source model, batch normalization, online TENT, and online-CoTTA on WideResNet-40-2 at degradation level 5 of the CIFAR10-C dataset: classification error rate (%), with the best results highlighted in bold.

[0080] Specifically, by incorporating Negative Label Smoothing (NLS), the lowest overall error rate of 17.0% was achieved. It demonstrates superior performance across most types of degradation behavior. However, we observed that the effectiveness of NLS varies across different types of degradation, with Online-CoTTA performing slightly better in the impulse noise category. This variation suggests that NLS parameters need careful tuning during testing to achieve optimal adaptability. In summary, while traditional methods such as batch normalization and Online-TENT are effective for image degradation, this method, focusing on negative label smoothing, provides significant enhancements, especially in high-noise environments. This demonstrates that targeted strategies like NLS can lead to considerable improvements in handling image degradation. Results on WideResNet-40-2: as shown Figure 3 As shown, various methods using the WideResNet-40-2 architecture are compared on the CIFAR10-C dataset at degradation level 5. These results primarily focus on the classification error rate (%) for different types of degradation, as well as the overall mean error rate. The source model establishes a baseline with a mean error rate of 18.3%. Batch normalization significantly improves upon this, reducing the error rate to 14.5%. This reduction underscores the effectiveness of batch normalization in adapting to various types of image degradation. Online-TENT, utilizing a test temporal adaptive technique, further reduces the error rate to 12.1%. Online-TENT is particularly effective in the "brightness" weather category, achieving the best results with an error rate of 6.7%, demonstrating its advantage in this specific type of degradation. The overall error rate recorded by Online-cotta is 13.2%. Notably, it performs exceptionally well in handling impulse noise, achieving the lowest error rate of 17.0% in this noise class, demonstrating its ability to handle certain challenging degradation scenarios. The proposed method has the lowest overall error rate of 11.7%, demonstrating superior performance in most types of degradation. It exhibits particular effectiveness in categories such as "out-of-focus," "glass," "motion," and "zoom," showing the lowest error rate among all methods.

[0081] In one embodiment, such as Figure 4As shown, this method is compared with the source model, batch normalization, Online TENT, and Online-CoTTA methods on PreActResNet-18 at degradation level 5 of the CIFAR10-C dataset: classification error rate (%), with the best results highlighted in bold. An experiment comparing various methods using the PreActResNet-18 architecture at degradation level 5 of the CIFAR10-C dataset is presented. This comparison outlines the classification error rate (%) for different damage types, as well as the overall mean error rate. The source model serves as the baseline with a mean error rate of 19.4%. Batch normalization adjusts the model's normalization layers, significantly reducing this rate to 15.0%. This considerable decrease highlights the effectiveness of batch normalization in adapting the model to damaged images. Online-TENT achieves test-temporal adaptation, further reducing the error rate to 12.6%. It is particularly effective in the "contrast" degradation category, recording a significant error rate of 11.6%, demonstrating its advantage in this specific challenge. Online-CoTTA updates the entire network parameters, utilizing data augmentation, with an overall error rate of 14.0%. It shows consistent performance across various corruption types, but does not outperform other methods in any particular category.

[0082] Our method achieves the lowest overall error rate of 11.5%. Notably, it consistently records the lowest error rate across all degradation scenarios, confirming its effectiveness within the PreActResNet-18 architecture. It performs exceptionally well in areas such as defocus, glass, magnification, and contrast, significantly outperforming other methods. Results from three different network architectures—WideResNet28-10, WideResNet-40-2, and PreActResNet-18—collectively demonstrate the superiority of our proposed method. It consistently achieves the lowest mean error rate across all models, a remarkable achievement considering the structural diversity of networks, including variations in depth, width, and pre-activation features. This consistency not only underscores the robustness of our method but also its adaptability to different network architectures, making it an efficient and versatile approach for handling image degradation.

[0083] In one embodiment, such as Figure 5As shown, at corruption level 5 on the CIFAR100-C dataset, this method is compared with the source model, batch normalization, online TENT, and online Cot2x4d on ResNext29 3x4d: classification error rate (%), with the best results highlighted in bold. Experimental results using ResNext29 on the CIFAR100-C dataset with a 32x4d model are presented. This experiment aims to demonstrate the effectiveness of various methods in adapting to the challenges of corrupted image environments.

[0084] In one embodiment, such as Figure 6 As shown, on the CIFAR100-C dataset at corruption level 5, this method is compared with the source model, batch normalization, online TENT, and online-CoTTA on WideResNet-40-2: classification error rate (%), with the best results highlighted in bold. The source model, without any adaptation, has the highest error rate of 46.4%, indicating its limited generalization ability in environments with varying distributions. This result highlights the necessity of adopting adaptive strategies under different conditions. Batch normalization significantly improves upon the source model, reducing the error rate to 35.4%. This simple yet effective method emphasizes the importance of adjusting batch normalization statistics to enhance the model's robustness to degradation. Online-TENT and Online-CoTTA, both optimized using test-time, achieve further improvements with error rates of 33.2% and 34.4%, respectively. In the WideResNet-40-2 model, Online-CoTTA outperforms Online-TENT, while in the ResNext29 model, the opposite is true for the 32x4d model. This change indicates that the effectiveness of test time optimization is affected by the underlying network architecture.

[0085] Our method consistently achieved the lowest error rate across all scenarios, remarkably reducing it to 31.3%. Compared to state-of-the-art methods, our method reduced the error rate by 1.9% in ResNext29 for 32x4d models. This superior performance is consistent across all 15 discrete degradation domains, highlighting the robustness and adaptability of our method. This significant improvement in error rate reflects the effectiveness of our method in handling a wide range of degradation challenges, affirming its potential as a leading solution for adapting to degradation environments.

[0086] In summary, the experimental results on the ResNext29 32x4d model clearly demonstrate the superior performance of our proposed method in processing damaged images, outperforming other contemporary methods in various damage scenarios. This underscores the importance of using advanced adaptive techniques to achieve robustness in challenging image processing tasks. Figure 6The results show experimental results using the WideResNet-WideR-40-2 model on the CIFAR100-C dataset. This experiment aims to evaluate the effectiveness of various methods. The source model lacks any form of adaptation, exhibiting a maximum error rate of 46.7%, indicating its susceptibility to distribution variations and image corruption. This underscores the critical need for robust adaptive strategies to enhance model generalization. Batch normalization significantly improves upon the source model, reducing the error rate to 39.3%. This result highlights the importance of adjusting batch normalization statistics to enhance model resilience in the presence of degradation. Online-TENT and Online-CoTTA both utilize test-time optimization, further improving model performance with error rates of 38.7% and 37.3%, respectively. The relative performance of these methods varies across different models, highlighting the impact of network architecture on the effectiveness of test-time optimization.

[0087] Our method achieves the lowest error rate across all scenarios, significantly reducing it to 35.5%. Compared to state-of-the-art methods, it reduces the error rate by 2.2% in the WideResNet-40-2 model. This excellent performance across all 15 different degradation domains reaffirms the adaptability and robustness of our method.

[0088] In summary, the inconsistent and ambiguous experimental results on the WideResNet-40-2 model establish the superior performance of our method in addressing image degradation, outperforming contemporary methods in various degradation scenarios. This highlights the effectiveness of our adaptive techniques in improving the robustness of the model in complex image processing tasks.

[0089] In one embodiment, such as Figure 7 As shown, the proposed method is compared with the source model, batch normalization, online TENT, and Online-CoTTA on ResNet-50 of the ImageNet-C dataset: classification error rate (%), with the best results highlighted in bold. Experiments were conducted using the standard ResNet-50 model provided by RobustPlatform, which was trained following a consistent protocol with a learning rate of 0.00025, an SGD optimizer momentum denoted as β, maintained at 0.9, and weight decay of 0. Experiments were performed with a batch size of 64 to ensure a balance between computational efficiency and model performance. ResNet-50 results: Experiments on the ImageNet dataset highlight the robustness and generality of our model on large and diverse datasets. Focusing on the ResNet-50 level (level 5) for the most severe degradation cases, the proposed method outperforms existing state-of-the-art methods on all 15 degradation types.

[0090] Notably, our method achieved an error rate of 66.5%, a significant improvement of 2.7% compared to the Online-TENT method. This reduction in error rate highlights the effectiveness of our approach, particularly in handling various degradation types on large-scale datasets. These results demonstrate that our method is not only effective under challenging conditions but also sets a new benchmark in terms of model resilience and adaptability.

[0091] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0092] In one embodiment, such as Figure 8 As shown, a damaged image processing device based on negative label smoothing is provided, including: a model building module 802, a label smoothing processing module 804, a negative label smoothing processing module 806, and a damaged image processing module 808, wherein:

[0093] Model building module 802 is used to build a noise label training model.

[0094] The label smoothing module 804 is used to input the set of label vectors obtained by label smoothing regularization of the original training data set with damaged images into the noisy label training model.

[0095] The negative label smoothing module 806 is used to evaluate the difference in noise data between the classification category samples by the negative label smoothing method of the noise label training model, and to divide the original training data set into the first training set and the pseudo-label training set.

[0096] The damaged image processing module 808 is used to construct an entropy minimization loss function based on the first training set and the pseudo-label training set, train the noise label training model through the entropy minimization loss function to obtain the trained noise label training model, and use the trained noise label training model to process the damaged image.

[0097] Specific limitations regarding the damage image processing apparatus based on negative label smoothing can be found in the limitations of the damage image processing method based on negative label smoothing described above, and will not be repeated here. Each module in the aforementioned damage image processing apparatus based on negative label smoothing can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0098] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a method for processing damaged images based on negative label smoothing. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0099] Those skilled in the art will understand that Figure 8-9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0100] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to perform the following steps:

[0101] Construct a noise label training model.

[0102] The set of label vectors obtained by label smoothing regularization of the original training data set containing damaged images is input into the noisy label training model.

[0103] The noise label training model evaluates the differences in noise data between classification category samples through negative label smoothing and divides the original training data set into a first training set and a pseudo-label training set.

[0104] Based on the first training set and the pseudo-label training set, an entropy minimization loss function is constructed. The noise label training model is then trained using the entropy minimization loss function to obtain a trained noise label training model. This trained noise label training model is then used to process damaged images.

[0105] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0106] Construct a noise label training model.

[0107] The set of label vectors obtained by label smoothing regularization of the original training data set containing damaged images is input into the noisy label training model.

[0108] The noise label training model evaluates the differences in noise data between classification category samples through negative label smoothing and divides the original training data set into a first training set and a pseudo-label training set.

[0109] Based on the first training set and the pseudo-label training set, an entropy minimization loss function is constructed. The noise label training model is then trained using the entropy minimization loss function to obtain a trained noise label training model. This trained noise label training model is then used to process damaged images.

[0110] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0111] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0112] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for processing damaged images based on negative label smoothing, characterized in that, The method includes: Construct a noise label training model; The set of label vectors obtained after label smoothing and regularization of the original training dataset containing damaged images is input into the noisy label training model; the original dataset containing damaged images is then optimized using the softmax function over a period of time. in, Corrupted image data that is adaptively accessed in the test time domain for the target domain. The label of the damaged image data is adaptively accessed in the test time domain for the target domain. The network parameters for the noise label training model are... Z represents the image classification categories of the original training dataset, and Z is the set of logits output by the neural network. Let be the i-th vector of logits output by the neural network. Let c be the c-th vector of logits output by the neural network; The optimized original dataset, after undergoing label smoothing regularization, yields a set of label vectors: in, The set of label vectors for the damaged image data. For label smoothing rate, For label smoothing regularization, The image classification categories for the original training dataset; The classification category samples are then input into the noise label training model. The noise label training model evaluates the noise data differences of the classification category samples using a negative label smoothing method, and divides the original training dataset into a first training set and a pseudo-label training set; the noise label training model evaluates the noise data differences of the classification category samples using a negative label smoothing method, and obtains the negative label vector set of the damaged image data: in, The set of negative label vectors for the damaged image data. For negative label smoothing rate, Smoothing for negative labels, The image classification categories for the original training dataset. The set of label vectors for the damaged image data. It is a one-hot vector; The original training data set is divided into a first training set and a pseudo-label training set based on the set of negative label vectors. An entropy minimization loss function is constructed based on the first training set and the pseudo-label training set. The noise label training model is trained using the entropy minimization loss function to obtain the trained noise label training model. The trained noise label training model is then used to process the damaged image.

2. The method according to claim 1, characterized in that, Constructing an entropy minimization loss function based on the first training set and the pseudo-label training set includes: in, The loss function is to minimize entropy. Corrupted image data that is adaptively accessed in the test time domain for the target domain. The label of the damaged image data is adaptively accessed in the test time domain for the target domain. The network parameters for the noise label training model are... Let c be the c-th vector of logits output by the neural network. Let be the i-th vector of logits output by the neural network. The image classification categories for the original training dataset. The noise label training model is used. The number of matching batches between the first training set and the pseudo-label training set.

3. The method according to claim 2, characterized in that, The noise label training model is trained using the entropy minimization loss function to obtain a trained noise label training model. The trained noise label training model is then used to process the damaged image, including: By replacing the negative label vector set with the pseudo-label training set to fit and optimize the entropy minimization loss function, the optimized entropy minimization loss function is obtained: in, The optimized entropy minimization loss function is... For negative label smoothing rate, Let c be the c-th vector of logits output by the neural network. Let be the i-th vector of logits output by the neural network. The image classification categories for the original training dataset. The number of matching batches between the first training set and the pseudo-label training set; The network parameters of the noise label training model are updated according to the optimized entropy minimization loss function to converge the noise label training model, thereby obtaining the trained noise label training model. The trained noise label training model is then used to process the damaged image.

4. A damaged image processing device based on negative label smoothing, characterized in that, The apparatus for implementing the method according to any one of claims 1 to 3, the apparatus comprising: The model building module is used to build a noise label training model; The label smoothing module is used to input the set of label vectors obtained by label smoothing regularization of the original training data set with damaged images into the noisy label training model. The negative label smoothing module is used by the noise label training model to evaluate the noise data difference of the classification category samples through the negative label smoothing method, and to divide the original training data set into a first training set and a pseudo-label training set. The damaged image processing module is used to construct an entropy minimization loss function based on the first training set and the pseudo-label training set, train the noise label training model using the entropy minimization loss function to obtain the trained noise label training model, and process the damaged image using the trained noise label training model.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 3.

Images