Semi-supervised model generalization method integrating label denoising
Through the pseudo-label generation and dual calibration generalization model combined with small-loss strategy and style obfuscation module, the generalization problem of deep learning models when distribution is inconsistent is solved, and efficient transfer and generalization of semi-supervised learning models in different scenarios is realized.
Patent Information
- Application Number
- CN202211736527.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-31
- Publication Date
- 2025-08-19
- Estimated Expiration
- 2042-12-31
AI Technical Summary
Existing deep learning models are difficult to generalize effectively when the training data and test data distribution are inconsistent, and traditional methods cannot fully utilize label-free data to enhance model generalization capabilities.
The semi-supervised model generalization method of fused label denoising is adopted to generate pseudo-labels of label-free source domain data through the pseudo-label generation model, and the dual calibration generalization model is used to learn on pseudo-label and real tags, combining small-loss strategy and style obfuscation module to improve the generalization ability of the model.
The model's migration and generalization capabilities in different scenarios have been improved, especially in autonomous driving, smart home and security monitoring systems, the early warning and adaptability of the intelligent system have been enhanced.
Smart Images
Figure CN115965815B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of domain generalization technology, and in particular to a semi-supervised model generalization method integrating label denoising. Background Art
[0002] Most existing deep learning models assume that the training (source) data and test (target) data come from the same domain / dataset and therefore follow the same distribution. However, in practice, this assumption is often invalid. For example, if we want to design a module for identifying pedestrians and traffic signs in self-driving cars and require that this module be deployed anywhere in the world in all weather conditions, we would need to collect driving data from different cities in different weather conditions, which is obviously an impossible task. Therefore, we need to train models with a certain level of generalization performance, so that they can make good predictions even when the distribution of the test data is inconsistent with that of the training data.
[0003] Domain generalization aims to address the above issues. The goal of domain generalization is to train a model using data from multiple source domains and deploy the model to any unseen target domain without any adaptation. Most domain generalization tasks follow the traditional supervised learning paradigm, where the model's training data consists of source domains with different styles. In-depth exploration of these styles can learn domain-invariant features and enhance the model's generalization capabilities. However, traditional domain generalization methods are not applicable in reality because collecting labeled training data from multiple source domains is very time-consuming and labor-intensive. In addition, the real world is full of unlabeled data, and traditional methods cannot fully utilize this unlabeled data to enhance the model's generalization capabilities. Summary of the Invention
[0004] The purpose of the present invention is to provide a semi-supervised model generalization method integrating label denoising, which is conducive to improving the generalization ability of the model.
[0005] To achieve the above objectives, the present invention adopts a technical solution: a semi-supervised model generalization method integrating label denoising, comprising:
[0006] Step 1: Generate pseudo labels for unlabeled source domain data using a pseudo label generation model;
[0007] Step 2: A dual calibration generalization model is used to learn on the source domains of pseudo labels and true labels. Clean samples are selected for exchange based on a small-loss strategy. Both parties perform updates on the selected clean samples, and a style confusion module is inserted to improve the generalization ability of the model.
[0008] Step 3: The intermediate domain includes the labeled source domain and the unlabeled source domain with clean samples. The progressive intermediate domain generation module linearly mixes these two parts of samples and enters the next cycle as the new labeled source domain data for the pseudo-label generation model.
[0009] Furthermore, step 1 specifically includes the following steps:
[0010] Step 1.1: Consider only the case where labeled source data comes from a single domain, while unlabeled source data comes from multiple domains. The pseudo-label generation model uses two independent classifiers for adversarial training on labeled and unlabeled samples, resulting in different characteristics through training. Ambiguous features may be classified into different categories by two different classifiers. By measuring and minimizing the ambiguous feature area, the classification ambiguity problem is resolved, avoiding ambiguity in sample prediction.
[0011] Step 1.2: Obtain a set of pseudo labels for unlabeled source domain data.
[0012] Furthermore, step 1.1 specifically includes the following steps:
[0013] Step 1.1.1: Train the feature extractor G and classifiers F1 and F2 to correctly classify the labeled source domain. For samples, use the standard cross entropy loss for loss calculation and return:
[0014]
[0015] Among them, Y is the label space corresponding to the labeled source domain, L ce (D x ,Y) represents the calculation of cross entropy loss for labeled source domain data and its corresponding labels, Represents minimizing the cross entropy loss and updating the network parameters of the feature extractor and the two classifiers;
[0016] Step 1.1.2, fix the feature extractor G, update the classifiers F1 and F2, and maximize the difference between the classifiers:
[0017]
[0018] Among them, D u is the unlabeled source domain, L adv (D u ) is expressed as:
[0019]
[0020] Indicates that from D u Sample d in the sample space u, p1 and p2 represent the two classifiers for d u The predicted output of Dis(,) represents the calculation of L1 distance, that is:
[0021] Dis(p1,p2)=|p1-p2|
[0022] Step 1.1.3. Fix the classifier and update the feature extractor to minimize the difference:
[0023]
[0024] In step 1.2, the pseudo-label set Q of the unlabeled source domain data is obtained as:
[0025]
[0026] Among them, φ represents the parameters of the pseudo-label generation model, p φ represents the prediction of the sample by the pseudo-label generation model, It means taking the category with the highest predicted probability as the pseudo label of the unlabeled source domain data.
[0027] Furthermore, step 2 specifically includes the following steps:
[0028] Step 2.1, use the dual calibration generalization model to learn on the source domain of pseudo labels and true labels; the dual calibration generalization model has two sub-networks with the same structure but different initialization parameters, each sub-network inputs the same batch, and each batch contains Some of the data are labeled and from Some pseudo-labeled data For labeled samples, their true labels are used for supervision; for pseudo-labeled samples, the two sub-networks filter out clean samples according to the small-loss strategy. and The pseudo labels of these clean samples can be considered as low noise and can be used as supervisory signals. In order to avoid the error accumulation caused by the sub-network updating on its own clean samples, the sub-networks exchange clean samples and update on the clean samples selected by each other. The loss function is as follows:
[0029]
[0030] in, represents the cross entropy loss function, θ1 and θ2 represent the model parameters of the two sub-networks respectively, Represents minimizing the calculated loss and updating the parameters of the two sub-networks;
[0031] Step 2.2: Insert a style confusion module into the sub-network structure of the dual calibration generalization model. The style confusion module mixes the statistics between different samples, forcing the network to extract style-independent features to achieve the purpose of generalization.
[0032]
[0033] Among them, β is the new mean of the sample, γ is the new variance of the sample, λ is the value randomly sampled from the Beta distribution, ε is a minimum value added to prevent the denominator from being zero, μ and σ are the mean and variance of the current sample, and are the mean and variance of other samples in the same batch, F and These are the features before and after entering the style confusion module;
[0034] Step 2.3: For the entire unlabeled source domain data portion in each mini-batch Label diversity regularization is used to minimize instance entropy for instance discrimination and maximize global entropy for label diversity:
[0035]
[0036] Among them, p θ represents the K-class prediction probability of the subnetwork θ, θ is θ1 or θ2, Indicates from Sample u in the sample space i ;
[0037] Step 2.4. Calculate the overall loss function of the dual calibration generalization model:
[0038]
[0039] Among them, L div (;) represents the label diversity regularization loss in step 2.3;
[0040] Step 2.5: Calculate the gradient and update the model parameters.
[0041] Furthermore, after the training in step 2 is completed, a small-loss strategy is used on the dual calibration generalization model to extract a clean sample set from the pseudo-labeled samples, and the clean samples are linearly mixed with the labeled samples to obtain a set between and The intermediate domain is used to replace the initial The source domain participates in the training of the pseudo-label generation model, improving the quality of pseudo-labels by narrowing the gap between domains.
[0042] Furthermore, by leveraging massive amounts of unlabeled data for semi-supervised learning, we achieve scenario-agnostic edge intelligence. This method is applied to cloud-based generalization improvements in security monitoring systems. This involves leveraging unlabeled source data from other security systems in the cloud to enhance the generalization capabilities of scenario-based security systems, thereby strengthening security early warning capabilities. Furthermore, this method can be applied to diverse fields, such as autonomous driving and smart homes, to improve the transfer and generalization capabilities of intelligent systems in various scenarios.
[0043] Compared with existing technologies, this invention offers the following advantages: Addressing the critical reliance of existing domain generalization methods on labeled data, this invention proposes a semi-supervised domain generalization method, decoupling the task into the label propagation problem and the noisy label domain generalization optimization problem. This method can be applied to diverse fields such as autonomous driving, smart homes, and intelligent surveillance, improving the transfer and generalization capabilities of intelligent systems in various scenarios. It possesses strong practicality and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 This is a flowchart of a method implementation according to an embodiment of the present invention.
[0045] Figure 2 Schematic diagram of the dual calibration generalization model in an embodiment of the present invention. DETAILED DESCRIPTION
[0046] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0047] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.
[0048] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.
[0049] like Figure 1 、 2 As shown, this embodiment provides a semi-supervised model generalization method integrating label denoising, including:
[0050] Step 1: Generate pseudo labels for unlabeled source domain data using a pseudo label generation model;
[0051] Step 2: A dual calibration generalization model is used to learn on the source domains of pseudo labels and true labels. Clean samples are selected for exchange based on a small-loss strategy. Both parties perform updates on the selected clean samples, and a style confusion module is inserted to improve the generalization ability of the model.
[0052] Step 3: The intermediate domain includes the labeled source domain and the unlabeled source domain with clean samples. The progressive intermediate domain generation module linearly mixes these two parts of samples and enters the next cycle as the new labeled source domain data for the pseudo-label generation model.
[0053] The specific implementation of each step is further explained below.
[0054] Step 1: Generate pseudo labels for unlabeled source domain data
[0055] Step 1.1: Conduct adversarial training on labeled source domain and unlabeled source domain
[0056] To simplify the description of the method, the present invention only considers the case where the labeled source domain data comes from a single domain, while the unlabeled source domain data comes from multiple source domains; the pseudo-label generation model uses two independent classifiers to perform adversarial training on labeled and unlabeled samples, so that they have different characteristics through training; the category-ambiguous features may be divided into different categories by two different classifiers, and the classification ambiguity problem is solved by measuring and minimizing the category-ambiguous feature area, avoiding category ambiguity in the prediction of samples and obtaining better classification results.
[0057] Step 1.1.1: Training on the source domain
[0058] Train the feature extractor G and classifiers F1 and F2 to correctly classify the labeled source domain For samples, use the standard cross entropy loss for loss calculation and return:
[0059]
[0060] Among them, Y is the label space corresponding to the labeled source domain, Indicates the calculation of cross entropy loss for labeled source domain data and its corresponding labels. represents minimizing the cross entropy loss and updating the network parameters of the feature extractor and the two classifiers.
[0061] Step 1.1.2: Maximize classifier variance
[0062] Fix the feature extractor G, update the classifiers F1 and F2, and maximize the difference between the classifiers:
[0063]
[0064] Among them, D u is the unlabeled source domain, L adv (D u ) is expressed as:
[0065]
[0066] Indicates that from D u Sample d in the sample space u , p1 and p2 represent the two classifiers for d u The predicted output of Dis(,) represents the calculation of L1 distance, that is:
[0067] Dis(p1,p2)=|p1-p2|
[0068] Step 1.1.3. Optimize feature extractor
[0069] Fix the classifier and update the feature extractor to minimize the difference:
[0070]
[0071] Step 1.2: Obtain the pseudo-label set Q of the unlabeled source domain data:
[0072]
[0073] Among them, φ represents the parameters of the pseudo-label generation model, p φ represents the prediction of the sample by the pseudo-label generation model, It means taking the category with the highest predicted probability as the pseudo label of the unlabeled source domain data.
[0074] Step 2: Improve the generalization ability of the model while denoising the pseudo labels
[0075] Step 2.1: Pseudo-label denoising
[0076] A dual calibration generalization model is used to learn on the source domain of pseudo labels and true labels; the dual calibration generalization model has two sub-networks with the same structure but different initialization parameters. Each sub-network inputs the same batch, and each batch contains Some of the data are labeled and from Some pseudo-labeled data For labeled samples, their true labels are used for supervision; for pseudo-labeled samples, the two sub-networks filter out clean samples according to the small-loss strategy. and The pseudo labels of these clean samples can be considered as low noise and can be used as supervisory signals. In order to avoid the error accumulation caused by the sub-network updating on its own clean samples, the sub-networks exchange clean samples and update on the clean samples selected by each other. The loss function is as follows:
[0077]
[0078] in, represents the cross entropy loss function, θ1 and θ2 represent the model parameters of the two sub-networks respectively, represents minimizing the calculated loss and updating the parameters of the two sub-networks.
[0079] Step 2.2: Improve model generalization
[0080] To further enhance the generalization capability of the dual calibration generalization model, a style confusion module is inserted into the sub-network structure of the dual calibration generalization model. The style confusion module mixes the statistics between different samples (the mean and variance of the sample normalization layer) to force the network to extract style-independent features, thereby achieving the purpose of generalization.
[0081]
[0082] Among them, β is the new mean of the sample, γ is the new variance of the sample, λ is the value randomly sampled from the Beta distribution, ε is a minimum value added to prevent the denominator from being zero, μ and σ are the mean and variance of the current sample, and are the mean and variance of other samples in the same batch, F and These are the features before and after entering the style confusion module.
[0083] Step 2.3: Label regularization
[0084] In step 2.1, clean samples with smaller loss are used to optimize the model, resulting in insufficient mining of the remaining unlabeled source domain data information. To solve this problem, the entire unlabeled source domain data portion in each mini-batch is Label diversity regularization is used to minimize instance entropy for instance discrimination and maximize global entropy for label diversity:
[0085]
[0086] Among them, p θ represents the K-class prediction probability of the subnetwork θ, θ is θ1 or θ2, Indicates from Sample u in the sample space i .
[0087] Step 2.4. Calculate the overall loss function of the dual calibration generalization model:
[0088]
[0089] Among them, L div (;) denotes the label diversity regularization loss in step 2.3.
[0090] Step 2.5: Calculate the gradient and update the model parameters.
[0091] Step 3: Progressive intermediate domain generation
[0092] After the training in step 2 is completed, a small-loss strategy is used on the dual calibration generalization model to extract a clean sample set from the pseudo-labeled samples, and the clean samples are linearly mixed with the labeled samples to obtain a set between and The intermediate domain is used to replace the initial The source domain participates in the training of the pseudo-label generation model, improving the quality of pseudo-labels by narrowing the gap between domains.
[0093] Step 4: Algorithm iteration
[0094] The method of progressive generation of the intermediate domain provides an interactive link for steps 1 and 2, allowing the two stages to be trained alternately and iteratively, with the two models continuously promoting each other, ultimately achieving an improvement in the generalization performance of the dual model.
[0095] In this embodiment, semi-supervised learning is conducted using massive amounts of unlabeled data to achieve scenario-agnostic edge intelligence. This method is applied to cloud-based generalization improvements in security monitoring systems. Specifically, unlabeled source data from other security systems is leveraged in the cloud to enhance the generalization capabilities of the scenario-based security system, thereby strengthening security early warning capabilities. Furthermore, this method can be applied to diverse fields such as autonomous driving and smart homes, improving the transfer and generalization capabilities of intelligent systems in various scenarios.
[0096] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0097] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0098] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.
[0099] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0100] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any other manner. Any person skilled in the art may utilize the above-disclosed technical content to modify or modify the present invention into equivalent embodiments. However, any simple modifications, equivalent variations, and modifications to the above embodiments that do not depart from the technical content of the present invention and are based on the technical essence of the present invention remain within the scope of protection of the present invention.
Claims
1. A semi-supervised model generalization method integrating label denoising, characterized by: include: Step 1: Generate pseudo labels for unlabeled source domain data using a pseudo label generation model; Step 2: A dual calibration generalization model is used to learn on the source domains of pseudo labels and true labels. Clean samples are selected for exchange based on a small-loss strategy. Both parties perform updates on the selected clean samples, and a style confusion module is inserted to improve the generalization ability of the model. Step 3: The intermediate domain includes the labeled source domain and the unlabeled source domain with clean samples. The progressive intermediate domain generation module linearly mixes these two parts of samples and enters the next cycle as the new labeled source domain data for the pseudo-label generation model; Step 2 specifically includes the following steps: Step 2.1, use the dual calibration generalization model to learn on the source domain of pseudo labels and true labels; the dual calibration generalization model has two sub-networks with the same structure but different initialization parameters, each sub-network inputs the same batch, and each batch contains Some of the data are labeled and from Some pseudo-labeled data For labeled samples, their true labels are used for supervision; for pseudo-labeled samples, the two sub-networks filter out clean samples according to the small-loss strategy. and The pseudo labels of these clean samples are considered to be low-noise and can serve as supervisory signals. To avoid error accumulation caused by sub-networks updating on their own clean samples, sub-networks exchange clean samples and update on the clean samples selected by each other. The loss function is as follows: in, represents the cross entropy loss function, θ1 and θ2 represent the model parameters of the two sub-networks respectively, Represents minimizing the calculated loss and updating the parameters of the two sub-networks; Step 2.2: Insert a style confusion module into the sub-network structure of the dual calibration generalization model. The style confusion module mixes the statistics between different samples, forcing the network to extract style-independent features to achieve the purpose of generalization. Among them, β is the new mean of the sample, γ is the new variance of the sample, λ is the value randomly sampled from the Beta distribution, ε is a minimum value added to prevent the denominator from being zero, μ and σ are the mean and variance of the current sample, and are the mean and variance of other samples in the same batch, F and These are the features before and after entering the style confusion module; Step 2.3: For the entire unlabeled source domain data portion in each mini-batch Label diversity regularization is used to minimize instance entropy for instance discrimination and maximize global entropy for label diversity: Among them, p θ represents the K-class prediction probability of the subnetwork θ, θ is θ1 or θ2, Indicates from Sample u in the sample space i ; Step 2.
4. Calculate the overall loss function of the dual calibration generalization model: Among them, L div (;) represents the label diversity regularization loss in step 2.3; Step 2.5: Calculate the gradient and update the model parameters. Semi-supervised learning is performed using massive unlabeled data to achieve scenario-unrestricted edge intelligence. This method is applied to the cloud-based generalization improvement of security monitoring systems, that is, using unlabeled source domain data from other different security systems in the cloud to enhance the generalization ability of the scenario security system and strengthen security early warning efforts.
2. The method for generalizing a semi-supervised model with fused label denoising according to claim 1, characterized in that: Step 1 specifically includes the following steps: Step 1.1: Consider only the case where the labeled source data comes from a single domain, while the unlabeled source data comes from multiple domains. The pseudo-label generation model uses two independent classifiers to perform adversarial training on labeled and unlabeled samples, resulting in different characteristics through training. Step 1.2: Obtain a set of pseudo labels for unlabeled source domain data.
3. The semi-supervised model generalization method for fusion label denoising according to claim 2 is characterized in that: Step 1.1 specifically includes the following steps: Step 1.1.1: Train the feature extractor G and classifiers F1 and F2 to correctly classify the labeled source domain D. χ For samples, use the standard cross entropy loss for loss calculation and return: Among them, Y is the label space corresponding to the labeled source domain, L ce (D x ,Y) represents the calculation of cross entropy loss for labeled source domain data and its corresponding labels, Represents minimizing the cross entropy loss and updating the network parameters of the feature extractor and the two classifiers; Step 1.1.2, fix the feature extractor G, update the classifiers F1 and F2, and maximize the difference between the classifiers: Among them, D u is the unlabeled source domain, L adv (D u ) is expressed as: Indicates that from D u Sample d in the sample space u , p1 and p2 represent the two classifiers for d u The predicted output of Dis(,) represents the calculation of L1 distance, that is: Dis(p1,p2)=|p1-p2| Step 1.1.
3. Fix the classifier and update the feature extractor to minimize the difference: In step 1.2, the pseudo-label set Q of the unlabeled source domain data is obtained as: Among them, φ represents the parameters of the pseudo-label generation model, p φ represents the prediction of the sample by the pseudo-label generation model, It means taking the category with the highest predicted probability as the pseudo label of the unlabeled source domain data.
4. The method for generalizing a semi-supervised model with fused label denoising according to claim 1, characterized in that: After the training in step 2 is completed, a small-loss strategy is used on the dual calibration generalization model to extract a clean sample set from the pseudo-labeled samples, and the clean samples are linearly mixed with the labeled samples to obtain a set between and The intermediate domain is used to replace the initial The source domain participates in the training of the pseudo-label generation model, improving the quality of pseudo-labels by narrowing the gap between domains.