Open set domain adaptation method based on feature decoupling and data reconstruction
By employing feature decoupling and data reconstruction, the problems of recognition accuracy and robustness of traditional domain adaptation methods in open set scenarios are solved, achieving more efficient cross-domain adaptation, reducing the false positive rate, and shortening the training time.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING UNIV OF POSTS & TELECOMM
- Filing Date
- 2026-03-09
- Publication Date
- 2026-05-26
AI Technical Summary
Traditional domain adaptation methods rely on strong assumptions when facing open set scenarios, resulting in poor performance in identifying unknown categories in the target domain. Furthermore, gradient reversal layers lose their ability to distinguish categories in the later stages of training, and the loss function design is too simplistic, failing to effectively reflect the dispersion of sample distribution, thus leading to insufficient robustness.
We employ a feature decoupling and data reconstruction approach, utilizing differential data augmentation, end-to-end model construction, dynamic gradient inversion layers, and phased training strategies. Combined with a multi-dimensional loss function, we dynamically adjust the domain adversarial strength and class discriminability. We also adopt a dual-branch structure and a class center management module to achieve robustness in feature extraction and recognition.
It improves the model's recognition accuracy and robustness in open set scenarios, reduces the false positive rate, shortens training time, and enhances cross-domain adaptability.
Smart Images
Figure CN122090079A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to an open set domain adaptation method based on feature decoupling and data reconstruction. Background Technology
[0002] Domain adaptation is a cutting-edge research direction in transfer learning, aiming to solve domain shift and unknown category recognition, and ultimately achieve the goal of acquiring transferable knowledge. Traditional domain adaptation methods can typically only correctly classify known categories in the target domain, requiring a prediction of one of the known categories. However, in practical applications, the target domain categories often contain the source domain categories, meaning there is interference from unknown categories in the target domain, significantly weakening its ability to solve the domain shift problem. Existing technologies have the following shortcomings:
[0003] 1. Reliance on strong distribution assumptions: Many traditional methods are built on strong assumptions, such as the source and target domains satisfying covariate shifts. However, in practical applications, class conditions often change, leading to a significant decrease in adaptation effectiveness. 2. Inability to perform end-to-end joint optimization and poor ability to cross new domains: Redesign or parameter tuning is required for new target domains.
[0004] 2. Traditional gradient inversion layers (GRLs) use fixed strength coefficients. Although strong adversarial training in the early stages can learn domain-invariant features, it can easily lead to the feature extractor losing its ability to distinguish between classes in the later stages, resulting in a decrease in the accuracy of known class recognition.
[0005] 3. The loss function design is too simple: it only relies on classification loss or reconstruction loss; the feature fusion dimension is insufficient: traditional fusion strategies only consider 2-3 dimensions such as confidence and class center distance, and do not include key information such as feature variance that reflects the dispersion of sample distribution, resulting in insufficient robustness of open set discrimination. Summary of the Invention
[0006] This invention aims to provide an open set domain adaptation method based on feature decoupling and data reconstruction to improve the accuracy of model recognition in different scenarios, namely, the model's adaptability to dynamic errors and the model's recognition robustness.
[0007] To achieve the above objectives, we employ the following approach: an open set domain adaptation method based on feature decoupling and data reconstruction, specifically including the following steps:
[0008] S1: First, divide the data into known classes shared by both the source and target domains and unknown classes that exist only in the target domain. Then, use differentiated data augmentation strategies to preprocess the data for different image domains.
[0009] S2: After preprocessing, an end-to-end training model is constructed, including a feature decoupling module, a reconstruction module, and a dynamic gradient inversion layer. The backbone network extracts image features and outputs 2048-dimensional basic features. The dynamic gradient inversion layer is used to learn domain-invariant features and enhance class discriminability. A dual-branch structure is adopted, where the classification branch prevents overfitting, and the domain classification branch accepts the gradient-inverted features and performs binary classification. A class center management module is set up to provide a foundation for subsequent feature alignment.
[0010] S3: Iterative training, phased dynamic training: A phased dynamic strategy is adopted to train the model, including dynamically adjusting the domain adversarial strength, updating class centers by moving average, adding open set contrastive loss, adaptively fusing multi-dimensional features, and gradient accumulation and pruning strategies; the validation set is used to evaluate the model performance during training; the open set contrastive loss categories are optimized to enhance the identification of unknown samples, multi-dimensional features are adaptively fused to match the dynamic distribution of data in training, and gradient accumulation and pruning are used to overcome memory limitations and prevent gradient explosion. After training, the model is tested and validated using validation set data to obtain performance metrics for different models; S4: Model saving and inference: The open set recognition performance metric AUROC of the validation set is used as the basis for saving, and the best model is selected for final inference.
[0011] Furthermore, a preferred embodiment is provided, wherein step S1 includes:
[0012] S1.1: For the Real domain: brightness changes by 0.15, contrast changes by 0.15, saturation changes by 0.1, and hue changes by 0.05;
[0013] S1.2: For the Clipart domain: brightness changes by 0.25, contrast changes by 0.25, saturation changes by 0.2, and hue changes by 0.1.
[0014] Furthermore, a preferred embodiment is provided, wherein step S2 includes:
[0015] S2.1: ConvNeXt-V2-Tiny is used as the backbone network to extract features from the image, outputting 2048-dimensional basic features.
[0016] S2.2: A dynamic GRL module is used to implement progressive domain adversarial training. Gradient inversion is used to maintain high-intensity domain adversarial training in the early stages of training (epoch ≤ 20), allowing the model to learn domain invariance. In the later stages of training (epoch > 20), the intensity is gradually reduced to enhance the model's class discriminative ability. The formula used is...
[0017]
[0018] in is the initial domain adversarial strength, and decay is the decay coefficient.
[0019] S2.3: A two-branch structure is used to process the obtained features. The feature classification branch uses GELU as the activation function to regularize the obtained feature data and prevent overfitting; while the domain classification branch is used to receive the feature data after the gradient is reversed. At the same time, its structure is symmetrical to the classification branch to ensure the fairness of the competition. This branch uses Sigmoid to perform binary classification on the data.
[0020] Furthermore, a preferred embodiment is provided, wherein step S3 includes:
[0021] S3.1: Training Initialization. For the first three epochs at the start of training, initial class centers are calculated using only the source domain data. Each batch is updated using momentum: new center = 0.9 × old center + 0.1 × current batch mean, corresponding to the feature center of the class. Then, every three epochs, all class centers are recalculated using features extracted by the current model, reflecting model evolution and considering the feature distributions of both the source and target domains.
[0022] S3.2: Classify the dataset using two newly added open-set contrastive losses. The weighted sum form of the contrastive loss is used to narrow the distance between similar samples to strengthen the class boundaries; its formula is:
[0023] ;
[0024] Among them, China Let Euclidean distance be the distance between a sample and its own class center. Let be the set of distances between the sample and all known class centers.
[0025] Meanwhile, the model uses unknown class loss to push out unknown class samples to strengthen class distinction, and the formula is as follows:
[0026] ;
[0027] Where margin is the class boundary threshold, and It is the distance between the sample and the nearest known class center.
[0028] S3.3: A multi-dimensional fusion of four losses is implemented, with weights fine-tuned every 5 epochs based on validation set performance. If a loss decreases too rapidly, its weight is appropriately reduced to ensure balanced optimization of the four losses. The weight update logic is as follows:
[0029] progress = current_epoch / total_epochs, weight = initial + (final -initial) * progress;
[0030] Furthermore, step S4 includes: identifying the open set performance metric of the validation set as AUROC in step S4, and saving the weights of the best-performing model that are periodically evaluated starting from the 10th epoch.
[0031] The beneficial effects of this invention are:
[0032] 1. This model integrates dimensions such as classification confidence, class center distance, VAE reconstruction error, and feature variance to improve the model's discrimination ability and achieve more robust open set discrimination. In the dynamic class center update strategy, the model adopts a scheme of updating the class center every 3 epochs, which ensures both stability and adaptability during training. These two strategies reduce the misclassification rate of the model by more than 25% in different cross-domain scenarios, thereby effectively improving the difficulty of using this type of model in cross-domain scenarios.
[0033] 2. In terms of domain adaptation, the data domain is analyzed and processed through feature decoupling and orthogonal constraints, enabling the model to extract domain-invariant features more accurately. At the same time, the model adopts a dynamic gradient reversal mechanism to strengthen its learning of domain-invariant features in the early stage of training and avoid interference with class discrimination in the later stage of training. In addition, a data reconstruction mechanism, namely the VAE reconstruction module, is introduced into the model to assist the model in training through reconstruction error, thereby reducing the difference in MSE between domains to 0.0001 and the feature reconstruction MSE to 0.0208, effectively reducing the difference in cross-domain distribution.
[0034] 3. Regarding the efficiency of training the model, the model adopts the modern convolutional neural network architecture (ConvNeXt-V2-Tiny) as its backbone network architecture, which has stronger computational efficiency and representation capabilities than traditional models. Therefore, the training time is reduced by 30% compared to ResNet50. At the same time, the model achieves 128-batch training through gradient accumulation, which means performing two consecutive forward and backward propagations, processing 64 samples each time. After accumulating gradients, the model parameters are updated all at once. During training, the model can converge after 35 epochs, which greatly improves the training time and cost of the model. Attached Figure Description
[0035] Figure 1 This is a schematic diagram of the model architecture of the present invention; Figure 2 The core performance accuracy curve of the monitoring model on classification tasks; Figure 3Evaluation curves for open sets and domain adaptation;
[0036] Figure 4 Various loss curves;
[0037] Figure 5 Inter-domain feature reconstruction error. Detailed Implementation
[0038] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.
[0039] This embodiment uses the DomainNet dataset as an example, with the source domain being the real image domain (Real) and the target domain being the sketch image domain (Clipart). There are 241 shared classes and 104 unknown classes. The model uses ConvNeXt-V2-Tiny as the base network, with a batch size of 16, 30 training epochs, and an initial learning rate of 1e-4. The entire implementation process includes four main stages: data preparation, model construction, phased training, and model inference.
[0040] The steps of the specific embodiment are as follows;
[0041] Step 1: Data Partitioning and Differentiation Enhancement. An open-set domain adaptation scenario is constructed from the original DomainNet dataset. First, all categories are read and divided into shared and unknown classes at a ratio of 70% to 30%. The source domain retains only shared class samples, while the target domain includes both shared and unknown class samples. A standard CSV label file is generated for subsequent data loading.
[0042] like Figure 1 As shown, the data input layer performs different enhancement operations on the Real domain and Clipart domain images respectively:
[0043] Real Domain: Brightness change 0.15, contrast change 0.15, saturation change 0.1, hue change 0.05, with small enhancements to maintain natural colors and realistic details.
[0044] Clipart domain: brightness change 0.25, contrast change 0.25, saturation change 0.2, hue change 0.1, and an additional ±15° random rotation, with a significant enhancement to accommodate the stylistic diversity of cartoon images.
[0045] Step 2: As Figure 1 As shown, an end-to-end training model is constructed, and the training model includes...
[0046] Shared backbone network: The pre-trained ConvNeXt-V2-Tiny is used as the feature extractor, and the output is 2048-dimensional basic features.
[0047] Dynamic Gradient Reversal Layer (Dynamic GRL): This layer dynamically adjusts the gradient reversal strength based on the training epochs. Its calculation formula is as follows: ;in is the initial domain adversarial strength, and decay is the decay coefficient; the model maintains high-intensity domain adversarial strength when epoch ≤ 20 to learn domain-invariant features; in the later stages of training when epoch > 20, the intensity is gradually reduced to enhance class discriminability.
[0048] The dual-branch structure consists of two branches: a classification branch and a domain classification branch. The classification branch has a 2048-dimensional input, passes through a 1024-dimensional hidden layer, and outputs a 241-dimensional classification result. Each layer uses the GELU activation function, and 40% of the neurons are randomly dropped during training. The domain classification branch follows the gradient inversion layer. Its structure is symmetrical to the classification branch; the 2048-dimensional input passes through a 1024-dimensional hidden layer and outputs a 1-dimensional result, using a sigmoid function to output a binary classification prediction for either the source or target domain. This branch uses the same network depth and width as the classification branch, allowing both tasks to perform adversarial learning with equal model capacity.
[0049] Reconstruction Module: Employs a Variational Autoencoder (VAE) with a latent space dimension of 1024. Both the encoder and decoder are three-layer MLPs using GELU activation functions, without dropout layers. This module assists the model in learning more discriminative feature representations through reconstruction error.
[0050] Class center management module: Maintains the feature centers of each known class for subsequent comparison loss calculation and feature alignment.
[0051] Step 3: Phased Dynamic Training: The first 3 epochs of training are a warm-up phase, using only source domain data to calculate initial class centers. Each batch uses a momentum update method: the feature centers of each class are updated using momentum. The specific formula for updating class centers using moving average is: ;in It is a history center. It is the mean of known class features in the current epoch;
[0052] Every 3 epochs, all class centers are recalculated using the features extracted by the current model, while simultaneously considering the feature distributions of both the source and target domains to adapt to dynamic changes in the feature space. To enhance the model's ability to distinguish between known and unknown classes, two contrastive losses are introduced:
[0053] Unknown class loss: Where margin is the class boundary threshold, and It is the distance between the sample and the nearest known class center.
[0054] To balance the optimization priorities of the two types of samples, a joint loss is introduced. .
[0055] Model optimization involves four types of losses: classification loss, domain adaptation loss, reconstruction loss, and contrastive loss. To ensure that each loss plays an appropriate role in different training stages, its weights are dynamically adjusted every 5 epochs based on the validation set performance, with the adjustment method being a linear change.
[0056] The classification loss weights were gradually reduced from 1.2 to 0.8;
[0057] The domain adaptation loss weight was increased from 0.3 to 0.7;
[0058] The reconstruction loss weight was increased from 0.5 to 0.9;
[0059] The weight of the contrast loss was increased from 0.2 to 0.5.
[0060] The AdamW optimizer is used, with an initial learning rate of 1e-4 and weight decay of 1e-5. The learning rate is scheduled using ReduceLROnPlateau: if the validation loss does not decrease for five consecutive epochs, the learning rate is multiplied by 0.5. To overcome memory limitations, a gradient accumulation strategy is employed: gradients are accumulated every two batches before updating the model parameters, resulting in an equivalent batch size of 32. A gradient clipping threshold of 5.0 is set to prevent gradient explosion.
[0061] Figure 2 The model's accuracy curve on the classification task is shown. It can be seen that the validation accuracy steadily increases with the number of training epochs, reaching 0.8410 at 20 epochs, and finally stabilizing at around 0.8664.
[0062] Figure 3 The evaluation curves for open sets and domain adaptation are presented, where both AUROC and H-score show an upward trend, indicating that the model's ability to detect unknown classes and its overall domain adaptation performance continue to improve. Figure 4 The curves of the various loss functions were recorded. The loss decreased smoothly and a good balance was achieved through dynamic weight adjustment.
[0063] Figure 5 The error curves for cross-domain feature reconstruction were obtained, and the difference in MSE between cross-domain features was eventually reduced to 0.0001, while the MSE for feature reconstruction was reduced to 0.0208, proving that the model effectively reduced the cross-domain distribution difference.
[0064] Step 4: Model Saving and Inference
[0065] Starting from the 10th epoch, the best model is periodically saved using the validation set AUROC as the evaluation metric. The validation set consists of 20% data from both the source and target domains to ensure a balanced class distribution. Classes with fewer than 10 samples are ignored during AUROC calculation; the macro-average is used as the final metric.
[0066] During the inference phase, the model integrates information from four dimensions for open set discrimination: classification confidence, distance between the sample and the nearest class center, VAE reconstruction error, and feature variance. The optimal combination of fusion weights is determined on the validation set using a grid search, with the goal of maximizing AUROC.
[0067] Experimental results
[0068] After 35 epochs of training, the model achieved an accuracy of 86.64% on the validation set, with an AUROC of 0.8869 and an H-score of 0.8214. The MSE for inter-domain feature reconstruction was 0.0208, the orthogonality score was 0.0221, and the GPU memory usage was approximately 6.5GB. Cross-domain testing on the VisDA dataset showed a target H-score exceeding 65%, and an AUROC exceeding 0.75 for unknown class detection. Compared to the traditional ResNet50 baseline, this invention achieves an average improvement of 15%-20% in open-set recognition accuracy, reduces training time by 30%, and lowers the false positive rate by over 25% in cross-domain scenarios, fully validating the effectiveness and robustness of the method.
Claims
1. An open set domain adaptation method based on feature disentanglement and data reconstruction, characterized in that, Includes the following steps: S1: First, the data is divided into known classes shared by both domains and unknown classes that exist only in the target domain. Differential enhancement preprocessing is then performed on different image domains. S2: An end-to-end training model is constructed. The training model includes a backbone network, a dynamic gradient inversion layer, a dual-branch structure, and a reconstruction module. The backbone network extracts image features and outputs 2048-dimensional basic features. The dynamic gradient inversion layer is used to learn domain-invariant features and enhance class discriminability. The dual-branch structure includes a classification branch and a domain classification branch. The classification branch is used to prevent overfitting; the domain classification branch accepts the features after gradient reversal and performs binary classification, and sets up a class center management module to provide a basis for subsequent feature alignment. S3: A phased dynamic strategy is used to train the model, including dynamically adjusting the domain adversarial strength, updating class centers by moving average, adding open set contrastive loss, adaptively fusing multi-dimensional features, and gradient accumulation and clipping. The model performance is evaluated using a validation set during training. S4: Use the open set recognition performance index of the validation set as the basis for saving, and select the best model for final inference.
2. The open set domain adaptation method based on feature disentanglement and data reconstruction of claim 1, wherein, The differential enhancement preprocessing in S1 specifically involves the following for the Real domain image: brightness change of 0.15, contrast change of 0.15, saturation change of 0.1, and hue change of 0.
05. For Clipart domain images: brightness changes by 0.25, contrast changes by 0.25, saturation changes by 0.2, and hue changes by 0.
1.
3. The open set domain adaptation method based on feature decoupling and data reconstruction according to claim 1, characterized in that, The gradient inversion intensity calculation formula of the dynamic gradient inversion layer in the S2 is: ; wherein is the initial domain antagonistic intensity, and decay is the decay coefficient. The model maintains high-intensity domain adversarial learning to learn domain-invariant features when epoch ≤ 20. In the later stages of training, when the number of epochs exceeds 20, the intensity is gradually reduced to enhance class discrimination.
4. The open set domain adaptation method based on feature disentanglement and data reconstruction of claim 1, wherein, The reconstruction module in S2 is a variational autoencoder, configured as follows: the latent space dimension is 1024, the activation function is GELU, and it does not contain a Dropout layer.
5. The open set domain adaptation method based on feature decoupling and data reconstruction as described in claim 1, characterized in that, The specific formula of the sliding average update class center in S3 is: ; wherein is the historical class center, is the known class feature mean of the current epoch; the sliding average is updated every 3 epochs with a weight of 0.
1.
6. The open set domain adaptation method based on feature decoupling and data reconstruction according to claim 1, characterized in that, The open set contrast loss in S3 includes known class contrast loss and unknown class contrast loss: a contrast loss in the form of a weighted sum. ;in Let Euclidean distance be the distance between a sample and its own class center. The set of distances between a sample and all known class centers; in the model, contrastive loss is used to bring samples of the same class closer together to strengthen class boundaries; unknown class loss: Where margin is the class boundary threshold, and It is the distance between the sample and the nearest known class center.
7. The open set domain adaptation method based on feature decoupling and data reconstruction according to claim 6, characterized in that, To balance the optimization priorities of the two types of samples, a joint loss is introduced. .
8. The open set domain adaptation method based on feature decoupling and data reconstruction according to claim 1, characterized in that, The adaptive fusion of multi-dimensional features in S3 refers to fusing information from four dimensions—classification confidence, class center distance, reconstruction error, and feature variance—during the model inference stage. This information is then dynamically selected on the validation set using multiple sets of preset weights, with the final selection based on the open set recognition metric AUROC.
9. The open set domain adaptation method based on feature decoupling and data reconstruction according to claim 1, characterized in that, The open set recognition performance metric for the validation set mentioned in step S4 is AUROC, and the weights of the best performing model are stored and evaluated periodically starting from the 10th epoch.
10. The open set domain adaptation method based on feature decoupling and data reconstruction according to claim 1, characterized in that, The backbone network adopts ConvNeXt-V2-Tiny and outputs 2048-dimensional basic features.