Online domain adaptive learning method based on multi-head self-integration

Abstract

The invention discloses an online domain adaptive learning method based on multi-head self-integration, and relates to the field of machine learning. The method comprises: pre-training a backbone network of a multi-head model; respectively training each classification head of the multi-head model; performing TTA of multi-head model self-integration on the noise data set; performing optimal integration weight search by using Bayesian optimization; and updating and optimizing the weight of the multi-head model. According to the method, the precision of the model on a distributed out-of-data set is effectively improved, and the practical application capability of the model is enhanced under the condition that the parameter quantity and the calculation quantity of the model are not remarkably increased; and an efficient and practical solution is provided for online domain adaptive learning.

Description

Technical Field

[0001] The present invention relates to the field of machine learning, and in particular to an online domain adaptation learning method based on multi-head self-integration. Background Art

[0002] Deep neural networks have shown impressive performance in many fields, such as image classification, semantic segmentation, and natural language processing. Under certain conditions, especially when the training data and test data are sampled from the same distribution, high accuracy is always guaranteed. In the real world, there is always a distribution shift between the training data and the test data. For example, in an autonomous driving scenario, a control system developed by a manufacturer may be trained using data collected under clear weather conditions, but will be deployed in more complex and challenging environments, such as rainy and foggy days. In this case, the performance of the neural network will usually drop significantly, affecting the normal use of the product function.

[0003] To mitigate the negative impact of distribution shifts, test time adaptation (TTA) has emerged as an important approach. Unlike unsupervised domain adaptation (UDA), TTA does not require any labeled source data. Instead, it focuses on dynamically adjusting the pre-trained model during reasoning based on specific features of unlabeled test data. This allows the model to improve its performance when encountering changes, unseen examples, or changing conditions at test time. Using ensembles can significantly improve performance on OOD data, but deep ensembles exponentially increase the overall size and computation within the model. Recent studies have explored strategies for sharing and reusing parameters and processing across ensemble sub-models. While these techniques reduce memory overhead, they suffer from reduced sub-model diversity and uncertainty calibration. In addition, their computational cost is similar to that of naive deep ensembles, as reasoning still needs to be performed through each individual sub-model.

[0004] Therefore, technicians in this field are committed to developing an online domain adaptation learning method based on multi-head self-ensemble. It can improve the accuracy without increasing the model size, and is expected to be promoted and play its role in other structures. Bayesian optimization is used to search for the optimal weight of the ensemble, and higher accuracy can be obtained without manual parameter adjustment. Summary of the invention

[0005] In view of the above-mentioned defects in the prior art, the technical problem to be solved by the present invention is to improve the accuracy of the TTA algorithm without increasing the number of parameters and the amount of calculation.

[0006] To achieve the above object, the present invention provides an online domain adaptation learning method based on multi-head self-integration, comprising the following steps:

[0007] Step 1: Pre-train the backbone network of the multi-head model;

[0008] Step 2: Train the classification heads of the multi-head model separately;

[0009] Step 3: Perform TTA of multi-head model self-integration on the noisy data set;

[0010] Step 4: Use Bayesian optimization to search for the optimal ensemble weights;

[0011] Step 5: Update the multi-head model weights.

[0012] Furthermore, the step 1 is performed on a standard data set.

[0013] Furthermore, the step 1 comprises the following steps:

[0014] Step 1.1, data preprocessing;

[0015] Step 1.2: Pre-training settings, hyperparameter configuration, and loss function selection;

[0016] Step 1.3: Model training: Freeze the intermediate classification head parameters when pre-training the backbone network.

[0017] Furthermore, the step 2 comprises the following steps:

[0018] Step 2.1: Train the classification head independently.

[0019] Step 2.2, optimize the target and iterate the training;

[0020] Step 2.3: Verify after training.

[0021] Furthermore, the step 3 comprises the following steps:

[0022] Step 3.1, multi-head model self-integration;

[0023] Step 3.2: Improve the accuracy on the OOD dataset;

[0024] Step 3.3: Result evaluation.

[0025] Furthermore, in step 3.1, the multi-head model self-integration directly utilizes multiple classification heads in the same model for integration.

[0026] Furthermore, the step 3.1 comprises the following steps:

[0027] Step 3.1.1, determine the integration goal;

[0028] Step 3.1.2, generate multi-head output;

[0029] Step 3.1.3, set the integration weight;

[0030] Step 3.1.4: Integrate prediction results.

[0031] Furthermore, the step 4 comprises the following steps:

[0032] Step 4.1: Build a proxy model;

[0033] Step 4.2: Define the optimization goal.

[0034] Step 4.3, evaluate the weight combination;

[0035] Step 4.4, iterative optimization;

[0036] Step 4.5: Determine the optimal weight.

[0037] Furthermore, the step 5 includes adjusting the weights of each classification head and the backbone network of the multi-head model.

[0038] Furthermore, in step 5, the weight updating method includes batch normalization and minimizing entropy.

[0039] The conventional integration of TTA existing technology methods increases a lot of storage and computing. The present invention proposes a basic model structure of self-integration. The high-dimensional information of different stages of the network backbone is used to improve the performance in OOD. The present invention significantly improves the performance and provides ideas for model design.

[0040] The multi-head self-integrated model will deteriorate using the traditional training method. The present invention pre-trains the backbone of the multi-head model and continues to train the multi-head classification heads separately. It ensures that the information flow of the backbone will not be compressed prematurely. The present invention enables the multi-head self-integrated model to take effect.

[0041] The multi-head self-ensemble model is different from the traditional ensemble method, which involves the weight ratio problem of each classification head. The present invention uses Bayesian optimization to search for the optimal ensemble weight. Bayesian optimization constructs a proxy model, iteratively selects weight combinations and evaluates their performance, thereby efficiently exploring the weight space. Through this process, the present invention can determine the optimal weight of each classification head in the final ensemble, so that the ensemble result achieves the best performance on the target task.

[0042] Compared with the prior art, the present invention has the following obvious substantial features and significant advantages:

[0043] 1. The present invention innovates in structure and mechanism, improving the problem of traditional deep integration consuming a large amount of memory and computing power, and can significantly improve performance with almost no increase in memory and computing power.

[0044] 2. The present invention can be used on edge devices for online lifelong learning and can adapt to various scenarios with changing weather and light.

[0045] 3. On the basis of performance improvement, the present invention does not increase the amount of calculation and power consumption, and is very friendly to edge devices. It is expected to be deployed on the edge chip of intelligent driving, and the performance, power consumption and memory can be balanced.

[0046] The concept, specific structure and technical effects of the present invention will be further described below in conjunction with the accompanying drawings to fully understand the purpose, characteristics and effects of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a multi-head self-integration algorithm flow of a preferred embodiment of the present invention;

[0048] Figure 2 It is a schematic diagram of model self-integration of a preferred embodiment of the present invention;

[0049] Figure 3 It is a schematic diagram of the staged training of the self-integrated model of a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following describes several preferred embodiments of the present invention with reference to the drawings in the specification, so that the technical content is clearer and easier to understand. The present invention can be embodied in many different forms of embodiments, and the protection scope of the present invention is not limited to the embodiments mentioned in the text.

[0051] In the drawings, components with the same structure are indicated by the same numerical reference numerals, and components with similar structures or functions are indicated by similar numerical reference numerals. The size and thickness of each component shown in the drawings are arbitrarily shown, and the present invention does not limit the size and thickness of each component. In order to make the illustration clearer, the thickness of the components is appropriately exaggerated in some places in the drawings.

[0052] like Figure 1 As shown, the overall flow chart of the algorithm of the present invention.

[0053] This embodiment discloses a multi-head self-integrated online domain adaptation learning method, which aims to improve the accuracy of the TTA algorithm without increasing the number of parameters and the amount of calculation. The method includes the following steps:

[0054] Step 1: Pre-train the backbone of the multi-head model

[0055] Pre-training the backbone network of the multi-head model on standard datasets (such as CIFAR-10, CIFAR-100, Tiny-ImageNet) is a key step. The specific steps are as follows:

[0056] 1. Data preprocessing

[0057] Cropping: Each image is cropped to resize it to the standard size required for model input. This step ensures that the input images have a consistent size so that the model can process them easily.

[0058] Random horizontal flipping: During the training process, the images are randomly flipped horizontally. This enhancement technique can increase the diversity of the data and help the model learn more robust feature representations.

[0059] Normalization: Normalize the pixel values ​​of the image to make it conform to the input standard required by the pre-trained model. This step helps speed up the training of the model and improve stability.

[0060] 2. Pre-training settings

[0061] Hyperparameter configuration: Use the same pre-trained hyperparameters as the baseline model. These hyperparameters usually include learning rate, batch size, optimizer type, etc. For example, use a learning rate of 0.1, a batch size of 128, and an optimizer of SGD (stochastic gradient descent).

[0062] Loss function selection: Select the standard cross entropy loss function to measure the prediction error of the model. The cross entropy loss function is widely used in classification tasks and can effectively measure the difference between the predicted distribution and the actual distribution.

[0063] like Figure 3 As shown, the staged training method of the self-integrated model of the present invention.

[0064] 3. Model training

[0065] Calculating loss: For the special structure of the multi-head model, although it contains multiple classification heads, in the pre-training stage, only the final output and label are used to calculate the loss. This means that only the final classification result is considered, and the output of the intermediate classification head is not concerned.

[0066] Back propagation: After obtaining the loss value, the model parameters are updated through the back propagation algorithm. The back propagation process minimizes the loss function through the gradient descent algorithm, thereby optimizing the model parameters.

[0067] Freeze the parameters of the intermediate classification head: When pre-training the backbone network, freeze the parameters of the intermediate classification head, that is, do not update these parameters. This ensures that during the pre-training phase, the model backbone network focuses on learning common features, and the learning effect of the backbone network will not be affected by the interference of the intermediate classification task.

[0068] Through the above steps, the backbone network of the multi-head model is pre-trained so that it can learn universal feature representations on standard datasets. This lays a solid foundation for subsequent specialized training and optimization, ensuring that the model can efficiently adapt to different tasks and fields.

[0069] Step 2: Continue to train multiple classification heads separately

[0070] After experiments, we found that although training multiple classification heads together can achieve good accuracy, the information is over-compressed in the early stage, resulting in poor accuracy in the subsequent integration stage. Therefore, after completing step 1, each classification head needs to be trained separately. The specific steps are as follows:

[0071] 1. Independently train the classification head

[0072] Individual training: In step 1, the pre-training of the backbone network has been completed. Now we enter the stage of independent training of each classification head. Only one classification head is trained at a time to ensure that the classification head can fully learn the features provided by the backbone network.

[0073] Freeze parameters: When training a classification head, you need to freeze the parameters of the backbone network and other untrained classification heads. Freezing these parameters prevents them from being updated during training, ensuring that the features of the backbone network and other classification heads remain stable.

[0074] Backbone network parameters: Freeze all parameters of the backbone network so that they are not updated during training. This step is to ensure that the feature extraction ability of the backbone network is not disturbed.

[0075] Other classification head parameters: Similarly, freeze the parameters of all untrained classification heads to prevent them from being affected during training.

[0076] Freeze BN layer parameters: In addition, the running_mean and running_var parameters of the batch normalization (BN) layer need to be frozen. These parameters are used to maintain the consistency of the mean and variance during training and inference. Freezing them can prevent changes during training and ensure the stability of the model.

[0077] 2. Training process

[0078] Optimization objective: When training each classification head separately, the standard cross entropy loss function is still used as the optimization objective. By minimizing the loss of the classification head, the parameters of the classification head are optimized.

[0079] Training details: Train one classification head at a time, using an appropriate learning rate and optimizer. During training, evaluate the performance of the classification head regularly to ensure that its performance on a specific task is gradually improving.

[0080] Iterative training: All classification heads are trained separately, and the training process of each classification head needs to go through multiple iterations until its performance on the validation dataset reaches the best state.

[0081] 3. Complete pre-training

[0082] Comprehensive training results: After each classification head is trained, all the trained classification heads are combined together. At this point, the backbone network and all classification heads have been trained independently, and the overall model has good feature extraction and classification capabilities.

[0083] Verify model performance: After completing the training of all classification heads, verify the entire multi-head model to evaluate its comprehensive performance on various tasks. By comparing the verification results, ensure that the performance of the model on various classification tasks meets expectations.

[0084] Through the above steps, the independent training process of the classification head of the multi-head model is completed. This training method can not only avoid excessive compression of information, but also ensure that the prediction accuracy of each classification head is higher in the subsequent integration stage, thereby improving the overall performance of the entire multi-head model.

[0085] like Figure 2 Shown is a schematic diagram of the self-integrated algorithm of the present invention.

[0086] Step 3: Perform TTA of multi-head model self-integration on the noisy dataset

[0087] After completing the pre-training of the backbone network and each classification head, the pre-trained multi-head model is used to conduct a test-time adaptation (TTA) experiment. The multi-head model self-integration method is used, and the specific process is as follows:

[0088] 1. Multi-head model self-integration

[0089] Different from the traditional deep integration of multiple models, multi-head model self-integration directly uses multiple classification heads in the same model for integration. This method can not only improve the accuracy of the model on out-of-distribution (OOD) datasets, but also avoid the problem of doubling the number of model parameters and calculations. The specific steps are as follows:

[0090] Determine the integration target: During the test phase, multiple different versions of the data are generated by performing data augmentation (such as rotation, scaling, cropping, etc.) on the input samples. These augmented data are used to make predictions on the same multi-head model.

[0091] Generate multi-head output: For each enhanced version of the data, use multiple classification heads of the multi-head model to make predictions. Each classification head generates a prediction result, which represents the classification opinion of the model on different feature representations.

[0092] Set ensemble weights: Assign certain weights to the output of each classification head. These weights can be determined through the previous Bayesian optimization step to ensure that the outputs of different classification heads complement each other when integrated, improving overall performance.

[0093] Integrated prediction results: The output results of multiple classification heads are weighted averaged or combined in other forms according to the preset weights. The final prediction result is a collection of weighted outputs of each classification head. In this way, the feature extraction and classification capabilities of different classification heads can be comprehensively utilized to improve the accuracy of the model during testing.

[0094] 2. Improve the accuracy on OOD datasets

[0095] Enhanced robustness: Multi-head model self-ensemble effectively smooths the errors and noise that may occur in a single classification head by combining the prediction results of multiple classification heads, thereby enhancing the robustness of the model on out-of-distribution datasets.

[0096] Avoid over-parameterization: Since the multi-head model is self-integrated within the same model rather than by adding additional independent models, the number of model parameters and the amount of computation will not increase significantly. This approach maintains the computational efficiency of the model while improving accuracy.

[0097] Practical application scenarios: In practical applications, the emergence of out-of-distribution datasets is a common challenge. Through multi-head model self-integration, the performance of the model on these datasets can be significantly improved, thereby enhancing the practical application value of the model.

[0098] 3. Experimental Results Evaluation

[0099] Performance comparison: After completing the TTA experiment, the improvement effect brought by the self-ensemble method is evaluated by comparing the performance indicators (such as accuracy, F1 score, etc.) before and after the multi-head model self-ensemble.

[0100] Further optimization: Based on the experimental results, it may be necessary to further optimize the ensemble weights or adjust the training strategy of the classification head to ensure that the multi-head model can achieve the best performance in various scenarios.

[0101] Through the above steps, the multi-head model self-ensemble method is used for test time adaptation, which can not only effectively improve the accuracy of the model on the out-of-distribution dataset, but also enhance the practical application ability of the model without significantly increasing the number of model parameters and the amount of calculation. This method provides an efficient and practical solution for online domain adaptation learning.

[0102] Step 4: Use Bayesian optimization to search for optimal ensemble weights

[0103] When multi-head models are self-integrated, it is necessary to assign optimal weights to the output of each classification head to ensure the best performance of the final integrated result. In order to find these optimal integration weights, the Bayesian optimization algorithm is used for search. Bayesian optimization constructs a proxy model, iteratively selects weight combinations and evaluates their performance, thereby efficiently exploring the weight space. The specific process is as follows:

[0104] 1. Specific implementation steps

[0105] Build the proxy model:

[0106] Initial weight combination: Select several initial weight combinations and use these combinations to perform ensemble predictions on the multi-head model and evaluate its performance on the validation dataset (such as accuracy, F1 score, etc.).

[0107] Training the proxy model: These initial weight combinations and their corresponding performance indicators are used as sample data to train a Gaussian process (GP) proxy model. This proxy model is used to approximately describe the relationship between the weight combination and the model performance.

[0108] Define optimization goals:

[0109] Exploration strategy: Based on the agent model, an exploration strategy (such as expected improvement (EI), probability improvement (PI) or upper confidence bound (UCB)) is defined to select the next set of weight combinations to be evaluated in the weight space. The exploration strategy comprehensively considers the current knowledge and uncertainty, and balances exploration and exploitation.

[0110] Evaluation weight combination:

[0111] New weight combinations: Based on the exploration strategy, a new set of weight combinations is selected and evaluated on the validation dataset using the multi-head model, recording its performance metrics.

[0112] Update the proxy model: Feedback the new weight combination and its performance to the proxy model, and update the parameters of the proxy model to make it more accurately approximate the performance distribution in the weight space.

[0113] Iterative Optimization:

[0114] Repeat the process: Repeat the above selection, evaluation and update process to iterate and optimize. After each iteration, the proxy model gradually converges and the performance of the weight combination gradually improves.

[0115] Convergence judgment: After multiple iterations, when the prediction performance of the surrogate model is stable and the best weight combination on the validation dataset is found, the iteration is stopped.

[0116] Determine the optimal weights:

[0117] Optimal combination selection: Select the weight combination with the best performance on the validation dataset as the final ensemble weight. These weights will be used in the final ensemble prediction of the multi-head model.

[0118] 2. Bayesian Optimization Formula

[0119] The key formula in the Bayesian optimization process is as follows:

[0120] Construction of proxy model:

[0121] The mean function μ(x) and covariance function k(x,x) of the Gaussian process surrogate model * ):

[0122] μ(x)=E[f(x)]

[0123] k(x,x * )=Cov(f(x),f(x * )]

[0124] Where x represents the weight combination and f(x) represents the model performance under that weight.

[0125] Choose an exploration strategy

[0126] Expected Improvement (EI):

[0127] α EI ((x) = E[(f(x)-f(x) + )) + ]

[0128] Among them, x + is the current best weight combination, (a) + Indicates taking the larger value between a and 0, α EI The expected output.

[0129] Update proxy model x new And its corresponding performance

[0130] μ n+1 (x) = μ n (x)+k(x,xmew ) T K(x new , x new ) -1 (f(x new )-μ n (x new ))

[0131] k n+1 (x,x * ) = k n (x,x * )-k(x,x new ) T K(x new , x new ) -1 k(x new , x * ) T

[0132] In the final integration, the optimal weight w is used i Take a weighted average of the outputs of each classification head:

[0133]

[0134] Among them, y final Represents the final integrated prediction result, y i represents the output of the i-th classification head, w i represents the weight of the i-th classification head, and N is the number of classification heads.

[0135] Through the above process, using Bayesian optimization to search for the optimal ensemble weight, the optimal weight of each classification head in the final ensemble can be determined, so that the ensemble result achieves the best performance on the target task. This step ensures the accuracy and reliability of the ensemble model.

[0136] Step 5: Update the multi-head model weights

[0137] Updating the weights of a multi-head model is an important step to ensure that the model has greater adaptability and predictive power. This process involves adjusting the weights of each classification head and backbone network of the model. Through continuous weight updates and optimizations, the model can continuously improve its performance on different tasks and domains, ultimately achieving the goal of online domain adaptation learning. There are two main methods for weight updates, namely Batch Normalization (BN) and Entropy Minimization.

[0138] 1. Batch Normalization (BN)

[0139] Batch normalization is a technique used to accelerate neural network training and improve its stability. By normalizing each small batch of data, BN can effectively reduce internal covariate shift, stabilize the distribution of gradients, and thus speed up convergence.

[0140] Calculate batch mean and variance:

[0141] During the training process, the mean and variance of the input data are calculated within the current mini-batch to capture the distribution characteristics of the data.

[0142] Normalization:

[0143] The input data is normalized using the calculated mean and variance to have a standard distribution.

[0144] Zoom and pan:

[0145] The normalized data is scaled and translated to restore its expressiveness through learnable parameters that are updated along with other model parameters during training.

[0146] BN can make the performance of the model more stable and consistent in different domains and improve the adaptability of the model.

[0147] 2. Minimize Entropy

[0148] Entropy minimization methods improve the model's performance on new domains by reducing the entropy of the model's predictions to increase their certainty. The lower the entropy, the more certain and confident the model's predictions are.

[0149] Calculate the predicted probability distribution:

[0150] For a given input data, use the model to calculate its predicted probability for each category.

[0151] Calculate entropy:

[0152] For each sample, the entropy of its predicted probability distribution is calculated. The lower the entropy value, the more confident the model's prediction is.

[0153] Minimize entropy loss:

[0154] During the training process, the entropy value is minimized as an additional loss term to improve the certainty of the model prediction by reducing the entropy loss.

[0155] Gradient Update:

[0156] Through the gradient descent algorithm, the model parameters are updated to minimize the total loss (including the original task loss and entropy loss), thereby improving the adaptability of the model on the new domain.

[0157] By combining batch normalization and entropy minimization, the weights of the multi-head model can be effectively updated and optimized, making the model more adaptable and predictive under the new weight configuration. In the process of continuous weight update and optimization, the model can continuously improve its performance on different tasks and domains, and ultimately achieve the goal of online domain adaptation learning. This process not only improves the generalization ability of the model, but also enhances its reliability and stability in practical applications.

[0158] The preferred specific embodiments of the present invention are described in detail above. It should be understood that ordinary technicians in the field can make many modifications and changes based on the concept of the present invention without creative work. Therefore, all technical solutions that can be obtained by technicians in the technical field based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. An online domain adaptation learning method based on multi-head self-ensemble, characterized in that: The following steps are involved: Step 1: Pre-train the backbone network of the multi-head model; Step 2: Train the classification heads of the multi-head model separately; Step 3: Perform TTA of multi-head model self-integration on the noisy data set; Step 4: Use Bayesian optimization to search for optimal ensemble weights; Step 5: Update the multi-head model weights.

2. The online domain adaptation learning method based on multi-head self-ensemble according to claim 1, characterized in that: The step 1 is performed on a standard data set.

3. The online domain adaptation learning method based on multi-head self-ensemble according to claim 1, characterized in that: The step 1 comprises the following steps: Step 1.1, data preprocessing; Step 1.2: Pre-training settings, hyperparameter configuration, and loss function selection; Step 1.3: Model training: Freeze the intermediate classification head parameters when pre-training the backbone network.

4. The online domain adaptation learning method based on multi-head self-ensemble according to claim 1, characterized in that: The step 2 comprises the following steps: Step 2.1: Train the classification head independently. Step 2.2, optimize the target and iterate the training; Step 2.3: Verify after training.

5. The online domain adaptation learning method based on multi-head self-ensemble according to claim 1, characterized in that: The step 3 comprises the following steps: Step 3.1, multi-head model self-integration; Step 3.2: Improve the accuracy on the OOD dataset; Step 3.3: Result evaluation.

6. The online domain adaptation learning method based on multi-head self-ensemble according to claim 5, characterized in that: In step 3.1, the multi-head model self-integration directly utilizes multiple classification heads in the same model for integration.

7. The online domain adaptation learning method based on multi-head self-ensemble according to claim 5, characterized in that: The step 3.1 comprises the following steps: Step 3.1.1, determine the integration goal; Step 3.1.2, generate multi-head output; Step 3.1.3, set the integration weight; Step 3.1.4: Integrate prediction results.

8. The online domain adaptation learning method based on multi-head self-ensemble as claimed in claim 1, characterized in that: The step 4 comprises the following steps: Step 4.1: Build a proxy model; Step 4.2: Define the optimization goal. Step 4.3, evaluate the weight combination; Step 4.4, iterative optimization; Step 4.5: Determine the optimal weight.

9. The online domain adaptation learning method based on multi-head self-ensemble as claimed in claim 1, characterized in that: The step 5 includes adjusting the weights of each classification head and the backbone network of the multi-head model.

10. The online domain adaptation learning method based on multi-head self-ensemble according to claim 1, characterized in that: In step 5, the weight updating method includes batch normalization and minimizing entropy.

Images