A multi-initialization meta-learning framework and method based on domain adaptation

Abstract

The present invention provides a multi-initial value meta-learning framework and method based on domain adaptation. The framework includes a cross-domain encoder, which encodes input data into a common feature vector through a common encoder and into a private feature vector through a private encoder; a cross-domain modulation network, which encodes the common feature vector into a domain-common modulation vector and encodes the private feature vector into a domain-specific modulation vector; a meta-separation network, which is used to update a meta-learner in a source domain and a target domain, wherein the parameters of the meta-learner are divided into a common part of the parameters modulated by the domain-common modulation vector and a private part of the parameters modulated by the domain-specific modulation vector. The learning framework and method can improve the accuracy of the algorithm in the few-sample problem to a certain extent, and are widely applicable to the meta-learning of cross-domain data.

Description

Technical Field

[0001] The present invention relates to the field of meta-learning technology, and in particular to a domain-adaptive based multi-initialization value meta-learning framework and method. Background Art

[0002] Artificial intelligence has been widely used in various technical fields. The basic problem is that it cannot learn as efficiently as humans. It needs to be trained and learned with training samples. The more perfect and more training samples there are, the better the results of the trained artificial intelligence model will be. However, in actual practice, there is often a problem of insufficient number of training samples. Therefore, how to conduct effective few-sample learning has become a highly concerned issue in the field of artificial intelligence learning.

[0003] Meta-learning is an effective method to solve the problem of few-shot learning. Meta-learning can also be understood as "learning how to learn". Existing meta-learning methods include metric learning-based methods, meta-optimization-based methods, and cyclic model-based methods. However, the loss functions of these meta-learning methods are only related to specific tasks, and there are no domain-independent or domain-adaptive constraints. Therefore, these methods perform well on single-domain tasks, but have the defect of insufficient generalization performance on cross-domain data.

[0004] Specifically, the different distributions of different categories in the meta-testing stage and the meta-training stage lead to the problem of domain conversion in the meta-learning method. Although the multi-initialization technology has been successful in identifying task modes, it still cannot solve the defect of insufficient generalization in cross-domain fields caused by the domain shift generated by different category distributions. In other words, the existing domain adaptation methods can only adapt the meta-learning method to the unimodal meta-testing domain, but not to the multimodal meta-testing domain. So far, how to alleviate the domain conversion between the meta-training and meta-testing stages in the multimodal setting remains a challenge.

[0005] In summary, current meta-learning methods are not well suited for learning cross-domain data, which limits their application in application scenarios that require cross-domain data. Summary of the invention

[0006] The purpose of the present invention is to provide a domain-adaptive multi-initial value meta-learning framework and method, which can be widely applied to meta-learning of cross-domain data and improve the accuracy of the algorithm in few-sample problems to a certain extent.

[0007] To achieve the above objectives, the present technical solution provides a multi-initialization meta-learning framework and method based on domain adaptation, and the multi-initialization meta-learning framework based on domain adaptation includes:

[0008] The cross-domain encoder encodes the input data into a shared feature vector through a shared encoder and into a private feature vector through a private encoder;

[0009] A cross-domain modulation network encodes the shared feature vectors into domain-common modulation vectors and the private feature vectors into domain-specific modulation vectors;

[0010] A meta-separation network is used to update the meta-learner in the source and target domains, where the parameters of the meta-learner are divided into a common part of parameters modulated by a domain-common modulation vector and a private part of parameters modulated by a domain-specific modulation vector.

[0011] The calculation formula of the loss function of the cross-domain encoder is:

[0012] L e =L r +L d +L s ,

[0013] Where L r is the reconstruction error loss function. By reducing the reconstruction error, the information extracted by the encoder is kept as much as possible before encoding. d is a cross-domain difference loss function, which increases the difference between the common feature vector and the private feature vector output by the encoder. s It is a cross-domain similarity loss function, which makes the cross-domain common feature vectors output by the encoder closer. The training goal of this scheme is to minimize the total loss of the cross-domain encoder parameters.

[0014] Reconstruction error loss function L r The specific formula is as follows:

[0015]

[0016] In this formula, For input data, is the predicted value after reconstructing the input data, s is the source domain, t is the target domain, and d is the set of the target domain and the source domain;

[0017] Cross-domain difference loss function L d The specific formula is as follows:

[0018]

[0019] in is the common eigenvector The matrix formed, is a private feature vector The matrix formed by , F is the F norm.

[0020] Cross-domain similarity loss function Ls The specific formula is as follows:

[0021]

[0022] Where m i For input data The field label, For input data The predicted domain label, N S is the number of source domain data, N t is the amount of target domain data.

[0023] The formula for the cross-domain modulation network to encode the common feature vector into a domain-common modulation vector is:

[0024]

[0025] in is the common eigenvector The corresponding domain common modulation vector, ω g is the cross-domain modulation network parameter.

[0026] The formula for the cross-domain modulation network to encode the private feature vector into a domain-specific modulation vector is:

[0027]

[0028] in is a private feature vector The corresponding domain-specific modulation vector, ω g is the cross-domain modulation network parameter.

[0029] The formula for the common part of the update parameters of the meta-separation network is as follows:

[0030]

[0031] in is the common part of the parameters of the i-th layer network, is the domain common modulation vector of the i-th layer network, θ ci is the common part of the parameters of the element separation network at the i-th layer, ⊙ is the modulation operator;

[0032] The formula for updating the private part of the parameters of the meta-separation network is as follows:

[0033]

[0034] in is the private part of the parameters of the i-th layer network, is the domain-specific modulation vector of the i-th layer network, is the parameter-private part of the meta-separation network at layer i, and ⊙ is the modulation operator.

[0035] Specifically, since the low and high layers of the neural network show different types of information. The low-level network often has good migration, which is not targeted at a specific task, but has universality for different tasks, and in the process of the network from low to high, its characteristics also transition from generality to specificity. Based on this important phenomenon, the present invention designs three parts: a cross-domain encoder, a cross-domain modulation network, and a meta-separation network. Among them, the common part of the parameters of the meta-separation network is the low-level network parameters, and the low-level networks of different tasks are shared across domains for meta-learning and joint training, and are modulated by domain-common modulation vectors; the private part of the parameters of the meta-separation network is the high-level network parameters, which are specific to a single task, so they are trained separately in different domains and modulated by domain-specific modulation vectors. By effectively utilizing the universality of the low-level network, the present invention can improve the generalization performance of meta-learning for cross-domain data; by effectively utilizing the specificity of the high-level network, the present invention can also improve the prediction speed and efficiency of meta-learning for cross-domain data. Since meta-learning is widely used to solve the few-sample problem, the present invention can be applied to the few-sample problem in various application fields.

[0036] The structure of the multi-initialization meta-learning model based on domain adaptation is as follows: Figure 1 As shown, data X in different domains s and X t The common feature vector is obtained by inputting the common encoder of the cross-domain encoder and The common feature vector is input into the cross-domain modulation network and converted into a domain common modulation vector and Data in different domains X s and X t The private feature vectors are obtained by inputting the private encoder of the cross-domain encoder respectively. and The private feature vector is input into the cross-domain modulation network and converted into a domain-specific modulation vector and The meta-separation network updates the meta-learner using the common part of the parameters modulated by the domain-common modulation vector and the private part of the parameters modulated by the domain-specific modulation vector in different domains. The cross-domain modulation network is used to identify the mode of the multimodal task, where the cross-domain encoder is used to extract the hidden representation, and then the network is used to find the domain-common and domain-private modulation vectors. Since the modulation vector implies the mode information of the task, the meta-separation network takes these modulation vectors as input to learn a good multi-initialization meta-learner to achieve rapid adaptation to new multimodal tasks. It can be seen that the meta-separation network learns a cross-domain meta-learner by using the modulation vectors of the source domain and the target domain respectively, where the common modulation vector is used in the lower layer to learn the domain-invariant knowledge, and the specific modulation vector is used in the high layer to learn the knowledge of the specific domain. Finally, in terms of generalization, the shared encoder is used to obtain the features of Dtest to their modulation vectors, and then the meta-learner learned in the target domain is used to achieve rapid adaptation.

[0037] In a second aspect, the present invention provides a domain-adaptive-based multi-initial value meta-learning method, which uses the domain-adaptive-based multi-initial value meta-learning model to learn, including the following steps:

[0038] Initialization: Randomly initialize the meta-separation network parameters, cross-domain encoder parameters, and cross-domain modulation network parameters;

[0039] Data sampling: Sample support set and query set data from the source domain and target domain data respectively;

[0040] Obtaining the parameters of the meta-separation network that is adaptive to the support set: input the support set data into a cross-domain encoder, and output a shared feature vector and a private feature vector; input the shared feature vector and the private feature vector into a cross-domain modulation network, and output a domain-common modulation vector and a domain-specific modulation vector; modulate the domain-common modulation vector to obtain the public part of the meta-separation network parameters, and modulate the domain-specific modulation vector into the private part of the meta-separation network parameters to obtain the modulated meta-separation network; input the support set data into the modulated meta-separation network, and calculate the first network gradient; use the first network gradient to update the meta-separation network parameters, and obtain the meta-separation network parameters that are adaptive to the support set, and traverse the support set data of the source domain and the target domain.

[0041] Update network parameters: According to the error of the query set on the meta-separation network adapted to the support set, calculate the second gradient, and update the public part of the meta-separation network parameters, the private part of the meta-separation network parameters, the cross-domain encoder, and the cross-domain modulation network. Post-regression performs the meta-separation network parameter acquisition step that is adaptive to the support set until the network converges and outputs all network parameters.

[0042] The formula for calculating the first network gradient is as follows:

[0043]

[0044] Where θ is the initialization element separation network parameter, T j is the sampled data, To support the set data;

[0045] The common part of the network parameters is updated:

[0046]

[0047] Update the private part of the meta-separated network parameters:

[0048]

[0049] Update cross-domain encoder

[0050]

[0051] Update cross-domain modulation network:

[0052]

[0053] Compared with the prior art, the beneficial effects and characteristics of the technical solution are as follows: First, a new domain-adaptive multi-initialization meta-learning method based on a single-dispersed network structure is proposed to improve the performance of meta-learning on multimodal tasks. For the problem of multi-initialization domain shift, a cross-domain modulation network based on a thin film network is proposed to encode the task into domain-common and domain-private modulation vectors. Based on the generated modulation vectors, a new meta-separation network (MSN) is proposed to update the meta-learner in the source and target domains. The parameters of the meta-learner are divided into a common part of the parameters modulated by the domain-common modulation vector and a private part of the parameters modulated by the domain-specific modulation vector. In addition, the shared common parameters in meta-learning are learned by the first few layers, while the private parameters are learned by the latter layers. The key reason is that the lower layers may generate common features, while the higher layers may learn specific features. In addition, the inequality measure is incorporated into the update process of meta-learning to further improve the generalization ability of meta-learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a schematic diagram of a multi-initial value meta-learning model framework based on domain adaptation according to the present invention.

[0055] Figure 2 This is a pseudocode diagram of the multi-initialization meta-learning method based on domain adaptation.

[0056] Figure 3 This is an experimental diagram used for reinforcement learning experiments. DETAILED DESCRIPTION

[0057] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0058] Example:

[0059] We employ a wide range of experiments, including regression, image classification, and reinforcement learning (RL), to evaluate the proposed approach on a variety of multimodal few-shot learning tasks. For comparison, we consider the following meta-learning methods as references:

[0060] 1.MAML: MAML is a representative of traditional model-independent meta-learning algorithms, which have been applied in a wide range of research fields;

[0061] 2. Multi-MAML: It consists of multiple MAML models, each of which is specially trained for tasks sampled from a single modality. It is worth noting that Multi-MAML is evaluated on a single modality, which means that the method does not need to distinguish the modality like other methods, which is impossible in practical applications: it is impossible to know in advance what modality the upcoming data is. Therefore, its performance is the upper bound of the performance of the MAML algorithm in the case of accurate modality identification, and its performance cannot be obtained in practice.

[0062] 3. MMAML: As a recent research result, MMAML applies feature linear modulation (FiLM) to identify the pattern of the task and then adjusts the meta-learner parameters to produce multiple initializations.

[0063] The method of this scheme is represented by MIML-DA.

[0064] Regression experiment:

[0065] Experimental conditions: To conduct a multimodal few-shot regression experiment, the applicant extracts five pairs of input and output data {x k ,y k} k＝1....K,Constructing multi-modal task assignment,Five different functions are considered in this scheme: sine function, linear function, quadratic function, conversion norm and hyperbolic tangent function, all of which are regarded as discrete task modes.,This scheme trains the model on mixed datasets of two modes (linear and sine function), three modes (quadratic, linear and sine function) and five modes (all five functions). For each task, 5 pairs of data are sampled, and then Gaussian noise is added to the output value to further increase the difficulty of identifying the function that generated the data. Gaussian noise with μ = 0 and σ = 0.3 is applied.

[0066] Experimental methods:

[0067] We first consider three baseline methods: MAML, Multi-MAML, and MMAML. All three methods have meta-networks. MMAML further enhances the meta-network with a modulation network. First, the data points sorted by x-values ​​are fed into the modulation network to generate a task-specific modulation vector, which is used to modulate the meta-separation network. Then, the modulated meta-separation network is further adjusted.

[0068] Experimental Results: As shown in Table 1, Table 1 illustrates the performance of our method and other baseline methods in terms of mean squared error (MSE), where the lowest value in each case is highlighted in bold. The results show that the proposed MIML-DA achieves the best performance in all cases. More specifically, traditional MAML has the largest error in all cases, and Multi-MAML with task identity significantly outperforms MAML, which indicates that MAML degrades under multimodal task distribution. Since the vectors produced by the modulation network hint at the mode of the input data, the modulated meta-learner can be better initialized. Therefore, gradient-based optimization methods can achieve better performance in this case, so MIML-DA and MMAML perform significantly better than LSTM learners. Finally, MIML-DA outperforms MMAML because the cross-domain modulation network and meta-separation network reduce the domain shift between the training and testing phases, thereby improving generalization ability.

[0069] Table 1

[0070]

[0071] Image classification experiment:

[0072] Conduct multi-modal few-shot image classification experiments, this classification task considers classifying images into N classes, where the number of labeled available samples is K, called N-way-shot classification; create a multimodal task similar to Triantafliou et al., combining multiple widely used datasets to form a meta-dataset composed of OmniglotLake et al. The applicant trains models on the meta-dataset, including two modes (Omniglot and Mini Imagenet), three modes (Omniglot, Mini Imagenet and FC100), and five modes (all five datasets).

[0073] The overall results are shown in Table II. It is observed that the proposed MIML-DA method achieves the best performance in almost all cases, except for one value. In general, the performance comparison between classification methods is similar to that of regression methods. As the number of modes increases, the performance gap between MIML-DA and the baseline becomes larger, indicating that our method is better able to handle multimodal task distributions. It is worth noting that Multi-MAML achieves good performance because each Multi-MAML is likely to overfit to a single dataset with fewer classes. In contrast, MMAML and MIML-DA learn models from all datasets. The results show that MMML-DA performs slightly better than MMAML and MMAML due to the characteristics of the modulation network, and MIML-DA performs better than MMAML and MMAML due to the characteristics of the cross-domain modulation network and MSN.

[0074] Table 2

[0075]

[0076] Reinforcement Learning Experiments:

[0077] We validate MIML-DA’s ability in multimodal meta-reinforcement learning on the MuJoCo physics simulator to adapt to new tasks based on limited experience. Figure 3 In the three environments of , the agent is rewarded at each time point to minimize the distance to an unknown goal sampled from a multimodal distribution.

[0078] We use ProMP instead of MAML as our baseline. In addition, the baseline Multi-ProMP uses the ProMP method proposed by Vuorio et al. (2019) to train a policy for each mode. Due to the symmetric distribution of the task and the random initialization, the agent only receives training moves of one mode. ProMP is also used to optimize the policy and modulation networks of MMAML and MIML-DA.

[0079] The results are shown in Tables III, IV, and V. As observed, in all three environments, MIML-DA consistently outperforms ProMP and MMAML in various modes. It is worth noting that Multi-ProMP shows good performance since each Multi-ProMP only considers a single mode.

[0080] Table 3

[0081]

[0082] Table 4

[0083]

[0084] Table 5

[0085]

[0086] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A multi-initialization meta-learning method based on domain adaptation, characterized in that: include: The cross-domain encoder encodes the input data into a shared feature vector through a shared encoder and into a private feature vector through a private encoder; A cross-domain modulation network encodes the shared feature vectors into domain-common modulation vectors and the private feature vectors into domain-specific modulation vectors; A meta-separation network for updating a meta-learner in a source domain and a target domain, wherein the parameters of the meta-learner are divided into a common part of the parameters modulated by a domain-common modulation vector and a private part of the parameters modulated by a domain-specific modulation vector; Initialization: Randomly initialize the meta-separation network parameters, cross-domain encoder parameters, and cross-domain modulation network parameters; Data sampling: Sample support set and query set data from the source domain and target domain data respectively; Acquisition of meta-separation network parameters adaptive to support set: inputting support set data into a cross-domain encoder, outputting a common feature vector and a private feature vector; inputting the common feature vector and the private feature vector into a cross-domain modulation network, outputting a domain-common modulation vector and a domain-specific modulation vector; modulating the domain-common modulation vector to obtain a common part of the meta-separation network parameters, modulating the domain-specific modulation vector into a private part of the meta-separation network parameters, and obtaining a modulated meta-separation network; Input the support set data into the modulated meta-separation network and calculate the first network gradient; use the first network gradient to update the meta-separation network parameters to obtain the meta-separation network parameters that are adaptive to the support set, and traverse the support set data of the source domain and the target domain; Update network parameters: According to the error of the query set on the meta-separation network adapted to the support set, calculate the second gradient, and update the public part of the meta-separation network parameters, the private part of the meta-separation network parameters, the cross-domain encoder, and the cross-domain modulation network. Post-regression performs the meta-separation network parameter acquisition step that is adaptive to the support set until the network converges and outputs all network parameters.

2. The domain adaptive multi-initial value meta-learning method according to claim 1, characterized in that: The loss function of the cross-domain encoder is calculated as: L e =L r +L d +L s ; Reconstruction error loss function L r The specific formula is as follows: In this formula, For input data, is the predicted value after reconstructing the input data, s is the source domain, t is the target domain, and d is the number of input data; Cross-domain difference loss function L d The specific formula is as follows: in is the common eigenvector The matrix formed, is a private feature vector The matrix formed; The cross-domain similarity loss function is as follows: Where m i For input data The field label, For input data The predicted domain label, N S is the number of source domain data, N t is the amount of target domain data.

3. The domain adaptive multi-initial value meta-learning method according to claim 1, characterized in that: The formula for the cross-domain modulation network to encode the common feature vector into a domain-common modulation vector is: in is the common eigenvector The corresponding domain common modulation vector, ω g is the cross-domain modulation network parameter; The formula for the cross-domain modulation network to encode the private feature vector into a domain-specific modulation vector is: in is a private feature vector The corresponding domain-specific modulation vector, ω g is the cross-domain modulation network parameter.

4. The domain adaptive multi-initial value meta-learning method according to claim 1, characterized in that: The formula for updating the common part of the parameters of the meta-separation network is as follows: in is the common part of the parameters of the i-th layer network, is the domain common modulation vector of the i-th layer network, θ ci is the common part of the parameters of the element separation network at layer i, and e is the modulation operator; The formula for updating the private part of the parameters of the meta-separation network is as follows: in is the private part of the parameters of the i-th layer network, is the domain-specific modulation vector of the i-th layer network, is the parameter-private part of the meta-separation network at layer i, and e is the modulation operator.

5. The domain adaptive multi-initial value meta-learning method according to claim 1, characterized in that: The common part of the parameters of the meta-separation network is the low-level network parameters. The low-level networks of different tasks are shared across domains for meta-learning and joint training, and are modulated by the domain-common modulation vector. The private part of the parameters of the meta-separation network are high-level network parameters, specific to a single task, modulated by a domain-specific modulation vector.

6. The domain adaptive multi-initial value meta-learning method according to claim 1, characterized in that: The formula for calculating the first network gradient is as follows: Where θ is the initialization element separation network parameter, T j is the sampled data, To support the set data.

7. The domain adaptive multi-initial value meta-learning method according to claim 1, characterized in that: The common part of the network parameters is updated: Update the private part of the meta-separated network parameters: Update cross-domain encoder Update cross-domain modulation network:

Images