An unsupervised meta-learning method based on visual causal intervention

Abstract

The application discloses an unsupervised meta-learning method based on visual causal intervention, which comprises an unsupervised meta-training stage and a supervised meta-testing stage. Specifically, in the meta-training stage, the method uses unlabeled support set samples to learn the causal relationship among input data, prior knowledge and predicted output and the causal relationship between hidden variables in the prior knowledge, eliminate the negative influence of context bias in the input data, and finally obtain unbiased meta-knowledge. In the meta-testing stage, the method uses the above obtained meta-knowledge as the initialization parameters of the model, combines supervised and unsupervised learning under the condition of a small amount of labeled support set samples, further eliminates the influence of data bias, and can accurately predict the label of the query set sample. The method can accurately predict the category of the output image in the few-shot image classification problem of the downstream task.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and classification recognition technology, and in particular relates to an unsupervised meta-learning method based on visual causal intervention. Background Technology

[0002] Unsupervised meta-learning methods have been widely researched and applied. Their aim is to perform meta-learning on unlabeled meta-training datasets to quickly adapt the learned meta-knowledge to new supervised tasks during meta-testing. Especially in cases with few samples, where the supporting dataset for meta-training or meta-testing consists of only a few images, unsupervised meta-learning methods can effectively prevent overfitting and achieve better results. With the rapid development of deep convolutional neural networks, researchers have gradually proposed several unsupervised meta-learning methods based on deep convolutional neural networks. At the same time, unsupervised meta-learning methods can reduce dependence on labeled data, thus attracting increasing attention from industry.

[0003] Current unsupervised meta-learning methods based on deep convolutional neural networks focus on using clustering models during meta-training to fuse features from unsupervised learning, generating reliable pseudo-labels for unlabeled images, and then constructing supervised meta-learning methods based on these pseudo-labels. While this approach has proven simple and effective to some extent, it fails to consider potential data biases inherent in each task during meta-training or meta-testing. Training bias typically arises from insufficient supporting data in a task, leading to a data distribution that doesn't adequately represent the true distribution of the entire dataset. This results in inaccurate generated pseudo-labels, hindering the learning of meta-knowledge.

[0004] Therefore, there is an urgent need to propose an unsupervised meta-learning method based on visual causal intervention to address the data bias problem in the meta-training or meta-testing stages. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention aims to provide an unsupervised meta-learning method based on visual causal intervention. This method eliminates data bias present in meta-training or meta-testing by learning latent visual factors and the causal relationships between them. The modeling part involves deep convolutional neural coding networks, deep convolutional neural decoding networks, and Gaussian mixture models. The optimization part involves learning directed acyclic graphs, maximizing the lower bound of evidence, and solving the Gaussian mixture model.

[0006] To achieve the above objectives, the present invention employs the following technical solution:

[0007] An unsupervised meta-learning method based on visual causal intervention includes the following steps:

[0008] (1) Construct a causal structure model graph for unsupervised meta-learning to obtain the causal relationship between input data, prior knowledge and predicted output;

[0009] (2) Define a one-to-one correspondence between prior variables and prior knowledge, define a causal relationship between prior variables, and express this causal relationship using a learnable directed acyclic graph;

[0010] (3) Define a causal space, encode the input image into a latent variable corresponding to the prior variable in the causal space, and the structure of the directed acyclic graph of the causal space remains unchanged when the latent variables corresponding to multiple images are added together.

[0011] (4) Define the intervention formula. Based on the directed acyclic graph, decompose the joint probability distribution of prior knowledge into the product of the conditional probability distributions of each causally related latent variable.

[0012] (5) Encode all support set input images in the current task to obtain the corresponding latent variables, construct an unsupervised Gaussian mixture model in the causal space, maximize the likelihood probability of the latent variables in an unsupervised manner, decode the latent variables to obtain the corresponding generated images, and finally maximize the similarity between the generated images and the input images to optimize the model.

[0013] (6) Encode all support set input images and query set input images in the current task to obtain the corresponding latent variables, construct a Gaussian mixture model located in the causal space, and maximize the likelihood probability of the latent variables using a semi-supervised approach to obtain the class probability of the query set input images.

[0014] Further, step (1) specifically involves: in the causal structure model diagram of unsupervised meta-learning, the input data is the cause of the predicted output, and the prior knowledge is the cause of both the input data and the predicted output; and the prior knowledge is a confounding variable, which can lead to context bias.

[0015] Furthermore, the directed acyclic graph model in step (2) is obtained by solving the equation:

[0016] D i =h i (D)+U i D j =h j (D)+U j ;

[0017] Among them, D i D j Let U be the i-th and j-th prior variables in prior knowledge D. i U j For a tolerable error that follows a Gaussian distribution, h iand h j It is a function that characterizes the causal structure.

[0018] Furthermore, the causal space in step (3) specifically refers to the assumption that there are n latent variables Z∈R located in the causal space. n×d And belonging to the same directed acyclic graph structure h, if a weight w∈R n×1 If |w|=1, then its weighted sum If it follows a Gaussian distribution, then its expression is:

[0019]

[0020] in This indicates a Gaussian distribution.

[0021] Further, the intervention formula in step (4) is as follows: For the input image x, the latent variable z ~ p(z|x) is obtained by encoder and sampling, then input into the directed acyclic graph function h, and constrained by p(z|h(z)), and finally the predicted y after intervention is obtained by classifier:

[0022] p(y|do(x))=E p(z|x) E p(z|h(z)) p(y|z)

[0023] Where do(·) is the intervention symbol.

[0024] Furthermore, the unsupervised Gaussian mixture model in the causal space in step (5) is specifically as follows:

[0025]

[0026] Where π is the K-dimensional weight, μ [K] , Let be the mean and diagonal covariance of the k-th mixed mode, scalar It is a scaling parameter.

[0027] Furthermore, maximizing the likelihood probability of the latent variables in steps (5) and (6) can be obtained by solving the following optimization equation:

[0028]

[0029] in, The closed-set solution method is to iteratively use an unsupervised expectation-maximization step.

[0030] Furthermore, in step (5), maximizing the similarity between the generated image and the input image can maximize the following lower bound of evidence:

[0031]

[0032] in, Let e ​​be the variational posterior distribution, and e be the encoding sampled in the causal latent space. q φ (e|z, x) and It is a posteriori network q φ Factorize (e, z|x), where p(e|z) is the prior distribution of causal structure in the causal space. A priori distribution in causal space specific to the task.

[0033] Furthermore, in step (6), the category probabilities of the input images in the query set are calculated using Bayes' rule:

[0034]

[0035] in, This represents the probability that the predicted category y is category c, given the latent variable z. The distribution of latent variables in the Gaussian mixture model for the current task. This represents the distribution of class modes in a Gaussian mixture model. Let represent the distribution of latent variables in the c-class mode of the Gaussian mixture model.

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0037] This invention proposes an unsupervised meta-learning method based on visual causal intervention. It constructs a causal structure model diagram for unsupervised meta-learning, analyzes the origins of data bias in meta-training and meta-testing, and defines causal relationships between prior variables, a causal space, and an intervention formula. Finally, to apply visual causal intervention to unsupervised meta-learning, it utilizes Gaussian mixture models in both unsupervised and semi-supervised learning methods. This approach addresses the data bias problem in unsupervised meta-learning, enabling classification with higher accuracy in few-shot learning and demonstrating significant practical value. Attached Figure Description

[0038] Figure 1 This is a flowchart of the unsupervised meta-learning method based on visual causal intervention of the present invention;

[0039] Figure 2 The diagram shows the meta-training and meta-testing structure in this embodiment of the invention, where (a) is the training phase diagram of unsupervised meta-learning; and (b) is the testing phase diagram of supervised meta-learning. Detailed Implementation

[0040] To make the motivation, technical details, and methodological advantages of this invention clearer, the invention will be further described below in conjunction with the accompanying drawings and embodiments. It should be noted that the specific embodiments described herein are only for clearly explaining the invention and are not intended to limit the invention. Rather, any embodiments made based on the rights or methods covered by this invention fall within the scope of this invention. For the convenience of those outside the art, relevant details will be described in detail in these embodiments.

[0041] This invention provides an unsupervised meta-learning method based on visual causal intervention. The method includes a deep convolutional neural network (DCNN) encoder, a causal space for intervention, and a DCNN decoder. The output of the DCNN encoder is projected onto the causal space for intervention, and the output of the causal space for intervention is connected to the input of the DCNN decoder. The DCNN is used to encode latent variables in meta-training or meta-test data. The causal space for intervention is used to learn the causal relationships between latent variables and to perform intervention based on these relationships. The DCNN decoder is used to decode the latent variables after intervention to obtain generated data.

[0042] refer to Figure 1 The flowchart of the unsupervised meta-learning method based on visual causal intervention described in this invention includes the following steps:

[0043] (1) Construct a causal structure model diagram for the unsupervised meta-learning method to obtain the causal relationship between input data X, prior knowledge D, and predicted output Y. Among them, input data X is the cause of predicted output Y, and prior knowledge D is the cause of both input data X and predicted output Y. Prior knowledge D is a confounding variable, which leads to context bias.

[0044] (2) Define D1, ..., D2 as the prior variables in prior knowledge D. d There is a causal relationship between them, and this causal relationship is expressed using a learnable directed acyclic graph (DAG). The DAG model can be obtained by solving the following equation:

[0045] D i =h i (D)+U i D j =h j (D)+U j

[0046] Among them, D i D j Let U be the i-th and j-th prior variables in prior knowledge D. i U j For a tolerable error that follows a Gaussian distribution, h i and hj It is a function that characterizes the causal structure.

[0047] (3) Define a causal space, and encode the input data X into a latent variable Z corresponding to the prior variable D in the causal space. This space supports the structure of the directed acyclic graph h when the latent variables corresponding to multiple images are added together. i with h j Unchanged. The causal space is specifically described as follows: Suppose there are n latent variables Z∈R located in the causal space. n×d They belong to the same directed acyclic graph structure h, if a weight w∈R n×1 If |w|=1, then its weighted sum It follows a Gaussian distribution and satisfies:

[0048]

[0049] in Represents Gaussian distribution

[0050] (4) Define the intervention formula p(y|do(x)). Based on the directed acyclic graph, decompose the joint probability distribution of prior knowledge into the product of the conditional probability distributions of each causally related latent variable, i.e. Where do(·) is the intervention symbol, Pa(D) i ) is the prior variable D i The set of factors. Further, the intervention formula is specifically described as follows: For the input image x, the latent variable z ~ p(z|x) is obtained through an encoder and sampling, then input into a directed acyclic graph function h, and constrained by p(z|h(z)). Finally, the predicted y after intervention is obtained through a classifier:

[0051] p(y|do(x))=E p(z|x) E p(z|h(z)) p(y|z)

[0052] (5) During meta-training, all support set input images in the current task are encoded to obtain the corresponding latent variables. A Gaussian mixture model located in the causal space is constructed. The likelihood probability of the latent variables is maximized in an unsupervised manner. Then, the latent variables are decoded to obtain the corresponding generated images. The similarity between the generated images and the input images is maximized to optimize the model.

[0053] The specific steps are as follows:

[0054] (5.1) From unsupervised datasets Constructing a meta-training dataset

[0055] (5.2) Regarding the task Input in The distribution of the corresponding encoding z is obtained using a deep neural convolutional network encoder F(), i.e.:

[0056]

[0057] (5.3) Applying the encoding z to the causal space yields the distribution of the latent variable e, i.e.:

[0058]

[0059] (5.4) The unsupervised Gaussian mixture model in the causal space is defined as follows:

[0060]

[0061] Where π is the K-dimensional weight, μ ,K] , Let be the mean and diagonal covariance of the k-th mixed mode, scalar It is a scaling parameter.

[0062] (5.5) Maximizing the likelihood probability of the latent variables and maximizing the similarity between the generated image and the input image can maximize the following lower bound of evidence:

[0063]

[0064] in, Let e ​​be the variational posterior distribution, and e be the encoding sampled in the causal latent space. q φ (e|z,x) and It is a posteriori network q φ Factorize (e,z|x), where p(e|z) is the prior distribution of the causal structure in the causal space. A task-specific prior distribution located in the causal space. For have:

[0065]

[0066] in, The closed-set solution method is to iteratively use an unsupervised expectation-maximization step;

[0067] (6) During meta-testing, all support set input images and query set input images in the current task are encoded to obtain the corresponding latent variables. A Gaussian mixture model located in the causal space is constructed. Using a semi-supervised approach, the likelihood probability of the latent variables is maximized to obtain the class probability of the query set input images. Specifically:

[0068] (6.1) Construct a meta-test task from the labeled dataset:

[0069]

[0070] in To support the set, For query set.

[0071] (6.2) Similar to steps (5.2)-(5.5), maximizing the likelihood probability of the latent variables and maximizing the similarity between the generated image and the input image can maximize the following lower bounds of evidence. The difference is that... The solution method for closed sets is to iteratively use a semi-supervised expectation-maximization step.

[0072] (6.3) Calculate the query set The class probability of the input image is calculated using Bayes' theorem:

[0073]

[0074] in, This represents the probability that the predicted category y is category c, given the latent variable z. The distribution of latent variables in the Gaussian mixture model for the current task. This represents the distribution of class modes in a Gaussian mixture model. Let represent the distribution of latent variables in the c-class mode of the Gaussian mixture model.

[0075] like Figure 2 As shown, Figure 2 Figure (a) shows the training and testing structure of the unsupervised meta-learning in an embodiment of the present invention. The left area represents the image encoding, Gaussian mixture model and causal intervention process, and the right area represents the decoding and causal intervention process. Figure (b) shows the testing stage of the supervised meta-learning, which includes image encoding, semi-supervised Gaussian mixture model and causal intervention process.

[0076] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An unsupervised meta-learning method based on visual causal intervention, characterized in that, Specifically, the steps include the following: (1) Construct a causal structure model graph for unsupervised meta-learning to obtain the causal relationship between input data, prior knowledge and predicted output; (2) Define a one-to-one correspondence between prior variables and prior knowledge, define a causal relationship between prior variables, and express this causal relationship using a learnable directed acyclic graph; (3) Define a causal space, encode the input image into a latent variable corresponding to the prior variable in the causal space, and the structure of the directed acyclic graph of the causal space remains unchanged when the latent variables corresponding to multiple images are added together. (4) Define the intervention formula and decompose the joint probability distribution of prior knowledge into the product of the conditional probability distributions of each causally related latent variable based on the directed acyclic graph. (5) Encode all the support set input images in the current task to obtain the corresponding latent variables, construct an unsupervised Gaussian mixture model in the causal space, maximize the likelihood probability of the latent variables in an unsupervised manner, decode the latent variables to obtain the corresponding generated images, and finally maximize the similarity between the generated images and the input images to optimize the model. (6) Encode all support set input images and query set input images in the current task to obtain the corresponding latent variables, construct a Gaussian mixture model in the causal space, and maximize the likelihood probability of the latent variables using a semi-supervised approach to obtain the class probability of the query set input images.

2. The unsupervised meta-learning method based on visual causal intervention according to claim 1, characterized in that, The specific step (1) is as follows: In the causal structure model diagram of unsupervised meta-learning, the input data is the cause of the predicted output, and the prior knowledge is the cause of both the input data and the predicted output; and the prior knowledge is a confounding variable, which will lead to context bias.

3. The unsupervised meta-learning method based on visual causal intervention according to claim 1, characterized in that, The directed acyclic graph model in step (2) is obtained by solving the equation: ； in, , For prior knowledge The first in , A prior variable, , Tolerable errors that follow a Gaussian distribution, and It is a function that characterizes the causal structure.

4. The unsupervised meta-learning method based on visual causal intervention according to claim 1, characterized in that, The causal space in step (3) specifically refers to the assumption that there is... A latent variable located in the causal space And they belong to the same directed acyclic graph structure. If a weight satisfy Then its weighted sum If it follows a Gaussian distribution, then its expression is: ;in This indicates a Gaussian distribution.

5. The unsupervised meta-learning method based on visual causal intervention according to claim 1, characterized in that, The intervention formula in step (4) is specifically as follows: for the input image Latent variables are obtained through encoder and sampling. Then the input is fed into the directed acyclic graph function. and its constraints Finally, the prediction after intervention is obtained through a classifier. : ;in This is an intervention symbol.

6. The unsupervised meta-learning method based on visual causal intervention according to claim 1, characterized in that, The unsupervised Gaussian mixture model in the causal space in step (5) is specifically as follows: ;in, yes Dimension weights , For the first The mean and diagonal covariance of each mixed mode, scalar It is a scaling parameter.

7. The unsupervised meta-learning method based on visual causal intervention according to claim 1, characterized in that, In steps (5) and (6), maximizing the likelihood probability of the latent variables can be obtained by solving the following optimization equation: ;in, The closed-set solution method is to iteratively use an unsupervised expectation-maximization step.

8. The unsupervised meta-learning method based on visual causal intervention according to claim 1, characterized in that, In step (5), maximizing the similarity between the generated image and the input image can maximize the following lower bound of evidence: ;in, For variational posterior distribution, Encoding for sampling in the causal latent space , and It is a posteriori network Factorize the expression. For the prior distribution of causal structure in causal space. A priori distribution in causal space specific to the task.

9. The unsupervised meta-learning method based on visual causal intervention according to claim 1, characterized in that, In step (6), the class probabilities of the input image in the query set are calculated using Bayes' rule: ;in, This indicates that the hidden variable is In the case of predicting the category for The probability of the category, The distribution of latent variables in the Gaussian mixture model for the current task. This represents the distribution of class modes in a Gaussian mixture model. In Gaussian mixture models Distribution of latent variables under categorical modalities.

Images