Causal decoupling method and device based on multi-scale noise and adversarial supervision

Abstract

The invention discloses a causal decoupling method and device based on multi-scale noise and adversarial supervision, and the method comprises the steps: carrying out the simulation of the causal relationship of variables in a causal graph, so as to generate observation image data, and constructing a training set and a test set according to the observation image data, the corresponding causal label information and the causal graph; constructing a causal decoupling model, and performing adversarial supervision training under multi-scale noise by using the training set; and obtaining anti-fact intervention data by using the test set and the trained causal graph matrix and utilizing the trained observation data coding module and observation data decoding module. According to the method, an auto-encoder and causal acyclic constraints are fully combined, the discrimination module is trained under multi-scale noise, and high-quality adversarial supervision is performed, so that the model representation learning ability is improved, the representation understanding of the model on data with causal relationships is enhanced, the accuracy of implicit causal network prediction is improved, and the prediction efficiency is improved. And the causal decoupling accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of image representation learning and causal decoupling technology, specifically to a causal decoupling method and apparatus based on multi-scale noise and adversarial supervision. Background Technology

[0002] Image representation learning is an important field in computer science, artificial intelligence, and computer vision. Its core objective is to use deep learning techniques to extract high-dimensional information from images, learn meaningful semantic features, and thus enable machines to achieve an abstract understanding of the real world. Image representation learning research is broad, covering various tasks, including but not limited to self-supervised learning, image generation, transfer learning, and cross-modal learning. Research on image representation learning allows models to learn the features and even the generation mechanisms behind the data, enabling them to adapt to new tasks with few or no samples. Through learning image representations, the specific working mechanisms and decision-making processes within black-box models can also be revealed, enhancing the interpretability of the models.

[0003] Causal disentanglement is a key direction in machine learning. Its core objective is to isolate independent causal variables from complex data, learn representations with explicit causal semantics, and thus understand the underlying mechanisms of the data. This enables models to maintain robustness and generalization ability in dynamic environments, with changing distributions, or in scenarios with few samples. Causal disentanglement increases the reliability of the model by identifying and separating latent causal variables in the data, allowing representations to independently encode these factors. By modeling causal relationships, models can generate samples that better conform to real-world laws and logical constraints, laying a solid foundation for the application of artificial intelligence in social, economic, and scientific fields. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing a causal decoupling method and apparatus based on multi-scale noise and adversarial supervision.

[0005] The objective of this invention is achieved through the following technical solution: The first aspect of this invention provides a causal decoupling method based on multi-scale noise and adversarial supervision, comprising the following steps: (1) The causal relationship of variables in the causal graph is simulated by using the prior causal mechanism to generate observation image data, and training set and test set are constructed based on the observation image data, their corresponding causal label information and causal graph; (2) Construct a causal decoupling model and use the training set to perform adversarial supervised training under multi-scale noise to obtain a well-trained causal decoupling model; wherein, the causal decoupling model includes an observation data encoding module, an observation data decoding module and a multi-scale noise discrimination module; (3) Input the observed image data, causal label information and the trained causal graph matrix from the test set into the trained observation data encoding module to obtain the causal latent vector features, select the causal variables that need to be intervened, calculate their intervention values, modify the corresponding causal variables in the causal latent vector features, obtain the causal latent vector features after intervention, and input them into the trained observation data decoding module to obtain counterfactual intervention data.

[0006] Furthermore, the observed image data is obtained specifically through the following method: A directed acyclic graph is constructed based on multiple potential causal variables to serve as the real causal graph. External noise and variable values ​​are input into the causal graph to construct causal label information under the simulation environment, and the observed values ​​of the variables in the causal graph are generated by simulation using a causal function model. Based on the causal graph, each node is sampled according to the relationship on the causal graph to obtain the causal graph observation data. The image data generated based on the causal graph observation data is the simulation-generated observation image data.

[0007] Furthermore, the observation data encoding module includes an encoding module convolutional layer, an encoding module causal embedding layer, and an encoding module data compression layer; the observation data decoding module includes a decoding module data decoding layer, a decoding module causal embedding layer, and a decoding module deconvolutional layer; the multi-scale noise discrimination module includes an adaptive multi-scale noise addition layer, a discrimination module convolutional layer, a discrimination module causal embedding layer, a discrimination module data compression layer, and a classification layer; The observed image data, causal label information, and causal graph matrix are input into the causal decoupling model. The observed image data is passed through the convolutional layer of the encoding module to obtain a convolutional representation. The convolutional representation, causal label information, and causal graph matrix are input together into the causal embedding layer of the encoding module for calculation. After passing through the data compression layer of the encoding module, the causal latent vector features are obtained. The causal latent vector features are first passed through the data decoding layer of the decoding module to obtain a decoded representation. The decoded representation, causal label information, and causal graph matrix are input together into the causal embedding layer of the decoding module for calculation. After passing through the deconvolutional layer of the decoding module, the reconstructed data is obtained. The reconstructed data and the observed image data are respectively fed into the adaptive multi-scale noise-adding layer to obtain the noisy reconstructed data and observed image data. The noisy reconstructed data and observed image data are simultaneously passed through the convolutional layer of the discriminator module to obtain convolutional features. The convolutional features, causal label information, and causal graph matrix are input together into the causal embedding layer of the discriminator module for calculation. After passing through the data compression layer and classification layer of the discriminator module, the true / false discrimination probability is output.

[0008] Furthermore, the adaptive multi-scale noise layer specifically includes: initializing the number of noise addition steps and generating a temporal embedding, and then using a forward noise addition method of a diffusion model to add noise to the reconstructed data and the observed image data to obtain the noise-added reconstructed data and the observed image data.

[0009] Furthermore, the initialization of the noise-adding steps and the generation of the temporal embedding specifically includes: Initialize maximum noise scale Given a time-scale embedding dimension d, the time scale is encoded using triangular embedding. The time-scale embedding basis is calculated using a frequency function. The noisy step sequence T is then processed using the time-scale embedding basis and subjected to sine and cosine transforms respectively. The embeddings after sine and cosine transforms are then concatenated to obtain a time-scale embedding vector of length d.

[0010] Furthermore, the method of adding noise to the reconstructed data and observed image data using a diffusion model specifically includes: Based on normal distribution noise and the number of noise addition steps, noise is added to the reconstructed data and observed image data by accumulating noise factor to obtain the noise-added reconstructed data and observed image data; wherein, the accumulating noise factor is obtained according to the noise variance and time scale embedding vector at the corresponding time scale.

[0011] Furthermore, the step of using the training set to perform adversarial supervised training under multi-scale noise to obtain a trained causal decoupling model specifically includes: The observed image data, causal label information, and causal graph matrix in the training set are input into the causal decoupling model to obtain causal latent vector features, reconstructed data, and true / false discrimination probabilities. The autoencoder loss is calculated based on observed image data, causal graph matrix, causal latent vector features, reconstructed data, and true / false discrimination probability. The discriminative adversarial loss is calculated based on the true / false discrimination probability corresponding to the observed image data and reconstructed data. During training, the autoencoder loss and discriminative adversarial loss are optimized alternately. When optimizing the autoencoder loss, only the parameters of the observation data encoding module and the observation data decoding module are updated; when optimizing the discriminative adversarial loss, only the parameters of the observation data decoding module and the multi-scale noise discrimination module are updated; until the preset number of training rounds is reached or the autoencoder loss and discriminative adversarial loss converge, the trained causal decoupling model is obtained.

[0012] Furthermore, the autoencoder loss is specifically obtained through the following method: The root mean square error between the observed image data and the reconstructed data is calculated as the reconstruction loss; the KL divergence loss is calculated based on the causal latent vector features and their prior distribution; the generation loss is calculated based on the true / false discrimination probability; the total variational loss and entropy loss are calculated based on the reconstructed data; the adversarial generation loss is calculated based on the generation loss, total variational loss, and entropy loss; the directed acyclic loss is calculated based on the causal graph matrix; and the autoencoder loss is calculated based on the reconstruction loss, KL divergence loss, adversarial generation loss, and directed acyclic loss.

[0013] Furthermore, step (3) specifically includes: The observed image data, causal label information, and trained causal graph matrix in the test set are fed into the trained observation data encoding module. The observed image data is passed through the convolutional layer of the encoding module to obtain convolutional representation. The convolutional layer representation, causal label information, and the trained final causal graph matrix are fed into the causal embedding layer of the encoding module for calculation. After passing through the data compression layer of the encoding module, the causal latent vector features are obtained. Select the causal variables that need intervention, calculate their intervention values, and modify the corresponding causal variables in the causal latent vector features according to the intervention values ​​to obtain the causal latent vector features after intervention. The causal latent vector features after intervention, causal label information, and the trained causal graph matrix are fed into the trained observation data decoding module. The causal latent vector features after intervention first pass through the data decoding layer of the decoding module to obtain the decoded representation. The decoded representation, causal label information, and the trained causal graph matrix are then input into the causal embedding layer of the decoding module for calculation. After passing through the deconvolution layer of the decoding module, the counterfactual intervention data is obtained.

[0014] A second aspect of this invention provides a causal decoupling device based on multi-scale noise and adversarial supervision, used to implement the above-described causal decoupling method based on multi-scale noise and adversarial supervision, the device comprising: The causal data synthesis module is used to simulate the causal relationship of variables in the causal graph using a priori causal mechanism, generate observed image data, and construct training and test sets based on the observed image data, their corresponding causal label information, and the causal graph. The model building and training module is used to build a causal decoupling model and perform adversarial supervised training under multi-scale noise using a training set to obtain a trained causal decoupling model; wherein, the causal decoupling model includes an observation data encoding module, an observation data decoding module, and a multi-scale noise discrimination module; The causal counterfactual intervention module is used to input the observed image data, causal label information, and trained causal graph matrix from the test set into the trained observation data encoding module to obtain causal latent vector features, select the causal variables that need intervention, calculate their intervention values, modify the corresponding causal variables in the causal latent vector features, obtain the intervened causal latent vector features, and input them into the trained observation data decoding module to obtain counterfactual intervention data.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) This invention is a causal decoupling method based on multi-scale noise and adversarial supervision, which can improve the accuracy of causal decoupling by performing adversarial training through the discrimination module under multi-scale noise.

[0016] (2) The observation data encoding module and the observation data decoding module involved in this invention are to explicitly model the causal relationship through a directed acyclic graph, and then embed the causal relationship into the model for training through a neural network. The main loss of the model and the directed acyclic loss are optimized at the same time.

[0017] (3) The multi-scale noise discrimination module involved in this invention is trained under multi-scale noise after adaptive noise addition. The causal embedding information output from the autoencoder is input into the multi-scale noise discrimination module. The observation data encoding module, the observation data decoding module and the multi-scale noise discrimination module are alternately modeled and trained.

[0018] (4) This invention uses noisy data at multiple scales to train a strong discriminative module, and trains the observation data encoding module and the observation data decoding module through adversarial supervision. During the training process, causal embedding and directed acyclic constraints are combined to enable the model to learn beneficial causal representations. This invention uses a multi-objective balanced loss function to guide the training of the model, and then uses an adversarial supervision method with adaptive noise scale to learn causal representations, which solves the problems of low accuracy of causal decoupling and insufficient model generalization ability in the prior art. Attached Figure Description

[0019] Figure 1 This is a flowchart of the causal decoupling method based on multi-scale noise and adversarial supervision of the present invention; Figure 2 This is a flowchart of the causal data synthesis module of the present invention; Figure 3 This is a flowchart of the causal decoupling model of the present invention; Figure 4 This is a flowchart of the observation data encoding module of the present invention; Figure 5 This is a flowchart of the observation data decoding module of the present invention; Figure 6 This is a flowchart of the multi-scale noise discrimination module of the present invention; Figure 7 This is a schematic diagram of the framework of multi-scale noise and adversarial supervision in this invention; Figure 8 This is a flowchart of the causal counterfactual intervention module of the present invention; Figure 9 This is a structural diagram of the causal decoupling device based on multi-scale noise and adversarial supervision of the present invention. Detailed Implementation

[0020] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. It is obvious that the drawings used in the following description are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0021] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0022] It should be understood that although the terms first, second, third, etc., may be used in this invention to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of this invention, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0023] The present invention will now be described in detail with reference to the accompanying drawings. Unless otherwise specified, the features of the following embodiments and implementations can be combined with each other.

[0024] See Figure 1 The causal decoupling method based on multi-scale noise and adversarial supervision of the present invention specifically includes the following steps: (1) The causal relationship of variables in the causal graph is simulated by using the prior causal mechanism to generate observation image data containing causal relationships. Training and test sets are constructed based on the observation image data, their corresponding causal label information and causal graph for subsequent training and testing of the causal decoupling model.

[0025] In this embodiment, as Figure 2 As shown, the observed image data is obtained through the following method: By making physical and logical assumptions about the simulation environment, variables with causal relationships are represented by a causal graph. Since the causal graph is used to describe the causal relationships between different variable characteristics, it must satisfy the directed acyclic property (DAG). Therefore, a DAG is first constructed based on multiple potential causal variables as the real causal graph. This causal graph is stored in the form of an adjacency matrix, that is, the causal graph is mapped to a causal graph matrix through causal vector mapping. Subsequently, the causal graph matrix is ​​directly used to represent the causal graph. Then, external noise and variable values ​​are input into the causal graph to construct causal label information in the simulation environment. Specifically, the variables in the causal graph are topologically sorted, and variable values ​​are given according to the order of the topological sort, which are the causal variables. Assign independent and identically distributed external noise Where Z represents the set of causal variables, Representing the causal variable i, Represents the set of external noise. The external noise assigned to causal variable i is represented by a Gaussian distribution with homoscedasticity, used to describe the disturbance factors in the system. The observed values ​​of the variables in the causal graph are generated using a causal function model. Finally, the causal graph observation data obtained after a single input (i.e., the external noise and variable values ​​input into the causal graph) is considered as a single sampling of the causal mechanism. The set constructed from several such samples is the causal graph observation data. The set of image data generated based on the causal graph observation data is the causal observation data, which is used as the simulated observation image data. Specifically, in the causal graph, variables that have no direct reference to them are root variables, and variables that have one or more references are non-root variables. Based on the causal graph, sampling is performed on each node according to the relationships on the causal graph to obtain the causal graph observation data. The image data generated based on the causal graph observation data is the simulated observation image data. For root variables, their observed values ​​are directly determined by the external noise; for non-root variables, their observed values ​​are determined by the external noise, the causal function, and the parent node's observed values.

[0026] It should be understood that observed image data can be generated not only through simulation but also directly from real-world scenarios, used to represent the behavior of causal systems under synthetic or real-world conditions. To obtain biased estimates of the observed image data, causal labeling information is also input into the causal decoupling model for training, without interfering with the data.

[0027] It should be noted that causal label information reflects the supervisory signal of the semantics of potential causal variables and is used to construct weak prior information to learn the causal network. Specifically, the causal label information here refers to the causal graph observation data in the generation process of each observed image data sample. The causal label information needs to be consistent with the observed image data; that is, the causal label information and the observed image data need to be concatenated into a vector and then input into the causal decoupling model. Although both the causal label information and the observed image data need to be input into the causal decoupling model, their input positions are different. The causal label information, as the quantification information of causal variables, is input into the causal embedding layer; while the observed image data is directly input into the convolutional layer and then into the causal embedding layer.

[0028] Furthermore, the causal function model of the nonlinear structure can be expressed as:

[0029] In the formula, This represents the observed value of causal variable i. This represents the observed value of the parent node of the causal variable i in the causal graph. This represents the external noise assigned to the causal variable i; This represents the nonlinear transformation corresponding to the causal variable i, used for... Directly apply nonlinear transformation; This represents the nonlinear transformation of the causal variable i and its corresponding external noise, used to analyze... and Nonlinear transformations are applied to simulate complex nonlinear causal relationships.

[0030] (2) Construct a causal decoupling model and perform adversarial supervised training using the training set under multi-scale noise to obtain a well-trained causal decoupling model. The causal decoupling model includes an observation data encoding module, an observation data decoding module, and a multi-scale noise discrimination module, such as... Figure 3 As shown.

[0031] In this embodiment, the observation data encoding module includes an encoding module convolutional layer, an encoding module causal embedding layer, and an encoding module data compression layer, such as... Figure 4 As shown; the observation data decoding module includes a data decoding layer, a causal embedding layer, and a deconvolution layer, as shown. Figure 5 As shown; the multi-scale noise discrimination module includes an adaptive multi-scale noise addition layer, a discrimination module convolutional layer, a discrimination module causal embedding layer, a discrimination module data compression layer, and a classification layer, as follows. Figure 6 As shown.

[0032] Specifically, such as Figure 3 As shown, observed image data, causal label information, and causal graph matrix are input into the causal decoupling model. The observed image data is passed through the convolutional layer of the encoding module to obtain a convolutional representation. The convolutional representation, causal label information, and causal graph matrix are input together into the causal embedding layer of the encoding module for calculation. After passing through the data compression layer of the encoding module, causal latent vector features are obtained. The causal latent vector features are first passed through the data decoding layer of the decoding module to obtain a decoded representation. The decoded representation, causal label information, and causal graph matrix are input together into the causal embedding layer of the decoding module for calculation. After passing through the deconvolutional layer of the decoding module, reconstructed data is obtained. The reconstructed data and observed image data are respectively fed into an adaptive multi-scale noise-adding layer to obtain noisy reconstructed data and observed image data. The noisy reconstructed data and observed image data are simultaneously passed through the convolutional layer of the discriminator module to obtain convolutional features. The convolutional features, causal label information, and causal graph matrix are input together into the causal embedding layer of the discriminator module for calculation. After passing through the data compression layer and classification layer of the discriminator module, the true / false discrimination probability is output, that is, the probability that the reconstructed data is the real observed image data. This true / false discrimination probability can be used for subsequent adversarial supervised training under multi-scale noise.

[0033] The specific workflow of the observation data encoding module is as follows: Figure 4 As shown, the observation data encoding module is used to perform distribution modeling on the observed image data, extract and compress data features, and output a high-dimensional latent vector representation containing causal information embedding, which is the causal latent vector feature. Therefore, the observation data encoding module can be used to extract the causal latent vector features of the observed image data. The observed image data is input into the observation data encoding module. Simultaneously, the causal graph (i.e., the causal graph matrix) and causal label information are put into the observation data encoding module through the causal embedding layer. After passing through the data compression layer of the encoding module, the causal latent vector features are finally generated, which can then be passed to the observation data decoding module.

[0034] The specific workflow of the observation data decoding module is as follows: Figure 5As shown, the observation data decoding module decodes the data distribution and generates reconstructed data. Its input is a high-dimensional causal latent vector representation containing causal characteristics, i.e., causal latent vector features. Simultaneously, the causal embedding layer of the decoding module feeds the optimized causal graph (i.e., the causal graph matrix) and causal label information into the observation data decoding module. Then, the deconvolution layer of the decoding module restores the data from the high-dimensional causal latent vector representation, generating reconstructed observation image data, i.e., reconstructed data. Therefore, the observation data decoding module, based on the causal graph matrix and causal label information, can decode the causal latent vector features to obtain reconstructed data, which is used to calculate loss and optimize the causal decoupling model.

[0035] The specific workflow of the multi-scale noise discrimination module is as follows: Figure 6 As shown, reconstructed data and observed image data are input as discrimination data into the multi-scale noise discrimination module. The module uses an adaptive multi-scale noise-adding layer to apply adaptive multi-scale noise to both the reconstructed and observed image data. In this layer, the number of noise-adding steps is initialized, and a temporal embedding is generated and input into the multi-scale noise discrimination module. Then, a forward noise-adding method using a diffusion model is used to add noise to both the real observed image data and the reconstructed data. The noise-adding reconstructed and observed image data are then passed through the convolutional layer of the discrimination module to obtain convolutional features. These features, along with causal label information and the causal graph matrix, are input into the causal embedding layer of the discrimination module for calculation. Finally, the data passes through the data compression layer and classification layer of the discrimination module, outputting the true / false discrimination probability. Therefore, the multi-scale noise discrimination module can be used to discriminate the reconstructed data, determining the probability that it is real observed image data, i.e., the true / false discrimination probability.

[0036] Furthermore, the adaptive multi-scale noise layer specifically includes: first, initializing the number of noise addition steps and generating temporal embeddings, and then using the forward noise addition method of the diffusion model to add noise to the reconstructed data and observed image data to obtain the noise-added reconstructed data and observed image data.

[0037] Furthermore, the number of noise-adding steps is initialized and the temporal embedding is generated, specifically including: initializing the maximum noise-adding scale. Given a timescale embedding dimension d, triangular embedding is used to encode the timescale. Specifically, the embedding of the noisy timescale sequence consists of a frequency function, angle calculation, and sine and cosine transforms. The frequency function and the corresponding timescale embedding basis are expressed as follows:

[0038]

[0039] In the formula, This represents the frequency of the k-th dimension in the time-scale embedding basis, where the value of k ranges from the first dimension to half of the total time-scale embedding dimension d. Indicates length is The time-scale embedding basis is calculated using the above formula to obtain a length of [missing information]. The temporal-scale embedding basis is then used to refine the noisy step sequence. After processing, sine and cosine transformations are performed respectively, where This indicates the first noise level. This indicates the second noise level. Indicates the third noise-adding scale, the maximum noise-adding scale. This is the upper limit of the noisy step sequence T. The embeddings after sine and cosine transformations are then concatenated to obtain a time-scale embedding vector of length d, expressed as:

[0040] In the formula, This represents the embedding vector at time scale t. This indicates a splicing operation. Represents the sine transform. This represents the cosine transform. Here, the embeddings after the sine and cosine transforms are concatenated, outputting a time-scale embedding vector of dimension d. Subsequently, based on the obtained time-scale embedding vector, a forward noise addition method using a diffusion model can be applied to add noise to the reconstructed data and the observed image data.

[0041] Furthermore, a forward noise addition method using a diffusion model is employed to add noise to both the reconstructed data and the observed image data. Specifically, this includes generating normally distributed noise. The variance sequence is initialized through linear scheduling. variance sequence It is mainly used to control the intensity of noise at different scales, and its construction process is represented as follows:

[0042] In the formula, This represents the noise variance at the t-th scale. This represents the maximum length of the sequence T with added noise. and Representing variance sequences respectively The noise variance at the first and last scales. The actual added noise is determined by a set of cumulative noise factors. For noise control, the noise needs to pass through a variance control unit when it is introduced. The formula for calculating the cumulative noise factor is:

[0043] In the formula, This represents the cumulative noise factor at the t-th scale. This represents a noise variance adjustment network, whose inputs are the original noise variance and the time embedding corresponding to the noise step at this point. This represents the embedding vector for time scale s. Let represent the noise variance at the s-th scale, where s represents the smallest noisy unit at a specific noisy scale, and its value ranges from 1 to 2. , representing the gradual noise addition process from the original data (i.e., the original image observation data or reconstructed data) to the t-th scale, with the initial number of noise addition steps being . By accumulating noise factors, noise can be quickly added to reconstructed data and observed image data. The noise addition formulas are as follows:

[0044]

[0045] In the formula, This represents the reconstructed data after adding noise. This represents the observed image data after noise has been added. X represents the reconstructed data, and X represents the observed image data. Indicates noise It follows a matrix with a mean of 0 and a covariance matrix that is the identity matrix. The normal distribution .

[0046] It should be understood that the number of noise-adding steps determines the number of temporal embeddings. For example, if the initial number of noise-adding steps is 500, then there will be 500 temporal embedding vectors, each with a length of d. That is, there will be a 500*d temporal embedding set, which is then used to calculate the cumulative noise factor to add noise to the reconstructed data and observed image data.

[0047] In this embodiment, the training set is used for adversarial supervised training under multi-scale noise to obtain a well-trained causal decoupling model, such as... Figure 3 and Figure 7 As shown, specifically, it includes: First, the observed image data in the training set... Causal label information The causal graph matrix A is input into the causal decoupling model to obtain the causal latent vector feature b and reconstruct the data. The model incorporates true / false discrimination probabilities, where the causal graph matrix A is initialized as an upper triangular matrix when input into the causal decoupling model. Then, the autoencoder loss is calculated based on the observed image data, the causal graph matrix, causal latent vector features, reconstructed data, and true / false discrimination probabilities. The adversarial loss is calculated based on the true / false discrimination probabilities corresponding to the observed image data and reconstructed data. Subsequently, the autoencoder loss and adversarial loss are backpropagated to obtain gradients. The model is then optimized using AdamW, with the autoencoder loss and adversarial loss being optimized alternately during training. When optimizing the autoencoder loss, only the parameters of the observation data encoding and decoding modules are updated; when optimizing the adversarial loss, only the parameters of the observation data decoding and multi-scale noise discrimination modules are updated. This process continues until a preset number of training epochs are reached or the autoencoder loss and adversarial loss converge. The trained observation data encoding, decoding, and multi-scale noise discrimination modules are then obtained, resulting in the trained causal decoupling model.

[0048] Furthermore, the autoencoder loss is calculated based on the observed image data, the causal graph matrix, the causal latent vector features, the reconstructed data, and the true / false discrimination probability. Specifically, this includes: calculating the root mean square error between the observed image data and the reconstructed data as the reconstruction loss; calculating the KL divergence loss based on the causal latent vector features and their prior distribution; calculating the generation loss based on the true / false discrimination probability; calculating the total variational loss and entropy loss based on the reconstructed data; calculating the adversarial generation loss based on the generation loss, total variational loss, and entropy loss; calculating the directed acyclic loss based on the causal graph matrix; and calculating the autoencoder loss based on the reconstruction loss, KL divergence loss, adversarial generation loss, and directed acyclic loss. The calculation formula is as follows:

[0049] In the formula, This represents the loss of an autoencoder with anti-supervision capabilities; This represents the root mean square error, and also the reconstruction loss; Represents observed image data, This represents the observation data of variable i in the observed image data, and n represents the total number of variables; This represents the reconstructed data corresponding to the observed image data X. This represents the reconstructed data of variable i in the reconstructed data; This represents the KL divergence loss; Indicates parameters The distribution of causal latent vector features b in the causal decoupling model under the input condition of observed image data X; This represents the prior distribution of the causal latent vector feature b; Indicates adversarial generation loss; Let be the directed acyclic constraint on the causal graph matrix A, and let represent the directed acyclic loss of the causal graph matrix A. , and They are in balance , and Hyperparameters.

[0050] The formula for calculating the adversarial generation loss is as follows:

[0051] In the formula, Indicates the generation loss. This represents the total variational loss. Represents entropy loss, , and They are in balance , and The hyperparameters are: The generation loss, total variational loss, and entropy loss are calculated using the following formulas:

[0052]

[0053]

[0054] In the formula, This represents the reconstructed data after adding noise when the noise scale is t. This indicates the multi-scale noise discrimination module's handling of the noisy reconstructed data. The probability of being judged as true is obtained based on the true / false judgment probability output by the multi-scale noise discrimination module; This indicates the reconstructed data after adding noise. Calculate the expectation; Indicates parameters The causal decoupling model reconstructs data after adding noise under the input condition of causal latent vector feature b. Distribution; Represents position coordinates The expected value is calculated for the data block at that location; Represents position coordinates Reconstructed data blocks at the location; This represents the difference in height between adjacent reconstructed data. This represents the difference in width between adjacent reconstructed data. Noise is represented by V, C, and K, which represent the batch size, number of channels, and number of buckets in the corresponding soft histogram, respectively. Indicates the first v batch, numberc The data for the [number] channel is in the soft histogram. k The probability of each bucket is calculated using the following formula:

[0055] In the formula, These are the normalized values ​​of the data calculated based on V and C. Indicates the reconstruction data in the first... v batch, number c The value at the g-th position of each channel, where G represents the number of sampling points used to calculate the soft histogram distribution. The soft histogram is represented by the first... k Average of each bucket This represents the standard deviation of the soft histogram, used to measure the smoothness of the histogram.

[0056] Directed acyclic loss of causal graph matrix A The calculation formula is:

[0057] In the formula, This represents the trace taken after calculating the matrix exponent of the Hadamard product of the causal graph matrix A, where n is the dimension of the causal graph; For the directed acyclic loss of the causal graph matrix A, when When the value approaches 0, it indicates that the causal graph is a directed acyclic graph.

[0058] Furthermore, the formula for calculating the adversarial loss is as follows:

[0059] In the formula, This represents the discriminative adversarial loss under multi-scale noise. This represents the reconstructed data after adding noise, where the noise scale is t. This represents the noise-added observed image data at a noise scale of t; This indicates the multi-scale noise discrimination module's handling of the noisy reconstructed data. The probability of being judged as true is obtained based on the true / false judgment probability output by the multi-scale noise discrimination module; This indicates the multi-scale noise discrimination module's handling of the noisy observed image data. The probability of being judged as true is obtained based on the true / false judgment probability output by the multi-scale noise discrimination module; This represents the observed image data after adding noise. Calculate the expectation; Represents the observed image data after adding noise. The true distribution.

[0060] Furthermore, in the adaptive multi-scale noise layer, the number of noise addition steps (i.e., the noise addition scale) is adaptively generated, specifically including: setting a noise scale update threshold. thr Obtain the maximum noise scale In optimizing loss , Reconstructed data output by the observation data decoding module in real time ,Will Input the multi-scale noise discrimination module to obtain the true / false discrimination probability. ,like Greater than thr This indicates that the generation quality of the observation data decoding module exceeds the threshold range, requiring an increase in the training difficulty of the multi-scale noise discrimination module, and a reduction in the noise scale. t Increase C Step. A judgment is made in each training session, until... t Increase to T max End of increment.

[0061] It should be understood that by training the causal decoupling model under adversarial supervision under multi-scale noise, a strong discriminative module with discriminative ability under high perturbation conditions can be trained. Combined with causal constraints, the observation data encoding module, observation data decoding module, and multi-scale noise discriminative module are trained under adversarial supervision under multi-scale noise to learn causal representations and obtain the final robust trained causal decoupling model.

[0062] In this embodiment, the architecture of the causal decoupling model based on multi-scale noise-cosm adversarial supervision is as follows: Figure 3 As shown, the observed image data, through the observation data encoding module, combines causal labels and causal graph features to obtain a high-dimensional latent vector that fuses causal representations, namely the causal latent vector features. This latent vector feature is then used by the observation data decoding module to obtain the corresponding reconstructed data. For multi-scale noise, the observation data encoding module, observation data decoding module, and multi-scale noise discrimination module underwent adversarial supervised training. To enhance the discrimination capability of the multi-scale noise discrimination module, adaptive multi-scale noise fusion was used to add noise to both the observed image data and the reconstructed data, following a forward noise addition formula. Through adversarial supervised training, the multi-scale noise discrimination module can discriminate the quality of the reconstructed data, enabling the observation data decoding module to generate data that conforms to the true distribution, thus enhancing the decoupling capability of the causal decoupling model for causal latent vector features.

[0063] (3) Input the observed image data, causal label information and the trained causal graph matrix from the test set into the trained observation data encoding module to obtain the causal latent vector features, select the causal variables that need to be intervened, calculate their intervention values, modify the corresponding causal variables in the causal latent vector features, obtain the causal latent vector features after intervention, and input them into the trained observation data decoding module to obtain counterfactual intervention data.

[0064] In this embodiment, based on the trained causal decoupling model, a trained observation data encoding module, a trained observation data decoding module, and a trained causal graph matrix can be obtained. Since the causal graph matrix is ​​initialized and then fed into the causal decoupling model for training, the causal graph matrix is ​​also trained when the training ends. At this time, the trained causal graph matrix can be directly used for subsequent testing and online applications. In short, the causal graph matrix can be regarded as a trainable parameter in the causal decoupling model, which can be used directly after training.

[0065] Specifically, such as Figure 8 As shown, the observed image data, causal label information, and the trained final causal graph matrix from the test set are fed into the trained observation data encoding module. The observed image data is passed through the convolutional layer of the encoding module to obtain a convolutional representation. The convolutional layer representation, causal label information, and the trained final causal graph matrix are then input into the causal embedding layer of the encoding module for calculation. After passing through the data compression layer of the encoding module, causal latent vector features are obtained. The causal variables requiring intervention are selected, and their intervention values ​​are calculated. Based on the intervention values, the corresponding causal variables in the causal latent vector features are modified to obtain the intervened causal latent vector features. The intervened causal latent vector features, causal label information, and the trained final causal graph matrix are fed into the trained observation data decoding module. The intervened causal latent vector features first pass through the data decoding layer of the decoding module to obtain a decoded representation. The decoded representation, causal label information, and the trained final causal graph matrix are then input into the causal embedding layer of the decoding module for calculation. After passing through the deconvolutional layer of the decoding module, counterfactual intervention data is obtained.

[0066] It should be noted that the trained observation data encoding module and the trained observation data decoding module can be used to output reconstructed data or to perform counterfactual intervention on existing data to generate counterfactual intervention data that conforms to causal relationships but does not exist in the test set.

[0067] Furthermore, counterfactual intervention data can verify whether the model truly learns the hidden mechanisms behind the data, test whether different latent variables control different semantics, assess the quality of causal representation learning, and test the model's ability to learn causal representation and causal decoupling. The proportion of data in the counterfactual intervention data that conforms to causal relationships is calculated to evaluate the model's causal decoupling ability. Specifically, the intervention classification accuracy can be used to evaluate the quality of generated counterfactual intervention data, and the model's representation learning ability can be evaluated by dividing the number of samples with correct semantic transformation in the causal counterfactual intervention data by the total number of samples.

[0068] In summary, the causal decoupling method based on multi-scale noise and adversarial supervision described in this invention aims to automatically learn variables with causal semantic features, establish constraints using causal embedding, and determine the causal mechanisms of these variables in the real physical world. It can provide broad support for downstream tasks, including transfer generalization, interpretable learning, multimodal learning and representation alignment, and consistent and controllable generation. Most current generation or representation models focus on fitting the surface distribution of observed data, ignoring the underlying generation mechanisms and facing bottlenecks in learning real physical mechanisms. While these methods have made significant progress in image and text generation tasks, they still suffer from insufficient generalization to out-of-distribution data. In complex situations, the models exhibit "illusions" and poor consistency modeling capabilities, making them unsuitable for scenarios with high accuracy requirements. This invention, through adversarial supervision training of the multi-scale noise discrimination module, observation data encoding module, and observation data decoding module, can separate background noise data unrelated to potential causal relationships through causal decoupling, enhancing the representation's ability to learn causal structures. Specifically, in complex scene object grasping and decision-making tasks for robots, the image data received by the visual sensor contains perturbations such as illumination changes, obstacle occlusion, and environmental background noise. Traditional representation learning methods rely on statistical correlation and are prone to overfitting to these perturbations, leading to task failure. The causal decoupling method based on multi-scale noise and adversarial supervision described in this invention, after learning a robust representation, outputs the representation to downstream tasks. This can separate factors directly related to the grasping decision task from irrelevant perturbations, accurately calculating the grasping position in the image even in scenes with illumination changes and partial occlusion. Furthermore, since the representation relies on a causal structure independent of background features, even when transferred to new task scenarios, it can reduce dependence on old task scenarios, improving the accuracy of downstream task execution and the robustness of the model.

[0069] It is worth mentioning that the embodiments of the present invention also provide a causal decoupling device based on multi-scale noise and adversarial supervision, used to implement the causal decoupling method based on multi-scale noise and adversarial supervision in the above embodiments. Figure 9As shown, the device specifically includes a causal data synthesis module, a model building and training module, and a causal counterfactual intervention module.

[0070] In this embodiment, the causal data synthesis module is used to simulate the causal relationships of variables in the causal graph using a priori causal mechanism, generating observed image data containing causal relationships, such as... Figure 2 As shown, training and test sets are constructed based on the observed image data, their corresponding causal label information, and the causal graph.

[0071] In this embodiment, the model building and training module is used to build a causal decoupling model and perform adversarial supervised training under multi-scale noise using a training set to obtain a trained causal decoupling model, such as... Figure 7 As shown.

[0072] In this embodiment, the causal counterfactual intervention module is used to input the observed image data, causal label information, and trained causal graph matrix from the test set into the trained observation data encoding module to obtain causal latent vector features. It then selects the causal variables to be intervened in, calculates their intervention values ​​to modify the corresponding causal variables in the causal latent vector features, obtains the intervened causal latent vector features, and inputs them into the trained observation data decoding module to obtain counterfactual intervention data, such as... Figure 8 As shown.

[0073] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A causal decoupling method based on multi-scale noise and adversarial supervision, characterized in that, Includes the following steps: (1) The causal relationship of variables in the causal graph is simulated by using the prior causal mechanism to generate observation image data, and training set and test set are constructed based on the observation image data, their corresponding causal label information and causal graph; (2) Construct a causal decoupling model and use the training set to perform adversarial supervised training under multi-scale noise to obtain a well-trained causal decoupling model; wherein, the causal decoupling model includes an observation data encoding module, an observation data decoding module and a multi-scale noise discrimination module; (3) Input the observed image data, causal label information and the trained causal graph matrix from the test set into the trained observation data encoding module to obtain the causal latent vector features, select the causal variables that need to be intervened, calculate their intervention values, modify the corresponding causal variables in the causal latent vector features, obtain the causal latent vector features after intervention, and input them into the trained observation data decoding module to obtain counterfactual intervention data.

2. The causal decoupling method based on multi-scale noise and adversarial supervision according to claim 1, characterized in that, The observed image data is obtained specifically through the following method: A directed acyclic graph is constructed based on multiple potential causal variables as the real causal graph; external noise and variable values ​​are input into the causal graph to construct causal label information in the simulation environment, and the observed values ​​of the variables in the causal graph are generated by simulation using a causal function model; Based on the causal graph, each node is sampled according to the relationships on the causal graph to obtain causal graph observation data. The image data generated based on the causal graph observation data is the simulation-generated observation image data.

3. The causal decoupling method based on multi-scale noise and adversarial supervision according to claim 1, characterized in that, The observation data encoding module includes an encoding module convolutional layer, an encoding module causal embedding layer, and an encoding module data compression layer; the observation data decoding module includes a decoding module data decoding layer, a decoding module causal embedding layer, and a decoding module deconvolutional layer; the multi-scale noise discrimination module includes an adaptive multi-scale noise addition layer, a discrimination module convolutional layer, a discrimination module causal embedding layer, a discrimination module data compression layer, and a classification layer; The observed image data, causal label information, and causal graph matrix are input into the causal decoupling model. The observed image data is passed through the convolutional layer of the encoding module to obtain the convolutional representation. The convolutional layer representation, causal label information, and causal graph matrix are input together into the causal embedding layer of the encoding module for calculation. After passing through the data compression layer of the encoding module, the causal latent vector features are obtained. The causal latent vector features first pass through the data decoding layer of the decoding module to obtain the decoded representation. The decoded representation, causal label information, and causal graph matrix are input together into the causal embedding layer of the decoding module for calculation. After passing through the deconvolution layer of the decoding module, the reconstructed data is obtained. The reconstructed data and the observed image data are respectively fed into the adaptive multi-scale noise-adding layer to obtain the noisy reconstructed data and observed image data. The noisy reconstructed data and observed image data are simultaneously passed through the convolution layer of the discriminator module to obtain the convolutional features. The convolutional features, causal label information, and causal graph matrix are input together into the causal embedding layer of the discriminator module for calculation. After passing through the data compression layer and classification layer of the discriminator module in sequence, the true and false discrimination probabilities are output.

4. The causal decoupling method based on multi-scale noise and adversarial supervision according to claim 3, characterized in that, The adaptive multi-scale noise layer specifically includes: initializing the number of noise addition steps and generating a temporal embedding, and then using a forward noise addition method of a diffusion model to add noise to the reconstructed data and the observed image data to obtain the noise-added reconstructed data and the observed image data.

5. The causal decoupling method based on multi-scale noise and adversarial supervision according to claim 4, characterized in that, The initialization of the noise-adding steps and the generation of the temporal embedding specifically include: Initialize maximum noise scale Given a time-scale embedding dimension d, the time scale is encoded using triangular embedding. The time-scale embedding basis is calculated using a frequency function. The noisy step sequence T is then processed using the time-scale embedding basis and subjected to sine and cosine transforms respectively. The embeddings after sine and cosine transforms are then concatenated to obtain a time-scale embedding vector of length d.

6. The causal decoupling method based on multi-scale noise and adversarial supervision according to claim 4, characterized in that, The method of adding noise to the reconstructed data and observed image data using a diffusion model specifically includes: Based on normal distribution noise and the number of noise addition steps, noise is added to the reconstructed data and observed image data by accumulating noise factor to obtain the noise-added reconstructed data and observed image data; wherein, the accumulating noise factor is obtained according to the noise variance and time scale embedding vector at the corresponding time scale.

7. The causal decoupling method based on multi-scale noise and adversarial supervision according to claim 1, characterized in that, The step of using a training set to perform adversarial supervised training under multi-scale noise to obtain a trained causal decoupling model specifically includes: The observed image data, causal label information, and causal graph matrix in the training set are input into the causal decoupling model to obtain causal latent vector features, reconstructed data, and true / false discrimination probabilities. The autoencoder loss is calculated based on observed image data, causal graph matrix, causal latent vector features, reconstructed data, and true / false discrimination probability. The discriminative adversarial loss is calculated based on the true / false discrimination probability corresponding to the observed image data and reconstructed data. During training, the autoencoder loss and discriminative adversarial loss are optimized alternately. When optimizing the autoencoder loss, only the parameters of the observation data encoding module and the observation data decoding module are updated; when optimizing the discriminative adversarial loss, only the parameters of the observation data decoding module and the multi-scale noise discrimination module are updated; until the preset number of training rounds is reached or the autoencoder loss and discriminative adversarial loss converge, the trained causal decoupling model is obtained.

8. The causal decoupling method based on multi-scale noise and adversarial supervision according to claim 7, characterized in that, The autoencoder loss is obtained specifically through the following method: The root mean square error between the observed image data and the reconstructed data is calculated as the reconstruction loss; the KL divergence loss is calculated based on the causal latent vector features and their prior distribution; the generation loss is calculated based on the true / false discrimination probability; the total variational loss and entropy loss are calculated based on the reconstructed data; the adversarial generation loss is calculated based on the generation loss, total variational loss, and entropy loss; the directed acyclic loss is calculated based on the causal graph matrix; and the autoencoder loss is calculated based on the reconstruction loss, KL divergence loss, adversarial generation loss, and directed acyclic loss.

9. The causal decoupling method based on multi-scale noise and adversarial supervision according to claim 1, characterized in that, Step (3) specifically includes: The observed image data, causal label information, and trained causal graph matrix in the test set are fed into the trained observation data encoding module. The observed image data is passed through the convolutional layer of the encoding module to obtain convolutional representation. The convolutional layer representation, causal label information, and the trained final causal graph matrix are fed into the causal embedding layer of the encoding module for calculation. After passing through the data compression layer of the encoding module, the causal latent vector features are obtained. Select the causal variables that need intervention, calculate their intervention values, and modify the corresponding causal variables in the causal latent vector features according to the intervention values ​​to obtain the causal latent vector features after intervention. The causal latent vector features after intervention, causal label information, and the trained causal graph matrix are fed into the trained observation data decoding module. The causal latent vector features after intervention first pass through the data decoding layer of the decoding module to obtain the decoded representation. The decoded representation, causal label information, and the trained causal graph matrix are then input into the causal embedding layer of the decoding module for calculation. After passing through the deconvolution layer of the decoding module, the counterfactual intervention data is obtained.

10. A causal decoupling device based on multi-scale noise and adversarial supervision, used to implement the causal decoupling method based on multi-scale noise and adversarial supervision as described in any one of claims 1-9, characterized in that, The device includes: The causal data synthesis module is used to simulate the causal relationship of variables in the causal graph using a priori causal mechanism, generate observed image data, and construct training and test sets based on the observed image data, their corresponding causal label information, and the causal graph. The model building and training module is used to build a causal decoupling model and perform adversarial supervised training under multi-scale noise using a training set to obtain a trained causal decoupling model; wherein, the causal decoupling model includes an observation data encoding module, an observation data decoding module, and a multi-scale noise discrimination module; The causal counterfactual intervention module is used to input the observed image data, causal label information, and trained causal graph matrix from the test set into the trained observation data encoding module to obtain causal latent vector features, select the causal variables that need intervention, calculate their intervention values, modify the corresponding causal variables in the causal latent vector features, obtain the intervened causal latent vector features, and input them into the trained observation data decoding module to obtain counterfactual intervention data.

Images