Face feature decoupling representation method and device with causal effect transmission and medium

Abstract

The application discloses a face feature decoupling representation method and device with causal effect transmission and a medium, relates to the technical field of causal discovery, image generation and decoupling representation learning. The method comprises the following steps: establishing a variational autoencoder, learning a latent feature distribution from a latent space based on the variational autoencoder in response to input face data; integrating a structural causal model into the variational autoencoder, modeling a causal effect to extract a causal feature from the latent feature distribution; establishing a graph attention network, inputting the causal feature into the graph attention network to decouple and transmit the causal effect; and designing a loss function, training the graph attention network by using a discriminator. The method can effectively mine and transmit the causal relationship between face features, improve the interpretability and transferability of feature representation, has strong application potential, and is especially suitable for face recognition, expression analysis and other tasks.

Description

Technical Field

[0001] The present invention relates to the technical fields of causal discovery, image generation and decoupled representation learning, and more specifically, to a facial feature decoupled representation method, device and medium with causal effect transmission. Background Art

[0002] With the rapid development of deep learning and artificial intelligence technologies, traditional centralized learning methods have gradually exposed their shortcomings in handling large-scale data, heterogeneous tasks, and complex systems. In particular, in the context of massive data and diverse tasks, efficient task allocation, feature representation, and model training have become a major challenge in current research. In this context, disentangled representation learning, as an emerging technical framework, has gradually become an effective tool for addressing this problem.

[0003] The core idea of ​​disentangled representation learning is to decouple data representation from task processing, enabling the learning model to better handle diverse tasks without relying on global task objectives. Compared to traditional unified models, disentangled representation learning uses a hierarchical representation of input data to independently learn features at different levels, thereby improving the model's generalization ability in multi-task environments. This approach is particularly suitable for modern applications with extremely high requirements for real-time performance, flexibility, and efficiency, such as autonomous driving, personalized recommendations, and intelligent manufacturing.

[0004] The widespread adoption of advanced technologies such as 5G, the Internet of Things (IoT), edge computing, and reinforcement learning has strongly supported the application of disaggregated representation learning. The low latency and high bandwidth of 5G technology make edge computing a crucial platform for efficient task processing. Building on this foundation, disaggregated representation learning flexibly addresses the data and task allocation issues between edge and cloud computing. This allows computing tasks to be intelligently scheduled and processed locally at the edge or in the cloud based on their real-time and resource requirements, improving overall system efficiency. The rise of disaggregated representation learning offers new solutions for processing multiple tasks and large amounts of data. Especially in today's highly dynamic and complex application environments, disaggregated representation learning not only improves model efficiency and performance but also lays the foundation for smarter and more efficient task processing.

[0005] Disentangled representation learning aims to learn a model that can identify and disentangle the underlying factors hidden in observable data. This process of separating the varying underlying factors into semantically meaningful variables helps learn interpretable representations of the data, mimicking the process of meaningful comprehension humans experience when observing an object or relationship. When humans observe an object, they attempt to understand its various attributes (such as shape, size, and color) and incorporate certain prior knowledge. Existing end-to-end black-box deep learning models take a shortcut by directly learning object representations that conform to data distribution and differentiation criteria, failing to extract hidden attributes with human-like generalization capabilities. Therefore, disentangled representation learning is proposed. However, existing face disentanglement methods can obtain latent representations of causal features in an image and then intervene on these latent representations and reflect them on the image. However, these interventions all modify a single dimension of the latent representation and fail to consider the propagation of causal effects. Therefore, even in the latent space, the causal relationship between parent and child nodes cannot be ignored.

[0006] In view of the above research background, based on the problems of potential representation intervention and causal feature decoupling in the current big data era, we studied and designed a causal decoupling method that incorporates graph attention networks, optimized the semi-supervised decoupling model, combined with relevant knowledge and technologies such as variational autoencoders, graph attention networks and generative adversarial networks, and used a small amount of label information to learn a stable, efficient and reliable causal feature decoupling method on face datasets. Summary of the Invention

[0007] To solve the above technical problems, the present invention provides a facial feature decoupling representation method, device and medium with causal effect transmission. Taking into account the complex attribute characteristics of facial datasets, in order to ensure that the structural causal model (SCM) can learn the causal structure that conforms to the prior, a nonlinear SCM is used to process the data, and a graph attention network (GAT) is used to realize the transmission of causal effects.

[0008] In a first aspect, the present invention provides a method for decoupling facial features with causal effect transmission, the method comprising:

[0009] Establishing a variational autoencoder, and in response to input face data, learning a latent feature distribution from a latent space based on the variational autoencoder;

[0010] Integrating a structural causal model into the variational autoencoder to model causal effects to extract causal features from the latent feature distribution;

[0011] Establishing a graph attention network, inputting the causal features into the graph attention network to transmit causal effects when decoupling;

[0012] A loss function is designed, and the variational autoencoder, discriminator, causal structure model, and graph attention network are used as the causal representation model, and the loss function is used to train the causal representation model.

[0013] Furthermore, the variational autoencoder includes an encoder and a decoder; wherein, the encoder is used to map input data to a latent space representation and control the encoding distribution through mean and variance parameters, and the decoder reconstructs the original data based on the sampled latent space vector.

[0014] Furthermore, a variational autoencoder is established by the following method:

[0015] Determine the loss function of the variational autoencoder, expressed as:

[0016]

[0017] Where, is the loss of the variational autoencoder; q(z|x), p(x|z) and p(z) are the latent variable, the reconstructed distribution and the prior distribution respectively; x is the observed data, z is the latent endogenous variable; KL is the KL divergence function; E q(z|x) [logp(x|z)] is the pixel-based cross entropy loss;

[0018] Utilizing Adversarial Similarity Loss Replace the pixel-based cross entropy loss E q(z|x) [logp(x|z)], adversarial similarity loss Expressed as:

[0019]

[0020] Where, Dis b (x) is represented by the feature vector mined by the discriminator in each layer, satisfying the Gaussian distribution The image generated by the decoder is designed by the decoder to lose the discriminator as Where z is encoded by the encoder Enc(x), N is a Gaussian distribution, Dis b (x) is the discriminant loss of layer b, is the discriminant loss of the output result, I is the unit matrix, Dis(x) is the total loss of all discriminant layers, and Dec(z) is the decoding result of the latent variable;

[0021] use and To update the gradient in the decoder:

[0022]

[0023] Where λ is the weight parameter, Θ Dec is the gradient of the decoder, Update the gradient derivative of the decoder;

[0024] Given observation data x and the corresponding latent endogenous variables z, z's latent exogenous variables ε and supervised labels l, to maximize the log-likelihood log pΘ (x) as the target, introducing the variational distribution q φ (ε,z|x,l) to approximate the true posterior distribution p Θ (ε,z|x,l), where φ and Θ are the parameters of the variational distribution and the model distribution, respectively;

[0025] Introduce KL divergence into log-likelihood, rearrange and simplify it, and use Bayes' theorem to convert log pΘ (x) is rewritten as the conditional probability log pΘ (x|z) and prior probability p Θ (z), thus we get the following evidence lower bound ELBO:

[0026]

[0027] Where D KL (q φ (ε,z|x,l)||p Θ (ε,z|l)) is the KL divergence;

[0028] The variational distribution is simplified by the following formula:

[0029] q φ (ε,z|x,l)=q φ (ε|x,l)f(z=(IA T ) -1 ε)

[0030] Where f is a nonlinear or linear function, A is the weighted binary adjacency matrix of the causal graph;

[0031] Based on the simplified variational distribution, a new lower bound of evidence is obtained, which is expressed as:

[0032]

[0033] Where, is the mean of the variation of the entire observation data, is the mean of the variational posterior of the latent variable.

[0034] Furthermore, a structural causal model is integrated into the variational autoencoder to model causal effects to extract causal features from the latent feature distribution, including:

[0035] Given the causal relationships, an adjacency matrix is ​​learned in a supervised manner, wherein the weights of the adjacency matrix reflect the causal strength;

[0036] Based on the adjacency matrix, z is determined by the following formula:

[0037] z=f2((IA T ) -1 f1(ε)

[0038] Where A is the weighted binary adjacency matrix of the causal graph, f1 and f2 are nonlinear and linear functions;

[0039] z=f2((IA T ) -1 f1(ε)) can be rewritten as

[0040] exist In the , add the new term KL(q(z i |l)||p(l)), we get:

[0041]

[0042] Where, KL(q(z i |l)||p(l)) is the KL divergence between the prior and the latent variable of dimension i; z i is the i-th dimension of z;

[0043] The generation of causal features is supervised by the first i items of z.

[0044] Furthermore, a graph attention network is established, and the causal features are input into the graph attention network to transmit causal effects when decoupling, including:

[0045] By z i Intervene and use the supervised label l to obtain z without causal aggregation:

[0046] z=do(Supervise([z]1,[z]2,...,[z] i ; l) i =c)

[0047] Where c is in the interval [-i,i] and z i There is a distance between the potential endogenous variables [z] i The number of, where [z] i represents the i-th z;

[0048] Based on the graph attention network GAT, the graph attention mechanism is used to reconstruct the formula by dynamically assigning weights to neighbor nodes, processing each z separately.i , concatenate the processed features and transpose them into the reconstructed vector z′:

[0049] z′=transpose([GAT(z1),GAT(z2),...,GAT(z i )])

[0050] In the formula, transpose is the matrix transposition, GAT(z i ) is to apply the attention mechanism to the i-th dimension latent variable;

[0051] The reconstructed vector z′ learns the weights of the causal edges in A through GAT, aggregates the influence of the parent nodes, and trains GAT to focus on the transmission of causal effects;

[0052] Let z={z i |1<i≤N,z i ∈R F}, where N is the number of z, F is the dimension of z, and the output z′={z′ i |1<i≤N,z′ i ∈R F Generated by GAT, combining aggregated information from neighboring nodes;

[0053] Training weight matrix W∈R F×F , R is a real number domain, and the weight matrix represents the relationship between input and output features;

[0054] Using the attention coefficient e ij represents the j-th dimension z of z j and the i-th dimension z i The importance between ij =a(Wz i ,Wz j ), For z i The weight matrix, For z j The weight matrix, α ij It is from z i to z j The weight of the edge is expressed as:

[0055]

[0056] Where, softmax i (e ij ) is the activation function used for normalization, exp is the exponential function with a base of a natural constant, k is the kth attention head, N i is the total number of parent nodes, e jk is the attention correlation score between nodes j and k;

[0057] Formulate α using the attention mechanism on a single node ij :

[0058]

[0059] Where LeakyRelu is a nonlinear activation function, is the kth dimension z of z k The weight of

[0060] Will As the final output feature of each node, a multi-head attention mechanism is used to enhance stability. The multi-head attention mechanism uses K independent attention heads to transform z into z′ through the following formula:

[0061]

[0062] In the formula, ‖ represents the splicing operation, represents the normalized attention coefficient calculated by the kth attention head, W k is the weight matrix corresponding to the linear transformation of the input.

[0063] Furthermore, the loss function is designed, including:

[0064] According to KL divergence, reconstruction loss Decoder loss and monitoring losses Establish the loss function, expressed as:

[0065]

[0066] Where, is the total loss of the model; β is the weight parameter;

[0067] KL is the KL divergence, which is used to fit the standard Gaussian distribution and is expressed as:

[0068]

[0069] It is used to shorten the distance between the i-th dimension of z and the true label, expressed as:

[0070]

[0071] Where z i denotes the i-th dimension of z, l denotes the label, and i is the number of attributes of interest.

[0072] Furthermore, after designing a loss function, using a variational autoencoder, a discriminator, a causal structure model, and a graph attention network as a causal representation model, and training the causal representation model using the loss function, the method further includes:

[0073] The identifiability of the causal representation model is analyzed.

[0074] Furthermore, the identifiability of the causal representation model is analyzed, including:

[0075] Let ~ be a binary relation on parameter Θ and define:

[0076]

[0077] Where Θ(E, D, A, T, λ) is the model parameter, E is the encoder, D is the decoder, A is the weighted binary adjacency matrix of the causal graph, and T is the time step; are the encoding output results of the corresponding parameters, B1 is the parameter matrix, B2 is the bias vector, b2, b1 are scalars, and x is a single observation data. is a linear mapping, is the inverse mapping transformation, For the entire dataset;

[0078] A number Θ is identifiable if the following conditions are met:

[0079] gather The measure of is zero, where φ ξ Indicates p Θ (ε,z|l)=p ε (ε)p Θ (z|l) medium density p ξ The characteristic function of

[0080] The function D as the decoder is differentiable, and the Jacobian matrix of the decoder function remains full rank;

[0081] Sufficient Statistics T i,s (z i ) is non-zero at all points, T i,s (z i ) represents the i-th dimension variable z i The sth statistic of ;

[0082] Additional observations satisfy l i ≠0.

[0083] In a second aspect, the present invention provides a facial feature decoupling representation device with causal effect transmission, the device comprising:

[0084] a feature distribution acquisition module configured to establish a variational autoencoder, and in response to input face data, learn a latent feature distribution from a latent space based on the variational autoencoder;

[0085] a causal feature extraction module configured to integrate a structural causal model into the variational autoencoder to model causal effects to extract causal features from the latent feature distribution;

[0086] a causal feature decoupling module, configured to establish a graph attention network, and input the causal features into the graph attention network to transmit causal effects during decoupling;

[0087] The model training module is configured to design a loss function, use a variational autoencoder, a discriminator, a causal structure model, and a graph attention network as a causal representation model, and use the loss function to train the causal representation model.

[0088] In a third aspect, the present invention provides a readable storage medium, wherein the readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the method as described above.

[0089] The present invention has at least the following beneficial effects:

[0090] This paper explores for the first time the learning of decoupled representations with causal effect transfer, solving the problem that changes in intervention features cannot trigger corresponding causal features, thereby avoiding situations where the generated results are contrary to common sense. In order to efficiently capture causal features and learn causal structures, a nonlinear or linear SCM is constructed, which encodes potential exogenous variables as potential endogenous variables, and designs a discriminator with hierarchical feature loss to replace the pixel-level loss in VAE, thereby enhancing the decoupled representation. In order to transfer causal effects, a GAT intervention mechanism is designed to aggregate the causal information of adjacent nodes by continuously learning the weights of the edges in the causal matrix, thereby naturally decoupling causal features. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] Figure 1 A schematic diagram of a system architecture for implementing a facial feature decoupling representation method with causal effect transmission according to an embodiment of the present invention is shown.

[0092] Figure 2 The process of a facial feature decoupling representation method with causal effect transmission according to an embodiment of the present invention is shown. Figure 1 .

[0093] Figure 3 The process of a facial feature decoupling representation method with causal effect transmission according to an embodiment of the present invention is shown. Figure 2 .

[0094] Figure 4 A structural diagram of a facial feature decoupling representation device with causal effect transfer according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0095] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments. The embodiments of the present invention are further described in detail below with reference to the accompanying drawings and specific embodiments, but are not intended to limit the present invention. For the various steps described herein, if there is no necessity for a contextual relationship between each other, the order in which they are described as examples herein should not be regarded as limiting, and those skilled in the art should know that they can be adjusted in order as long as the logic between them is not destroyed, resulting in the inability to implement the entire process.

[0096] Traditional facial feature representation learning methods mostly rely on pixel-level reconstruction or representation learning, making it difficult to effectively capture causal relationships and their influence on features. However, in many application scenarios, understanding and communicating causal effects between features is crucial for improving model interpretability and generalization. To address this issue, embodiments of the present invention provide a facial feature decoupling representation method with causal effect communication. This method combines the advantages of variational autoencoders (VAEs) and generative adversarial networks (GANs) to learn causal structures and intervene in causal features in the latent space, thereby achieving more robust and meaningful facial feature decoupling. The innovations include: first, using the intermediate layer features of the discriminator as the reconstruction loss, replacing the pixel-level cross-entropy loss in traditional VAEs, ensuring effective learning of causal structures; second, employing an additive noise model to process partially labeled latent spaces, thereby converting latent exogenous variables into latent endogenous variables and achieving decoupling of causal features in the latent space; finally, by introducing a graph attention network (GAT) mechanism, causal effects are effectively communicated, and a probability generation mechanism is injected into the model to further optimize the decoupling effect. This method can effectively explore and transmit the causal relationship between facial features, improve the interpretability and transferability of feature representation, and has strong application potential, especially suitable for tasks such as face recognition and expression analysis.

[0097] like Figure 1 Figure 2 shows the system architecture for implementing this method for decoupling facial feature representation with causal effect transfer. The system architecture consists of four components: a variational autoencoder (VAE), a discriminator, a causal structure model (SCM), and a graph attention network (GAT). Based on the complexity of the prior causal structure of the input image data, the SCM is categorized into nonlinear and linear SCMs.

[0098] VAEs primarily consist of two parts: an encoder and a decoder. The encoder maps input data to a latent space representation, controlling the encoding distribution through mean and variance parameters. The decoder reconstructs the original data based on the sampled latent space vectors. The encoder's output space is Gaussian distributed. When new data is input, it is encoded in the latent space according to the rules of variational inference.

[0099] Considering the invariance of causal relationships in high-dimensional feature spaces, we propose a novel feature extraction method. Specifically, we introduce the Discriminator as the core component of feature extraction, simultaneously feeding the original input image and the output of the GAT into the discriminator. Unlike the pixel-level loss function used by VAEs, we utilize the discriminator's hierarchical feature loss to guide model learning. This approach effectively captures causal features in the data while avoiding the potential noise interference associated with direct feature extraction in the pixel space.

[0100] SCMs come in two types: linear and nonlinear, designed to process simple and complex datasets, respectively. Because facial features are often intertwined and difficult to distinguish, nonlinear SCMs are used for facial datasets. The latent feature distribution of the facial data after passing through the encoder is fed into the nonlinear SCM to obtain the causal feature distribution and simultaneously update the edge weights of the causal graph.

[0101] While the causal features obtained through SCM reflect the causal relationships between variables, they do not fully consider the influence mechanisms of parent nodes. Therefore, we propose a causal effect transfer method using GAT. Inputting the extracted causal features into GAT effectively simulates the information flow process in the causal graph. GAT's attention mechanism adaptively learns the influence weights of different parent nodes on the current node. This not only considers direct causal relationships but also captures the complex causal effects generated by the combined action of multiple parent nodes, enabling the model to more comprehensively understand and express the causal dependencies between variables.

[0102] Considering that the performance of GAT on face datasets varies with the number of attention heads, we change the number of attention heads while keeping the SCM parameters unchanged.

[0103] In the facial feature decoupling representation method with causal effect transmission, the decoupling representation process is as follows:

[0104] (1) Considering that the original face dataset is too large for a single image, it is necessary to perform operations such as cropping, rotation, and normalization on the dataset. The processed data is then fed into the encoder to obtain the latent distribution of the features. In order to improve the training efficiency and generalization ability of the model, we first use the face detection algorithm to locate and crop the face area, and uniformly adjust the image size to 224×224 pixels. Subsequently, the model's robustness to posture changes is improved by randomly rotating the data by ±10 degrees. The pixel values ​​of the image are normalized and mapped to the interval [-1,1] to accelerate network convergence. After these preprocessing steps, the data is fed into the encoder composed of a multi-layer convolutional neural network. Under the prior p(l), the encoder compresses the high-dimensional image features into a low-dimensional latent space ε=E(x,l)+ζ through dimensionality reduction operations, where ε is the latent exogenous variable, l is the label, and ζ is the noise variable. This makes the distribution of similar face images in the latent space more compact, laying the foundation for subsequent face recognition tasks.

[0105] (2) The potential exogenous variable ε needs to be processed by nonlinear SCM to be transformed into an endogenous variable related to facial attributes. The overall encoding process is q φ (z,ε|x,l)≡q(z|ε)q ζ (ε-E(x,l)), the decoding process is p Θ (x|z,ε,l)=p Θ (x|z)≡p ξ (xD(z)), where the joint prior of z and ε is p Θ (ε,z|l)=p ε (ε)p Θ (z|l), where p ε (ε) satisfies the standard normal distribution N(0, I) (I represents the identity matrix), indicating that z and ε need to be constrained by the label and KL divergence. The prior p of z Θ (z|l) is a factorized Gaussian distribution conditioned on label l Where F1 and F2 represent arbitrary functions. In order to align z and l in the initial dimension of the latent space, we set F1(l) = l and F2(l) ≡ 1. The mean μ(z) and variance σ(z) of z are given by the sufficient statistics T(z) = (μ(z), σ(z)) = (T 1,1 (z1),...,T n,2 (z n )). In this case, the statistics of T(z) satisfy E[T(z)]=μ(z) and Var[T(z)]=σ 2 (z), which provides an important basis for the subsequent analysis of probability distribution.

[0106] (3) Given that intervening on a node will lead to changes in all levels of child nodes with causal relationships, the advantage of our method is that it is more natural than using the disentangled features in the latent space obtained by traditional models to generate images. The latent endogenous variable z needs to pass through GAT to aggregate the causal information of adjacent nodes, so that the causal effect can be transmitted between parent and child nodes. When the i-th dimension z in the latent endogenous variable i and the jth dimension z j There is a causal effect i →z j When the method is z i Intervene and let z j Change accordingly.

[0107] Specifically, if Figure 2 As shown, this is the process of the facial feature decoupling representation method with causal effect transmission Figure 1 The facial feature decoupling representation method with causal effect transmission can be implemented by the following steps S10 to S40.

[0108] S10: Establish a variational autoencoder, and in response to input face data, obtain a potential feature distribution from a latent space based on the variational autoencoder.

[0109] In order to deeply explore the causal structure in face data, it is necessary to learn feature distribution from the latent space.

[0110] In some implementations, a variational autoencoder (VAE) is designed as follows:

[0111] Loss of VAE It includes a reconstruction loss and KL divergence, where q(z|x), p(x|z), and p(z) are the latent variable, the reconstruction distribution, and the prior distribution respectively:

[0112]

[0113] In order to achieve more accurate feature distribution in the latent space, an adversarial similarity loss is used Replace the pixel-based cross entropy loss E q(z|x) [logp(x|z)]. Considering that images of the same category exhibit highly similar feature distributions, is from the discriminator Dis b The middle layer of (·) is derived as Among them, Dis b (x) is the feature vector representation mined by the discriminator in each layer. The obtained feature representations satisfy the Gaussian distribution in The image generated by the decoder, through the decoder in VAE, the loss of the discriminator can be designed as Where z is encoded by the encoder Enc(x). and To update the gradient in the decoder Where λ is a weight parameter. Given the observation data x and the corresponding latent endogenous variable z, the latent exogenous variable ε of z and the supervised label l, the goal is to maximize the log-likelihood: log pΘ (x). In order to effectively utilize labels and variables, a variational distribution q is introduced φ (ε,z|x,l) to approximate the true posterior distribution p Θ (ε,z|x,l), where φ and Θ are the parameters of the variational distribution and the model distribution respectively. In order to maximize the log-likelihood, this embodiment introduces the KL divergence into the log-likelihood and then rearranges and simplifies it. Using Bayes' theorem, we transform log pΘ (x) is rewritten as the conditional probability log pΘ (x|z) and prior probability p Θ (z), we get the following evidence lower bound (ELBO):

[0114]

[0115] D KL (q φ (ε,z|x,l)||p Θ (ε,z|l)) is the KL divergence. The variable z is derived from ε via the SCM and subsequent nonlinear or linear transformations. This process simplifies the variational posterior distribution:

[0116] q φ (ε,z|x,l)=q φ (ε|x,l)f(z=(IA T ) -1 ε)

[0117] f(·) is a nonlinear or linear function, and A is a weighting matrix. A new lower bound can be obtained:

[0118]

[0119] The first term is the reconstruction loss, the second term ensures that ε conforms to the standard normal distribution, and the third term constrains the first i dimensions of z to correspond one-to-one with the labels. To minimize the KL divergence, as well as This completes the design of VAE, making the model more focused on mining potential features and having better image generation effects.

[0120] S20: Integrate a structural causal model into the variational autoencoder to model causal effects to extract causal features from the latent feature distribution.

[0121] By integrating the structural causal model (SCM) into the VAE, the causal effect is modeled so that causal features can be extracted from the distribution of the latent space. Given the causal relationship, the method learns an adjacency matrix in a supervised manner, whose weights reflect the causal strength. The structural causal model is then used as a priori p z .

[0122] z=f2((IA T ) -1 f1(ε)

[0123] Where A is the weighted binary adjacency matrix of the causal graph (directed acyclic graph, or DAG), I is the identity matrix, and f1 and f2 are nonlinear or linear functions. The exogenous noise ε follows a Gaussian distribution N(0, I). The parameter set β(f1, f2, A) needs to be optimized in the parameter space B. In order to adapt to the SCM in the method, z = f2((IA T ) -1 f1(ε)) can be rewritten as z=f β ((IA T ) -1 ε). f(·) is reversible and can be rewritten as The ε in SCM is encoded by x. In the , a new term KL(q(z i |l)||p(l)):

[0124]

[0125] In order to supervise the generation of causal features through the first i items of z. The model uses z = f β ((IA T ) -1 ε) realizes the mining of causal features, which learns z by generating a distribution consistent with the supervised features. Initially, we fill A with zeros to approximate the causal graph, and then train and update f2((IA T ) -1 f1(ε)) and The parameters of are used to obtain the causal features, i.e., the potential endogenous variables z. This method combines the DAG structure, the encoder network, and the KL divergence regularization term to achieve learning and inference of causal relationships while ensuring the interpretability and generalization ability of the model.

[0126] S30: Establish a graph attention network, and input the causal features into the graph attention network to transmit causal effects when decoupling.

[0127] In some embodiments, causal effects are delivered while being decoupled as follows.

[0128] By z i We intervene in the first i dimensions of , and use the supervised label l to obtain z without causal aggregation:

[0129] z=do(Supervise([z]1,[z]2,...,[z] i ; l) i =c)

[0130] c is in the interval [-i,i] and z i There is a distance between the potential endogenous variables [z] i The number of, where [z] i Denotes the i-th z. To further characterize causal transmission, the graph attention mechanism is used to reconstruct the formula z by dynamically assigning weights to neighboring nodes. Unlike the traditional GAT model, our model processes each z separately i , then concatenate the processed features and transpose them to z′:

[0131] z′=transpose([GAT(z1),GAT(z2),...,GAT(z i )])

[0132] The reconstructed z′ learns the weights of the causal edges in A through GAT and aggregates the influence of the parent node. By training GAT, the model can pay more attention to the transmission of causal effects. Let z = {z i |1<i≤N,z i ∈R F}, where N is the number of z and F is its dimension. Output z′={z′ i |1<i≤N,z′ i ∈R F} is generated by GAT, combining aggregate information from neighboring nodes. To derive the output from the input, a weight matrix W∈R F×F The matrix represents the relationship between input and output features. ij represents z j and z i The importance between ij =a(Wz i ,Wz j ). α ij It is from z i to z j The weight of the edge.

[0133]

[0134] GAT is a single-layer feedforward neural network parameterized by a weight vector α and adopts the nonlinear activation function LeakyRelu. It uses the attention mechanism on a single node to formulate α ij :

[0135]

[0136] In addition, method α ij Calculate the linear aggregation of the corresponding features, which will be used as the final output feature of each node We adopt a multi-head attention mechanism to enhance the stability of the model, which uses K independent attention heads and is expressed by the formula Convert z to z′. ‖ represents the concatenation operation, represents the normalized attention coefficient calculated by the kth attention head, W k is the weight matrix corresponding to the linear transformation of the input. The multi-head attention mechanism effectively captures the relationships and feature dependencies at different levels between nodes through the parallel computation of multiple independent attention heads, improving the model's expressiveness and stability in complex graph structures.

[0137] S40: Design a loss function, use the variational autoencoder, the discriminator, the causal structure model, and the graph attention network as the causal representation model, and use the loss function to train the causal representation model.

[0138] Design loss function It comes from the discriminator and VAE. In the discriminator, the loss is used to distinguish between the real image x and the generated image The total loss of the method consists of four parts: KL divergence, reconstruction loss Decoder loss and monitoring losses

[0139]

[0140] Among them, KL divergence is used to fit the standard Gaussian distribution:

[0141]

[0142] For optimizing decoders Used to shorten the distance between the first i dimensions of z and the true label, where β is a weight parameter:

[0143]

[0144] z i denotes the first i dimensions of z, l denotes the labels, and i is the number of attributes of interest. is the reconstruction loss. By comprehensively considering KL divergence, reconstruction loss, decoder loss, and supervision loss, this loss function not only ensures that the model effectively learns the data distribution, but also achieves decoupling and supervision constraints on potential causal features, thereby improving the quality of generated images and the accuracy of causal effect transmission.

[0145] In some embodiments, as Figure 3 As shown, this is the process of the facial feature decoupling representation method with causal effect transmission Figure 2 After step S40, the method further includes:

[0146] S50: Using a variational autoencoder, a discriminator, a causal structure model, and a graph attention network as a causal representation model; analyzing the identifiability of the causal representation model.

[0147] This example performs identifiability analysis and expands the identifiability results of causal representation learning based on structural causal priors. Let ~ be a binary relation on Θ and define:

[0148]

[0149] definition If B1 is invertible and is a diagonal matrix whose diagonal elements are equal to z i If the model parameters are related, then the model parameters are identifiable. Assume that the observation data is sampled from the generative model and the model parameters are Θ(E, D, A, T, λ). When the following conditions are met: (1) the set The measure of is zero, where φ ξ Indicates p Θ (ε,z|l)=p ε (ε)p Θ (z|l) medium density p ξ characteristic function; (2) the decoder function D is differentiable and its Jacobian matrix maintains full rank; (3) the sufficient statistic T i,s (z i ) is non-zero at almost all points, where T i,s (z i ) represents the variable z i The sth statistic, where 1≤i≤n,1≤s≤2; (4) additional observations satisfy l i≠0. Then, the parameter Θ is identifiable. By strictly constraining the above conditions, we prove the identifiability of the model parameters, thereby ensuring that the causal generative model can accurately learn the data distribution and causal structure at the theoretical level, providing robust theoretical support for practical applications.

[0150] In some embodiments, the method further includes step S60: using PyCharm software to perform a face dataset decoupling experiment on graphics card A100, minimizing the loss mentioned in step S40 to implement a decoupling method with causal effect transfer, and verifying the performance of the model in terms of reconstruction quality, causal feature extraction and generation effect.

[0151] The embodiment of the present invention also provides a facial feature decoupling representation device with causal effect transmission, such as Figure 4 As shown, the device includes:

[0152] The feature distribution acquisition module 401 is configured to establish a variational autoencoder, and in response to input face data, obtain a potential feature distribution from a latent space based on the variational autoencoder;

[0153] a causal feature extraction module 402 configured to integrate a structural causal model into the variational autoencoder to model causal effects to extract causal features from the latent feature distribution;

[0154] The causal feature decoupling module 403 is configured to establish a graph attention network and input the causal feature into the graph attention network to transmit the causal effect during decoupling;

[0155] The model training module 404 is configured to design a loss function, use the variational autoencoder, the discriminator, the causal structure model and the graph attention network as the causal representation model, and use the loss function to train the causal representation model.

[0156] In some embodiments, the variational autoencoder includes an encoder and a decoder; wherein, the encoder is used to map input data to a latent space representation and control the encoding distribution through mean and variance parameters, and the decoder reconstructs the original data based on the sampled latent space vector.

[0157] In some embodiments, the feature distribution acquisition module is further configured to establish a variational autoencoder by the following method:

[0158] Determine the loss function of the variational autoencoder, expressed as:

[0159]

[0160] Where, is the loss of the variational autoencoder; q(z|x), p(x|z) and p(z) are the latent variable, the reconstructed distribution and the prior distribution respectively; x is the observed data, z is the latent endogenous variable; KL is the KL divergence function; E q(z|x) [logp(x|z)] is the pixel-based cross entropy loss;

[0161] Utilizing Adversarial Similarity Loss Replace the pixel-based cross entropy loss E q(z|x) [logp(x|z)], adversarial similarity loss Expressed as:

[0162]

[0163] Where, Dis b (x) is represented by the feature vector mined by the discriminator in each layer, satisfying the Gaussian distribution The image generated by the decoder is designed by the decoder to lose the discriminator as Where z is encoded by the encoder Enc(x), N is a Gaussian distribution, Dis b (x) is the discriminant loss of layer b, is the discriminant loss of the output result, I is the unit matrix, Dis(x) is the total loss of all discriminant layers, and Dec(z) is the decoding result of the latent variable;

[0164] use and To update the gradient in the decoder:

[0165]

[0166] Where λ is the weight parameter, Θ Dec for, Update the gradient derivative of the decoder;

[0167] Given observation data x and the corresponding latent endogenous variables z, z's latent exogenous variables ε and supervised labels l, to maximize the log-likelihood log pΘ (x) as the target, introducing the variational distribution q φ (ε,z|x,l) to approximate the true posterior distribution p Θ (ε,z|x,l), where φ and Θ are the parameters of the variational distribution and the model distribution, respectively;

[0168] Introduce KL divergence into log-likelihood, rearrange and simplify it, and use Bayes' theorem to convert log pΘ (x) is rewritten as the conditional probability log pΘ (x|z) and prior probability pΘ (z), thus we get the following evidence lower bound ELBO:

[0169]

[0170] Where D KL (q φ (ε,z|x,l)||p Θ (ε,z|l)) is the KL divergence;

[0171] The variational distribution is simplified by the following formula:

[0172] q φ (ε,z|x,l)=q φ (ε|x,l)f(z=(IA T ) -1 ε)

[0173] Where f is a nonlinear or linear function, A is the weighted binary adjacency matrix of the causal graph;

[0174] Based on the simplified variational distribution, a new lower bound of evidence is obtained, which is expressed as:

[0175]

[0176] Where, yes, yes.

[0177] In some embodiments, the causal feature extraction module is further configured to:

[0178] Given the causal relationships, an adjacency matrix is ​​learned in a supervised manner, wherein the weights of the adjacency matrix reflect the causal strength;

[0179] Based on the adjacency matrix, z is determined by the following formula:

[0180] z=f2((IA T ) -1 f1(ε)

[0181] Where A is the weighted binary adjacency matrix of the causal graph, f1 and f2 are nonlinear and linear functions;

[0182] z=f2((IA T ) -1 f1(ε)) can be rewritten as

[0183] exist In the , add the new term KL(q(z i |l)||p(l)), we get:

[0184]

[0185] Where, KL(q(z i |l)||p(l)) is; z i is the i-th dimension of z;

[0186] The generation of causal features is supervised by the first i items of z.

[0187] In some embodiments, the causal feature decoupling module is further configured to:

[0188] By z i Intervene and use the supervised label l to obtain z without causal aggregation:

[0189] z=do(Supervise([z]1,[z]2,...,[z] i ; l) i =c)

[0190] Where c is in the interval [-i,i] and z i There is a distance between the potential endogenous variables [z] i The number of, where [z] i represents the i-th z;

[0191] Based on the graph attention network GAT, the graph attention mechanism is used to reconstruct the formula by dynamically assigning weights to neighbor nodes, processing each z separately. i , concatenate the processed features and transpose them into the reconstructed vector z′:

[0192] z′=transpose([GAT(z1),GAT(z2),...,GAT(z i )])

[0193] In the formula, transpose is the matrix transposition, GAT(z i ) is to apply the attention mechanism to the i-th dimension latent variable;

[0194] The reconstructed vector z′ learns the weights of the causal edges in A through GAT, aggregates the influence of the parent nodes, and focuses on the transmission of causal effects by training GAT;

[0195] Let z={z i |1<i≤N,z i ∈R F}, where N is the number of z, F is the dimension of z, and the output z′={z′ i |1<i≤N,z′ i ∈R FGenerated by GAT, combining aggregated information from neighboring nodes;

[0196] Training weight matrix W∈R F×F , R is a real number domain, and the weight matrix represents the relationship between input and output features;

[0197] Using the attention coefficient e ij represents the j-th dimension z of z j and the i-th dimension z i The importance between ij =a(Wz i ,Wz j ), For z i The weight matrix, For z j The weight matrix, α ij It is from z i to z j The weight of the edge is expressed as:

[0198]

[0199] Where, softmax i (e ij ) is the activation function used for normalization, exp is the exponential function with a base of a natural constant, k is the kth attention head, N i is the total number of parent nodes, e jk is the attention correlation score between nodes j and k;

[0200] Formulate α using the attention mechanism on a single node ij :

[0201]

[0202] Where LeakyRelu is a nonlinear activation function, is the kth dimension z of z k The weight of

[0203] Will As the final output feature of each node, a multi-head attention mechanism is used to enhance stability. The multi-head attention mechanism uses K independent attention heads to transform z into z′ through the following formula:

[0204]

[0205] In the formula, ‖ represents the splicing operation, represents the normalized attention coefficient calculated by the kth attention head, W k is the weight matrix corresponding to the linear transformation of the input.

[0206] In some embodiments, the model training module is further configured to:

[0207] According to KL divergence, reconstruction loss Decoder loss and monitoring losses Establish the loss function, expressed as:

[0208]

[0209] Where, is the total loss of the model; β is the weight parameter;

[0210] KL is the KL divergence, which is used to fit the standard Gaussian distribution and is expressed as:

[0211]

[0212] It is used to shorten the distance between the i-th dimension of z and the true label, expressed as:

[0213]

[0214] Where z i denotes the i-th dimension of z, l denotes the label, and i is the number of attributes of interest.

[0215] In some embodiments, the device further includes an analysis module, which is configured to: use a variational autoencoder, a discriminator, a causal structure model, and a graph attention network as a causal representation model; and analyze the identifiability of the causal representation model.

[0216] In some embodiments, the analysis module is configured to:

[0217] Let ~ be a binary relation on parameter Θ and define:

[0218]

[0219] Where Θ(E, D, A, T, λ) is the model parameter, E is the encoder, D is the decoder, A is the weighted binary adjacency matrix of the causal graph, and T is the time step; are the encoding output results of the corresponding parameters, B1 is the parameter matrix, B2 is the bias vector, b2, b1 are scalars, and x is a single observation data. is a linear mapping, is the inverse mapping transformation, For the entire dataset;

[0220] A number Θ is identified if the following conditions are met:

[0221] gather The measure of is zero, where φ ξ Indicates p Θ (ε,z|l)=p ε (ε)p Θ (z|l) medium density p ξ The characteristic function of

[0222] The function D as the decoder is differentiable, and the Jacobian matrix of the decoder function remains full rank;

[0223] Sufficient Statistics T i,s (z i ) is non-zero at all points, T i,s (z i ) represents the i-th dimension variable z i The sth statistic of ;

[0224] Additional observations satisfy l i ≠0.

[0225] It should be noted that the structures of the various facial feature decoupling representation devices with causal effect transmission described in this embodiment belong to the same technical concept as the facial feature decoupling representation method with causal effect transmission described previously, and achieve the same beneficial effects through the same principles, which will not be repeated here.

[0226] An embodiment of the present invention further provides a readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the method described in any of the above embodiments.

[0227] Furthermore, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present invention having equivalent elements, modifications, omissions, combinations (e.g., schemes where various embodiments intersect), adaptations, or changes. The elements in the claims are to be interpreted broadly based on the language employed in the claims and are not limited to the examples described in this specification or during the prosecution of this application, which examples are to be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, with the true scope and spirit being indicated by the following claims and the full scope of their equivalents.

[0228] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of their solutions) can be used in combination with each other. For example, those of ordinary skill in the art may use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features can be grouped together to simplify the present invention. This should not be interpreted as an intention that a feature of an invention that is not claimed for protection is necessary for any claim. On the contrary, the subject matter of the present invention may be less than all the features of the embodiments of a particular invention. Thus, the following claims are incorporated into the specific embodiments as examples or embodiments, wherein each claim is independently a separate embodiment, and it is considered that these embodiments can be combined with each other in various combinations or arrangements. The scope of the present invention should be determined with reference to the appended claims and the full scope of equivalents to which these claims are entitled.

Claims

1. A facial feature decoupling representation method with causal effect transmission, characterized in that: The method comprises: Establishing a variational autoencoder, and in response to input face data, learning a latent feature distribution from a latent space based on the variational autoencoder; Integrating a structural causal model into the variational autoencoder to model causal effects to extract causal features from the latent feature distribution; Establishing a graph attention network, inputting the causal features into the graph attention network to transmit causal effects when decoupling; A loss function is designed, and the variational autoencoder, discriminator, causal structure model, and graph attention network are used as the causal representation model, and the loss function is used to train the causal representation model.

2. The facial feature decoupling representation method with causal effect transfer according to claim 1, characterized in that: The variational autoencoder includes an encoder and a decoder; wherein the encoder is used to map input data to a latent space representation and control the encoding distribution through mean and variance parameters, and the decoder reconstructs the original data based on the sampled latent space vector.

3. The facial feature decoupling representation method with causal effect transmission according to claim 2 is characterized in that: The variational autoencoder is constructed as follows: Determine the loss function of the variational autoencoder, expressed as: Where, is the loss of the variational autoencoder; q(z|x), p(x|z) and p(z) are the latent variable, the reconstructed distribution and the prior distribution respectively; x is the observed data, z is the latent endogenous variable; KL is the KL divergence function; E q(z|x) [logp(x|z)] is the pixel-based cross entropy loss; Utilizing Adversarial Similarity Loss Replace the pixel-based cross entropy loss E q(z|x) [logp(x|z)], adversarial similarity loss Expressed as: Where, Dis b (x) is represented by the feature vector mined by the discriminator in each layer, satisfying the Gaussian distribution The image generated by the decoder is designed by the decoder to lose the discriminator as Where z is encoded by the encoder Enc(x), N is a Gaussian distribution, Dis b (x) is the discriminant loss of layer b, is the discriminant loss of the output result, I is the unit matrix, Dis(x) is the total loss of all discriminant layers, and Dec(z) is the decoding result of the latent variable; use and To update the gradient in the decoder: Where λ is the weight parameter, Θ Dec is the gradient of the decoder, Update the gradient derivative of the decoder; Given observation data x and the corresponding latent endogenous variables z, z's latent exogenous variables ε and supervised labels l, to maximize the log-likelihood log pΘ (x) as the target, introducing the variational distribution q φ (ε,z|x,l) to approximate the true posterior distribution p Θ (ε,z|x,l), where φ and Θ are the parameters of the variational distribution and the model distribution, respectively; Introduce KL divergence into log-likelihood, rearrange and simplify it, and use Bayes' theorem to convert log pΘ (x) is rewritten as the conditional probability log pΘ (x|z) and prior probability p Θ (z), thus we get the following evidence lower bound ELBO: Where D KL (q φ (ε,z|x,l)||p Θ (ε,z|l)) is the KL divergence; The variational distribution is simplified by the following formula: q φ (ε,z|x,l)=q φ (ε|x,l)f(z=(I-A T ) -1 ε) Where f is a nonlinear or linear function, A is the weighted binary adjacency matrix of the causal graph; Based on the simplified variational distribution, a new lower bound of evidence is obtained, which is expressed as: Where, is the mean of the variation of the entire observation data, is the mean of the variational posterior of the latent variable.

4. The facial feature decoupling representation method with causal effect transfer according to claim 3, characterized in that: Integrating a structural causal model into the variational autoencoder to model causal effects to extract causal features from the latent feature distribution includes: Given the causal relationships, an adjacency matrix is ​​learned in a supervised manner, wherein the weights of the adjacency matrix reflect the causal strength; Based on the adjacency matrix, z is determined by the following formula: z=f2((I-A T ) -1 f1(ε)) Where A is the weighted binary adjacency matrix of the causal graph, f1 and f2 are nonlinear and linear functions; z=f2((IA T ) -1 f1(ε)) can be rewritten as f -1 (z) = A T f -1 (x)+ε; exist In the , add the new term KL(q(z i |l)||p(l)), we get: Where, KL(q(z i |l)||p(l)) is the KL divergence between the prior and the latent variable of dimension i; z i is the i-th dimension of z; The generation of causal features is supervised by the first i items of z.

5. The facial feature decoupling representation method with causal effect transfer according to claim 4 is characterized in that: Establishing a graph attention network, inputting the causal features into the graph attention network to transmit causal effects when decoupling, including: By z i Intervene and use the supervised label l to obtain z without causal aggregation: from=to(Supervise([from]1,[from]2,...,[from] i ;l) i =c) Where c is in the interval [-i,i] and z i There is a distance between the potential endogenous variables [z] i The number of, where [z] i represents the i-th z; Based on the graph attention network GAT, the graph attention mechanism is used to reconstruct the formula by dynamically assigning weights to neighbor nodes, processing each z separately. i , concatenate the processed features and transpose them into the reconstructed vector z′: z′=transpose([GAT(z1),GAT(z2),...,GAT(z i )]) In the formula, transpose is the matrix transposition, GAT(z i ) is to apply the attention mechanism to the i-th dimension latent variable; The reconstructed vector z′ learns the weights of the causal edges in A through GAT, aggregates the influence of the parent nodes, and focuses on the transmission of causal effects by training GAT; Let z={z i |1<i≤N,z i ∈R F }, where N is the number of z, F is the dimension of z, and the output z′={z i ′|1<i≤N,z i ′∈R F Generated by GAT, combining aggregated information from neighboring nodes; Training weight matrix W∈R F×F , R is a real number domain, and the weight matrix represents the relationship between input and output features; Using the attention coefficient e ij represents the j-th dimension z of z j and the i-th dimension z i The importance between ij =a(Wz i ,Wz j ), For z i The weight matrix, For z j The weight matrix, α ij It is from z i to z j The weight of the edge is expressed as: Where, softmax i (e ij ) is the activation function used for normalization, exp is the exponential function with a base of a natural constant, k is the kth attention head, N i is the total number of parent nodes, e jk is the attention correlation score between nodes j and k; Formulate α using the attention mechanism on a single node ij : Where LeakyRelu is a nonlinear activation function, is the kth dimension z of z k The weight matrix of Will As the final output feature of each node, a multi-head attention mechanism is used to enhance stability. The multi-head attention mechanism uses K independent attention heads to transform z into z′ through the following formula: In the formula, ‖ represents the splicing operation, represents the normalized attention coefficient calculated by the kth attention head, W k is the weight matrix corresponding to the linear transformation of the input.

6. The facial feature decoupling representation method with causal effect transfer according to claim 4, characterized in that: Design loss functions, including: According to KL divergence, reconstruction loss Decoder loss and monitoring losses Establish the loss function, expressed as: Where, is the total loss of the model; β is the weight parameter; KL is the KL divergence, which is used to fit the standard Gaussian distribution and is expressed as: It is used to shorten the distance between the i-th dimension of z and the true label, expressed as: Where z i denotes the i-th dimension of z, l denotes the label, and i is the number of attributes of interest.

7. The facial feature decoupling representation method with causal effect transmission according to claim 1 is characterized in that: After designing a loss function, using a variational autoencoder, a discriminator, a causal structure model, and a graph attention network as a causal representation model, and training the causal representation model using the loss function, the method further includes: The identifiability of the causal representation model is analyzed.

8. The facial feature decoupling representation method with causal effect transfer according to claim 7, characterized in that: The identifiability of the causal representation model is analyzed, including: Let ~ be a binary relation on parameter Θ and define: Where Θ(E, D, A, T, λ) is the model parameter, E is the encoder, D is the decoder, A is the weighted binary adjacency matrix of the causal graph, and T is the time step; are the encoding output results of the corresponding parameters, B1 is the parameter matrix, B2 is the bias vector, b2, b1 are scalars, and x is a single observation data. is a linear mapping, is the inverse mapping transformation, For the entire dataset; A number Θ is identified if the following conditions are met: gather The measure of is zero, where φ ξ Indicates p Θ (ε,z|l)=p ε (ε)p Θ (z|l) medium density p ξ The characteristic function of The function D as the decoder is differentiable, and the Jacobian matrix of the decoder function remains full rank; Sufficient Statistics T i,s (z i ) is non-zero at all points, T i,s (z i ) represents the i-th dimension variable z i The sth statistic of ; Additional observations satisfy l i ≠0.

9. A facial feature decoupling representation device with causal effect transmission, characterized in that: The device comprises: a feature distribution acquisition module configured to establish a variational autoencoder, and in response to input face data, learn a latent feature distribution from a latent space based on the variational autoencoder; a causal feature extraction module configured to integrate a structural causal model into the variational autoencoder to model causal effects to extract causal features from the latent feature distribution; a causal feature decoupling module, configured to establish a graph attention network, and input the causal features into the graph attention network to transmit causal effects during decoupling; The model training module is configured to design a loss function, use a variational autoencoder, a discriminator, a causal structure model, and a graph attention network as a causal representation model, and use the loss function to train the causal representation model. 10 . A non-transitory computer-readable storage medium storing instructions, which, when executed by a processor, perform the method according to claim 1 .

Images