A Method and System for Analyzing Dynamic Facial Features of Mental States Based on Causal Graph Learning

Abstract

This invention discloses a method and system for dynamic facial feature analysis of psychological states based on causal graph learning, comprising the following steps: acquiring facial videos of the target object, extracting facial image frames and forming a facial image sequence; locating and aligning facial key points in each facial image frame of the facial image sequence, and outputting a standardized facial image sequence and key point coordinate sequence; constructing a dynamic facial image based on the standardized facial image sequence and key point coordinate sequence; inputting the dynamic facial image into a trained causal graph learning model, and outputting the psychological state classification result of the target object; the causal graph learning model is trained based on a training dataset composed of multiple dynamic facial image samples, using a five-fold cross-validation strategy, and during the training process, the causal feature decoupling module and classifier integrated in the model are jointly optimized with the goal of minimizing the combined loss function.

Description

Technical Field

[0001] This invention relates to the field of intelligent medical technology, and more specifically, to a method and system for dynamic facial feature analysis of psychological states based on causal graph learning. Background Technology

[0002] With the rapid development of computer vision and deep learning technologies, automated analysis technology based on facial dynamic features has become a research hotspot in the field of intelligent healthcare.

[0003] Existing technologies mainly rely on deep learning models, such as using convolutional neural networks (CNNs) to extract static image features or using recurrent neural networks (RNNs) to process time series data. However, such methods have the following technical defects: (1) Insufficient capture of dynamic relationships of local features: Existing technologies usually perform global feature extraction on the entire facial image, ignoring the dynamic changes and interrelationships of local facial regions (such as eyes, mouth, etc.), resulting in insufficient sensitivity to subtle dynamic features; (2) Confusion between interference features and causal features: Existing deep learning models are easily affected by irrelevant interference features (such as changes in illumination, background noise, and individual physiological differences) during training, resulting in a lack of causal correlation in the learned features, reducing the robustness and interpretability of the analysis; (3) Inefficient processing of graph structure data: Traditional models have difficulty effectively modeling the topological relationships of local facial regions.

[0004] In the prior art, such as Chinese patent CN120340098A, a method for constructing dynamic network sequences of facial features is proposed. However, its nodes are abstract feature values ​​(such as action unit intensity and geometric coordinates) rather than concrete facial anatomical regions, and its edge weights are based on linearly correlated Pearson coefficients, making it difficult to effectively capture nonlinear and anomalous dynamic temporal correlations.

[0005] As can be seen from the above, the relevant technologies have failed to effectively solve the problems of capturing the dynamic temporal correlation of local facial regions and suppressing interference features. Summary of the Invention

[0006] To address the aforementioned problems in existing technologies, this invention provides a method and system for analyzing dynamic facial features of psychological states based on causal graph learning. This method can significantly improve the accuracy of capturing subtle facial expression changes, enhance the stability of analysis under interference environments, and optimize the representation of dynamic interaction relationships.

[0007] The objective of this invention is achieved through the following technical solutions.

[0008] The summary section of this application is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0009] Some embodiments of this application propose a method and system for dynamic facial feature analysis of psychological states based on causal graph learning, in order to solve the technical problems mentioned in the background section above.

[0010] As a first aspect of this application, some embodiments of this application provide a method for dynamic facial feature analysis of psychological states based on causal graph learning, including the following steps:

[0011] Step 1: Acquire facial video of the target object, extract facial image frames, and form a facial image sequence; Step 2: Perform facial key point localization and frame alignment on each facial image frame in the facial image sequence, and output a standardized facial image sequence and key point coordinate sequence. Step 3: Construct a dynamic face map based on the standardized facial image sequence and key point coordinate sequence; the dynamic face map includes a node feature matrix for representing the temporal dynamics of the appearance of different anatomical regions of the face and a weighted adjacency matrix for representing the asynchronous motion cooperative relationship between different anatomical regions of the face. Step 4: Input the dynamic facial image into the trained causal graph learning model and output the psychological state classification result of the target object; The causal graph learning model is trained based on a training dataset composed of multiple dynamic facial image samples, using a five-fold cross-validation strategy. During the training process, the causal feature decoupling module and classifier integrated in the model are jointly optimized with the goal of minimizing the combined loss function. The causal feature decoupling module is used to decouple causal features and interference features from the input dynamic face image, and the classifier module outputs the predicted classification result based on the causal features.

[0012] Furthermore, step two involves facial key point localization and frame alignment, specifically including: Facial key points were located in each facial image frame using a standard of 68 facial key points. The outer corners of the left and right eyes and the tip of the nose were selected as reference points from facial key points, and a weight proportional to the detection confidence was assigned to each reference point to construct a weighted diagonal matrix. ; Based on the benchmark point and weight diagonal matrix The affine transformation matrix that aligns the current frame to the reference frame is calculated using the weighted least squares method. The calculation formula is as follows: ; Let be the affine transformation matrix. This is the homogeneous coordinate matrix of the reference point in the current frame. This is the coordinate matrix of the corresponding point in the reference frame. This is a weighted diagonal matrix; The key point coordinate sequence is based on the affine transformation matrix Perform synchronized updates; Based on affine transformation matrix After updating all facial image frames and facial keypoint coordinates, output a standardized facial image sequence and keypoint coordinate sequence.

[0013] Furthermore, step three specifically includes: Based on the facial motion coding system, multiple regions of interest are defined as nodes of the dynamic facial map. The geometric boundary of each region is determined by the coordinates of the associated facial key points, the preset offset, the width, and the height. The depth features of each node in each frame of the facial image sequence are extracted, and after dimensionality reduction, they are concatenated in time to form the dynamic feature vector of the node; the dynamic feature vectors of all nodes constitute a node feature matrix used to characterize the temporal dynamics of the appearance of different anatomical regions of the face. Based on prior knowledge of facial anatomy, the connection relationships between nodes are defined. For each pair of nodes that are connected, the dynamic time warping algorithm is used to calculate the distance between the time series corresponding to their dynamic feature vectors, and the obtained dynamic time warping distance is converted into edge weights representing the degree of similarity. All edge weights constitute a weighted adjacency matrix to characterize the asynchronous motion cooperative relationship between different anatomical regions of the face.

[0014] Furthermore, the extraction of deep features specifically includes: Using a ResNet-50 convolutional neural network pre-trained on the VGGFace2 dataset, the initial depth features of each node in the image region of each frame are extracted. The initial deep features are reduced to 10 dimensions using a trainable fully connected layer; The dimensionality-reduced features of each node across all frames of the image sequence are concatenated in time to form the final dynamic feature vector of that node.

[0015] Furthermore, the dynamic time-warped distance is converted into edge weights, specifically using a Gaussian kernel function. The edge weight calculation formula is as follows: ; in, and For node indexing, For the first Each node and Edge weights between them The normalized DTW distance. This is the scale parameter.

[0016] Furthermore, the causal feature decoupling module includes a mask generator and a bi-branch graph convolutional network; The mask generator is configured to perform graph convolution operation on the node feature matrix of the dynamic face map with a graph convolution layer, and generate causal node masks, causal edge masks, and interference node masks and interference edge masks that are complementary to the causal node masks and causal edge masks, respectively, based on the node hidden features output after the operation. The bi-branch graph convolutional network is configured to use causal node masks and causal edge masks, as well as interference node masks and interference edge masks, to weight the input node feature matrix and weighted adjacency matrix respectively, and then perform graph convolution operations to output causal hidden features and interference hidden features; the causal hidden features are input into the classifier.

[0017] Furthermore, the mask generator is a multilayer perceptron based on an attention mechanism, where the generation formulas for causal node masks and causal edge masks are as follows: ; ; in, and For node indexing, For the first Each node's causal node mask To connect the first The node and the first Causal edge mask of edges of each node. For the first Hidden features of each node, It represents the Hadamah accumulation. Indicates passage And the normalization operation implemented by the Softmax function.

[0018] Furthermore, the combined loss function includes supervised classification loss. Interference loss and causal intervention loss , is represented as: ; in, , is a hyperparameter; supervised classification loss is the cross-entropy loss between the predicted output of the causal path and the true label; interference loss is the KL divergence loss between the predicted output of the interference path and the uniform distribution; causal intervention loss is the classification loss after fusing the causal features of the current sample with the interference features of other samples.

[0019] As a second aspect of this application, some embodiments of this application provide a dynamic facial feature analysis system for psychological states based on causal graph learning, for implementing the above-mentioned method, including: The data acquisition module is used to acquire facial videos of the target object, extract facial image frames, and form a facial image sequence; The preprocessing module is used to locate facial key points and align frames in each facial image frame of the facial image sequence, and output a standardized facial image sequence and key point coordinate sequence. The dynamic face image construction module is used to construct dynamic face images based on standardized facial image sequences and key point coordinate sequences. The analysis module is used to input dynamic facial images into a trained causal graph learning model and output the psychological state classification results of the target object.

[0020] As a third aspect of this application, some embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the above-described method.

[0021] Compared with the prior art, the advantages of this invention are: (1) This invention explicitly represents the dynamic relationship between different anatomical regions of the face by constructing a dynamic facial map. It uses its node feature matrix to encode the temporal dynamics of the appearance of each region and the weighted adjacency matrix to encode the collaborative relationship of asynchronous motion between regions. Then, the dynamic facial map is input into the causal graph learning model for analysis, which solves the problems of insufficient capture of local feature dynamic relationships and inefficient graph structure data processing. It can more accurately represent the dynamic interaction relationship of subtle facial expressions. (2) In the data preprocessing stage, the present invention uses affine transformation based on weighted least squares to perform frame alignment. By selecting robust key points such as the outer corners of the left and right eyes and the tip of the nose as benchmarks, and assigning weights according to their detection confidence, the standard deviation of pixel positions in the same region of interest (ROI) in the aligned image sequence is significantly reduced, thereby reducing the noise introduced by the small rigid movements of the head. (3) This invention uses the Dynamic Time Warping (DTW) algorithm to calculate the minimum cumulative distance between the feature sequences of two connected nodes in the dynamic graph, and maps this distance to edge weights through the Gaussian kernel function. This effectively quantifies the nonlinear and heterogeneous temporal coordination between different regions of the face, and solves the problem that traditional methods (such as linear correlation based on Pearson coefficient) are difficult to capture asynchronous dynamic associations. (4) This invention designs a causal feature decoupling module that includes a mask generator and a dual-branch graph convolutional network, and uses a combined loss function consisting of supervised classification loss, interference loss and causal intervention loss for joint optimization, so that the model decouples causal features from interference features (such as illumination and individual differences), and finally classifies based only on causal features, thus solving the problem that existing deep learning models are easily affected by irrelevant interference features and that features lack causal correlation. Attached Figure Description

[0022] Figure 1 This is a flowchart of data preprocessing in one embodiment of the present invention; Figure 2 This is a schematic diagram of ROI partitioning in one embodiment of the present invention; Figure 3 This is a schematic diagram of the construction process of a dynamic facial graph (DFGraph) in one embodiment of the present invention; Figure 4 This is a schematic diagram of the overall architecture of a causal graph learning model in one embodiment of the present invention; Figure 5 This is a schematic diagram of a heatmap of causal interest regions generated when outputting classification results in one embodiment of the present invention. Detailed Implementation

[0023] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0024] The present invention provides a method for dynamic facial feature analysis of psychological states based on causal graph learning, comprising the following steps: S1. Data Acquisition and Preprocessing like Figure 1 As shown, this step aims to acquire high-quality, standardized facial image sequences and their corresponding geometric feature data through data acquisition and preprocessing, and to reduce noise interference introduced by environmental, device and individual behavioral differences.

[0025] S101. Facial video data acquisition: Use a high-definition camera device with a resolution of at least 1080p (1920×1080 pixels) (such as Logitech C920 or Huawei software-defined camera) to acquire facial video data of the target object, extract facial image frames and form a facial image sequence.

[0026] In one specific embodiment, a 4K (3840×2160 pixels) resolution camera was used with a frame rate set to 30fps to ensure that dynamic features related to micro-expressions, eye muscle activity, and subtle changes in the corners of the mouth could be clearly captured.

[0027] The capture time is 30 to 375 seconds to ensure that enough video data containing complete facial expressions and dynamics can be captured.

[0028] To ensure the standardization and consistency of facial video data during the acquisition process, constraints regarding frontal viewpoint, horizontal posture, and shooting distance must be met, including: For a frontal view, the angle between the center line of the subject's face (a virtual line formed by the center points of the eyes and the tip of the nose) and the camera's optical axis should be less than 15 degrees. The horizontal orientation requirement is that the deviation between the horizontal line connecting the eyes of the target object and the horizontal axis (x-axis) of the pixel coordinate system of the acquired video image frame should be less than 5 degrees. The shooting distance requirement is that the distance between the target object and the camera should be maintained between 0.5 meters and 1.0 meter to ensure that the size ratio of the face in the picture is relatively stable.

[0029] In one specific embodiment, the aforementioned angles and distances during the acquisition process can be quantified in real time using a face pose estimation algorithm (such as MediaPipe Face Mesh) to guide the target object to make adjustments and to filter video clips that meet the standards.

[0030] Based on the collected facial video data, facial image frames are extracted through equal-interval sampling and formed into an ordered facial image sequence containing multiple frames of facial images.

[0031] The number of facial image frames is determined by an optimal trade-off between the minimum duration required to capture complete micro-expression dynamics and the computational complexity of subsequent models: if the number of frames is too small (less than 100 frames), it will not be able to cover statistically significant dynamic patterns, while if the number of frames is too large, it will introduce redundant information and significantly increase the computational burden.

[0032] Specifically, 200 frames of facial images are extracted from the collected facial video data at equal intervals to form a facial image sequence.

[0033] S102, Keyframe Alignment: First, the MTCNN algorithm is used to process the facial image sequence frame by frame: The MTCNN algorithm first performs face detection and outputs a face bounding box that defines the facial region, thereby achieving the initial localization of the face; at the same time, within the detected face region, the MTCNN algorithm will further regress 5 initial facial key points, including the pixel coordinates of the center of the eyes, the tip of the nose, and the corners of the mouth.

[0034] After the initial face localization is completed using the MTCNN algorithm, a key point detection model (such as the dlibshape_predictor_68_face_landmarks model based on traditional machine learning or a shape regression network based on deep learning) is used to perform sub-pixel-level feature localization within the localized face region, and finally outputs the precise two-dimensional pixel coordinates of 68 facial key points that conform to the iBUG 300-W international standard.

[0035] These 68 key points are not randomly distributed, but systematically define the core anatomical structures of the face, including the jawline (points 1-17), left eyebrow (points 18-22), right eyebrow (points 23-27), nose (points 28-36), left eye (points 37-42), right eye (points 43-48), and the inner and outer contours of the mouth (points 49-68). Obtaining this set of standardized, high-precision two-dimensional pixel coordinates is used for inter-frame affine transformation alignment in subsequent steps to eliminate head motion, and also provides indispensable anchor points for accurately dividing the 23 regions of interest (ROIs) in step S2.

[0036] In this embodiment, the dlibshape_predictor_68_face_landmarks model is used as the key point detection model. In challenging scenarios such as complex lighting and occlusion, the depth shape regression network can be switched to ensure positioning accuracy.

[0037] To eliminate inter-frame misalignment caused by minute rigid head movements (such as translation and rotation), frame alignment is performed using an affine transformation based on weighted least squares.

[0038] First, select a frame from the facial image sequence as a reference frame. .

[0039] Specifically, reference frame The median frame of the time series can be selected, or the frame with the closest pose and the highest average confidence can be selected by calculating the pose angle and the average confidence of key points in each frame.

[0040] Iterate through each non-reference frame in the facial image sequence and denote it as the current frame. .in, The frame number is... Not a reference frame ( And perform the following operations: (1) Selecting a reference point: Select the current frame A set of facial key points that are stable for head movements are used as reference points.

[0041] Specifically, select the current frame The three spatially stable facial key points—the outer corner of the left eye, the outer corner of the right eye, and the tip of the nose—are used as reference points.

[0042] (2) Construct the homogeneous coordinate matrix: Current frame The pixel coordinates of the three reference points are converted into homogeneous coordinates to form a 3×3 matrix. Similarly, the reference frame The coordinates of the three reference points are converted into homogeneous coordinates to form a 3×2 matrix. .

[0043] Based on the confidence scores output by the keypoint detection model, weights are assigned to the three baseline points: (The higher the confidence level, the greater the weight), construct a 3×3 diagonal weight matrix. .

[0044] (3) Solve for the affine transformation matrix: By solving a weighted least squares problem, the affine transformation matrix that minimizes the alignment error is calculated; that is, the optimal affine transformation matrix from the current frame to the reference frame. : ; in, Denotes the Frobenius norm; This represents the set of all affine transformation matrices. Specifically, this weighted least squares problem has a closed-form solution, which can be solved directly through matrix operations, as follows: ; In the formula: and These represent the homogeneous coordinate matrices of three reference points in the current frame and the reference frame, respectively. yes The transpose of the matrix, It is a diagonal matrix of weights. These represent the confidence weights for the three benchmark points; It is the square of the Frobenius norm, used to measure the total squared error between the transformed coordinates and the target coordinates.

[0045] Affine transformation matrix It is a 2×3 matrix containing a 2×2 rotation and scaling submatrix and a 2×1 translation vector, used to describe the rigid body transformation relationship from the current frame to the reference frame coordinate system. The weight allocation is proportional to the keypoint detection confidence.

[0046] (4) Perform image transformation: Using the obtained affine transformation matrix For the current frame A bilinear interpolation geometric transformation is performed to align the facial key points with the reference frame. This process results in an average reprojection error of less than 2 pixels for the aligned reference points in the current frame, and a significant reduction in the standard deviation of pixel positions of the same region of interest (ROI) in the facial image sequence (e.g., from 15.3 pixels to 2.1 pixels), greatly improving the stability of temporal analysis.

[0047] S103, Image Denoising and Normalization Output: On the aligned facial image sequence, a Gaussian filter with a kernel size of (3,3) is applied for mild spatial smoothing to suppress uneven lighting, sensor noise and image compression artifacts, while preserving the subtle texture details of the face to the maximum extent.

[0048] The final output consists of a standardized facial image sequence and a keypoint coordinate sequence. The facial image sequence contains an array of 200 aligned and denoised grayscale or RGB facial images; the keypoint coordinate sequence is a 200×68×2 array corresponding to the facial image sequence, recording the pixel coordinates of 68 keypoints in each frame of the facial image sequence.

[0049] S2, Construction of Dynamic Facial Image like Figure 3 As shown, this step aims to transform the standardized facial image sequence and keypoint coordinate sequence output from step S1 into graph-structured data that can explicitly represent the dynamic relationships between different anatomical regions of the face. First, the boundaries of each region of interest (ROI) are determined based on facial keypoints; then, node features are extracted and edge weights are calculated, ultimately constructing a dynamic facial graph (DFGraph).

[0050] S201, Define graph nodes: Based on the Facial Action Coding System (FACS), 23 pre-defined Regions of Interest (ROIs) are defined as 23 nodes in the dynamic facial map. These ROIs include all key anatomical units such as the forehead, eyebrows, eyes, cheeks, nose, mouth, and chin, ensuring that no core muscle groups related to emotional expression are missed.

[0051] like Figure 2 The diagram shown is a schematic diagram of ROI division in this embodiment, illustrating the geometric definition and location distribution of 23 regions of interest (ROIs) constructed based on 68 facial key points.

[0052] Specifically, each ROI is precisely defined by its rectangular geometric boundaries. As shown in Table 1, this definition is based on an "anchor point plus offset" strategy: each ROI is associated with one or more specific points from 68 facial key points as positioning anchor points. Based on the anchor point coordinates (or their geometric centers, if there are multiple anchor points), a preset two-dimensional offset (Δx, Δy) is added to determine the center position of the ROI, and then combined with preset width W and height H to determine the boundary of the ROI.

[0053] In this invention, each preset region of interest (ROI) corresponds to a node in the dynamic facial image. Therefore, the index of the ROI... ( = 1, 2, ..., 23) is the index of the graph node corresponding to the ROI.

[0054] For each frame in the facial image sequence, the first... The boundaries of each ROI are calculated using the following formula: ; In the formula: ( , ) indicates that the image in this frame is related to the first digit. The pixel coordinates of the location anchor points (or the average of multiple location anchor points) associated with each ROI; ( , ) is the first Preset center offset for each ROI; and The first The width and height of each ROI are preset; ( , ) is the calculated first The center coordinates of the rectangles of each ROI; Defined the first A rectangular pixel region of a ROI.

[0055] Table 1 shows the preset values ​​for the anchor point index, 2D offset, and width and height of the ROI in the example. Through the above steps, the rectangular boundaries of the 23 ROIs can be accurately and consistently determined in each frame of the facial image.

[0056] For a sequence of 200 facial images, a set of 23×200 rectangular image blocks with strictly corresponding spatial locations is formed, providing a stable and structured input for subsequent feature extraction.

[0057] Table 1 Example of ROI definition

[0058] S202, Extract node features After converting the facial image sequence into a sequence of 23 Regions of Interest (ROIs), a mathematical expression, i.e., node features, that accurately describes the temporal changes of each ROI needs to be extracted. The specific process is as follows: (1) Basic feature extraction: A ResNet-50 deep convolutional neural network pre-trained on the VGGFace2 face recognition dataset was used as the basic feature extractor, and the final classification layer was removed. The ResNet-50 deep convolutional neural network consists of one 7×7 convolutional layer, 16 Bottleneck residual blocks (each residual block contains a combination of 1×1, 3×3, and 1×1 convolutions) and one global average pooling layer.

[0059] For the input of the first ROI (i.e., nodes) ) in the frame( Given an image patch (i = 1, 2, ..., 200), the ResNet-50 deep convolutional neural network extracts the initial 2048-dimensional depth feature vector of the image patch through a global average pooling layer.

[0060] (2) Feature dimensionality reduction and frame-level representation: The ResNet-50 deep convolutional neural network reduces the dimensionality of the initial 2048-dimensional deep feature vector to a 10-dimensional feature vector through a trainable fully connected layer. This 10-dimensional feature vector encodes the first... The visual state of each ROI in the current frame.

[0061] (3) Temporal feature splicing: The first The 10-dimensional feature vectors of each ROI are concatenated strictly in chronological order across all 200 frames of facial images to form a 2000-dimensional composite feature vector, which serves as the first ROI. The feature vectors of each graph node fully encode the dynamic change pattern of the facial region throughout the video clip.

[0062] (4) Construct the node feature matrix: The feature vectors of all 23 nodes are stacked vertically to form the node features of the entire dynamic face image, which are represented as the node feature matrix X.

[0063] S203, Constructing Edges and Weights This step aims to define and quantify the dynamic relationships between nodes (i.e., ROIs). These relationships are represented in the dynamic face graph as edges connecting nodes, with the strength of the association determined by edge weights. In this invention, edge weights are calculated through the following process, and their values ​​quantify the temporal similarity of dynamic patterns between two facial regions: (1) Define the connection relationship: This embodiment is based on facial anatomy and function priors and employs a sparse connectivity strategy: an edge is established between two ROIs only when they are anatomically adjacent or functionally closely related (e.g., both belonging to the eye region). As shown in the "Adjacent Regions" column of Table 1, the connectivity relationships are defined according to the preset adjacent regions and represented as a binary adjacency matrix. , of which matrix elements The definition is as follows: ; In the formula: and All are node indexes. These represent the first and second parts of the animation. The first node and the second The feature vector of each node Represents the set of edges.

[0064] =1 indicates the first The node and the first There are edges between the nodes. =0 indicates the first The node and the first There are no edges between the nodes.

[0065] (2) Calculate the dynamic association weight: Each edge is assigned a weight to quantify the temporal synergy of dynamic features between two connected ROIs.

[0066] Specifically, considering the inherent asynchronicity of facial muscle movements (i.e., different regions have different movement rhythms), this embodiment uses the Dynamic Time Warping (DTW) algorithm to calculate edge weights. The smaller the DTW distance, the more similar the muscle movement patterns of the two regions. The specific process is as follows: For any two connected nodes in the dynamic graph, reshape their feature vectors into 200 consecutive 10-dimensional feature vectors.

[0067] In a specific embodiment, the first Feature vectors of each node Converted into 200 consecutive 10-dimensional vectors, forming a time series: Each element Corresponding to the The node at the th The 10-dimensional feature vector of the frame. Similarly, the 10-dimensional feature vector of the first frame. Feature vectors of each node Converted into 200 consecutive 10-dimensional vectors, forming a time series: Each element Corresponding to the The node at the th The 10-dimensional feature vector of the frame.

[0068] Calculate sequence and Minimum cumulative distance between : ; In the formula: For a regular path that satisfies the constraints of monotonicity and continuity, It is a Euclidean distance metric.

[0069] To reduce computational complexity, this embodiment employs slope constraints (e.g., 0.5 to 2.0) and window constraints (e.g., ... This optimizes the search space and improves efficiency through GPU parallel acceleration.

[0070] To transform the DTW distance into a measure of similarity and ensure its value falls within a suitable range (e.g., between 0 and 1), a Gaussian kernel function is used to perform a non-linear mapping on the DTW distance, resulting in an output range for similarity that is normalized to... Interval weight values Gaussian kernel mapping ensures that the smaller the distance (higher the synergy), the larger the weight. Weight values. The calculation formula is: ; In the formula: Indicates the first Each node and The edge weights between them are the normalized DTW distances; For the scale parameter, in this embodiment 0.1. The larger the value, the higher the similarity.

[0071] (3) Determine the edge weights and construct the matrix: Calculated similarity weight values That is, defined as a node and The edge weights between all node pairs are combined to form an edge weight matrix. .

[0072] edge weight matrix With binary adjacency matrix Perform the Hadamard product (i.e., element-wise multiplication) to obtain the final weighted adjacency matrix. : ; That is, when hour, ;when hour, . Weighted adjacency matrix The non-zero elements.

[0073] The weighted adjacency matrix is ​​obtained through the above steps. It precisely quantifies the dynamic coordination relationship between different facial regions over time. The larger the value, the more likely the node is to be affected. and The higher the synchronization of muscle movement patterns.

[0074] At this point, a complete Dynamic Face Graph (DFGraph) data structure has been constructed, which can be mathematically represented as a pair of tuples. .in, The node feature matrix is ​​a matrix in which each row corresponds to the feature vector of a ROI node, encoding the temporal dynamics of the appearance of each facial region. The weighted adjacency matrix encodes the dynamic cooperative relationships of asynchronous motion between regions.

[0075] The resulting dynamic facial image Its data structure has a clear physical meaning. Node feature matrix Each row corresponds to a node of a facial anatomical region of interest (ROI), and the vector in that row is formed by sequentially concatenating the visual features of that region extracted from 200 consecutive frames of images. This method of sequentially concatenating high-dimensional feature vectors allows for... Each line fully encodes the temporal evolution of the appearance (such as texture and shape) of that specific anatomical region throughout the video clip, i.e., the temporal dynamics of appearance.

[0076] Weighted adjacency matrix non-zero elements Indicates the connection of the first Each node and The edge weights are calculated using the Dynamic Time Warping (DTW) algorithm, which calculates the similarity between the temporal feature sequences of two nodes (i.e., two facial regions). The core advantage of the DTW algorithm lies in its ability to non-linearly align two time series that may have different speeds (i.e., asynchronous), thereby calculating the most closely matched similarity metric between them. Therefore, The value precisely quantifies the facial first The region and the first The degree of coordination of muscle movement patterns between different regions is quantified, and this quantification fully considers the asynchronicity of possible different movement rhythms, thus characterizing the asynchronous movement coordination relationship.

[0077] Based on the above steps, several pairs are finally output. .

[0078] Dynamic facial maps constructed based on each target object As a data sample, it forms a training dataset, which serves as the input for subsequent causal graph learning models.

[0079] S3. Training and Optimization of Cause-and-Effect Graph Learning Model This step aims to design and train a graph neural network model with causal decoupling capabilities—the causal graph learning model—based on the training dataset constructed in step S2. The goal of the causal graph learning model is to automatically separate causal features that have a direct causal relationship with psychological states from the input dynamic facial images, while identifying and suppressing interfering features from the environment, individual differences, etc.

[0080] S301, Model Architecture Design like Figure 4 As shown, the causal graph learning model consists of a causal feature decoupling module and a classifier module connected in series.

[0081] Specifically, the causal feature decoupling module comprises a mask generator and a two-branch graph convolutional network. The causal feature decoupling module receives a dynamic face map sample. As input, it outputs the decoupled causal features and interference features.

[0082] Mask generator: The mask generator employs an attention-based multilayer perceptron (MLP) to generate a pair of complementary, learnable soft masks for each node and edge in the dynamic face image. The specific process is as follows: First, for an input dynamic face image sample, the hidden feature representation matrix of the nodes is obtained through a graph convolutional layer (GCN). . Each row corresponds to a feature vector of a node.

[0083] Then, the mask generator uses the node hiding features. Compute node mask and edge mask Its mathematical expression is as follows: ; ; In the formula: and They are the first The node and the first Hidden features of each node, This represents the Hadamard product (element-by-element multiplication). Indicates passage and the normalization operation implemented by the Softmax function; It is an edge mask matrix The element in represents the first element. The node and the first Causal importance weights of connections between nodes It is a node mask matrix The element in represents the first element. The node and the first Causal importance weights between nodes.

[0084] Specifically, node mask and edge mask These are all causal masks, and their corresponding interference masks are respectively and These masks are weight matrices with values ​​between 0 and 1, used to perform weighted filtering of node features and edge weights, respectively. These masks satisfy the following complementary relationship: ; ; Two-branch graph convolutional network: A two-branch graph convolutional network consists of two parallel graph convolutional layers (GCNs) to form causal feature paths and interference feature paths, which process graph data modified by different masks respectively.

[0085] The total number of layers in the bibranch graph convolutional network is ( (where is a positive integer), let the layer index of the bibranch graph convolutional network be . ( = 1, 2, ..., ), and The initial input is the node feature matrix. The working process of the two paths is as follows: Causal feature path: Multiply the node features element-wise with the causal node mask, and multiply the weighted adjacency matrix element-wise with the causal edge mask. Then, input the processed graph data into a graph convolutional layer (denoted as ). Learning to obtain causal hidden features .

[0086] Specifically, in the The layer multiplies the causal hidden features output from the previous layer with the causal node mask, and simultaneously multiplies the weighted adjacency matrix with the causal edge mask. The processed graph data is then input into the next layer. One GCN layer (denoted as ) Output the causal hidden features of the current layer. .

[0087] Interference Feature Path: Multiply the node features element-wise with the interference node mask, and simultaneously apply the weighted adjacency matrix... The edges are multiplied element-wise with the interference edge mask, and then fed into another separate graph convolutional layer (denoted as ). Learning to obtain interfering hidden features .

[0088] Specifically, in the The layer multiplies the interference hiding features output from the previous layer with the interference node mask, and simultaneously multiplies the weighted adjacency matrix with the interference edge mask. The processed graph data is then input into the next layer. One GCN layer (denoted as ) Output the obstruction hidden features of the current layer. .

[0089] The mathematical expression for the two pathways is: ; ; In the formula: It is a weighted adjacency matrix; It is a causal node mask. It is a causal edge mask; It is a mask for interfering nodes; It is an interference edge mask; This represents the Hadamard product. Through stacking multiple layers, the model can progressively extract highly decoupled feature representations. Finally, the last layer (the...) is output. Causal Hidden Features of Layers and interference to hide features .

[0090] Specifically, the classifier module is a multilayer perceptron (MLP), whose input is the last layer (the first layer) of the causal feature decoupling module. Causal Hidden Features of Layers This ensures that the final decision is based solely on causal factors. The workflow is as follows: Feature aggregation: First, since the causal hidden features output by the causal graph learning module are a node-level feature matrix containing the final feature representation of each ROI in the dynamic face map, in order to classify the entire dynamic face map, a readout function (R) is used to aggregate the node-level causal hidden feature matrix of the last layer into a graph-level feature vector.

[0091] In one specific implementation, global average pooling is used as the read function, that is, for The feature vectors of all 23 nodes are averaged element-wise to obtain a 1×512-dimensional graph-level causal feature vector.

[0092] Classification prediction: Subsequently, the graph-level causal feature vector is input into an MLP classifier Φ, which consists of several fully connected layers and a non-linear activation function (such as ReLU). The last layer of the classifier uses the Softmax activation function and outputs a probability distribution vector. : ; In the formula: The causal branch is at the last level (the first) The node feature representation of the layer.

[0093] The dimensions correspond to different categories of mental states (e.g., four categories in the analysis of depressive states), and each element represents the predicted probability that the input sample belongs to the corresponding category.

[0094] S302, Design of Combination Loss Function To guide the model to effectively decouple features and accurately classify them, this invention designs a combined loss function. Joint optimization was carried out.

[0095] ; in, and It is a hyperparameter used to balance the weights of various losses.

[0096] Cross-entropy loss is applied to the predicted output of causal pathways. By minimizing cross-entropy loss The model can learn more accurate classification features, thus improving classification accuracy. The process can be represented as follows: ; In the formula: This represents the training dataset, which contains multiple graph samples. ; Image sample The true label; It is the output of the classifier module, representing the predicted probability distribution; It is the logarithm of the predicted probability.

[0097] It is the interference loss, which affects the predicted output of the interference path. This forces its predictions to tend towards a uniform distribution. This is achieved by minimizing the disturbance loss. The model ensures that the predicted interfering features are approximately uniformly distributed, thereby reducing their interference with the classification results. The process is expressed as follows: ; In the formula: This represents the Kullback-Leibler divergence, used to measure the difference between two probability distributions; A uniform distribution means that the probability is equal for all categories. It is the output of the interference feature module, represented as the predicted probability distribution.

[0098] To achieve backdoor adjustment in causal inference, this invention introduces causal intervention loss. The calculation of causal intervention loss is based on a hierarchical sampling strategy, which enhances the stability of causal features through simulated backdoor adjustment. The causal features of the current sample are fused with interfering features sampled from other samples before classification, and the loss is calculated.

[0099] Specifically, for a dynamic facial image sample First, calculate the predicted results after intervention. The process is represented as follows: ; In the formula: It is a sample The last layer of causal hidden features; From the sample Constructed hierarchical interference feature set A random sample of a hidden perturbation feature (usually composed of hidden perturbation features from other samples in the same training batch); · The readout function (global average pooling) is used to aggregate node features into graph-level features; This represents the classifier module (i.e., the multilayer perceptron MLP), which is used to map features to a probability distribution. This represents the fusion of the causal core of the current sample with the interfering background features to simulate different environmental conditions. The formula simulates the fusion of the causal features of the current sample with the interfering features of others to simulate different environmental conditions.

[0100] Subsequently, the causal intervention loss is calculated based on this prediction result, and the process is expressed as follows:

[0101] In the formula: This represents the training dataset, which contains multiple dynamic facial image samples to be analyzed. ; This is a sample The constructed hierarchical set of interference features; Indicates the hierarchical interference set Selected for the current sample Interference samples for causal intervention; It is a dynamic facial image sample The transpose of the real label; This is the predicted result after intervention.

[0102] S303, Model Training and Optimization Strategies This step trains the causal graph learning model using the training dataset and guides model optimization through a combined loss function. The training process aims to learn all learnable parameters of the model, including the weights of the graph convolutional layers, the weights of the multilayer perceptron in the mask generator, and the weights of the classifier, to achieve effective decoupling of causal features from perturbation features and accurate classification of the final state. The specific training process is as follows: Data partitioning: A 5-fold cross-validation strategy is employed to robustly and objectively evaluate the model's performance. First, all dynamic facial image data samples in the training dataset are randomly shuffled and uniformly divided into five disjoint subsets, each containing approximately 20% of the samples. The proportion of samples from each category in each subset is ensured to be largely consistent with the overall distribution of the original dataset, thus maintaining the representativeness of the data distribution.

[0103] Training and evaluation cycle: Conduct five independent training and evaluation rounds. In each round, perform the following: One of the five subsets is selected as the test set to ultimately evaluate the model's generalization ability on unknown data. The remaining four subsets are used as the initial training set.

[0104] Model initialization: Before each round of training, all parameters to be learned in the causal graph learning model are randomly initialized. These parameters include, but are not limited to, those in the bi-branch graph convolutional network. and The weight matrices and bias terms of each layer, the weights of the multilayer perceptron in the mask generator, and the classifier Weights in a multilayer perceptron.

[0105] Iterative training process: For each training set, the maximum number of training epochs is set to 200. Within each training cycle, the following operations are performed: (1) Forward propagation: Dynamic facial image samples (including node feature matrices and weighted adjacency matrices) from the actual training set are input in batches into the causal graph learning model. The model passes through a mask generator and a two-branch graph convolutional network in sequence, and finally the classifier outputs the predicted probability distribution based on causal features. Simultaneously, the model also calculates and outputs predictions based on interference features. And intervention prediction for calculating causal intervention losses .

[0106] In one specific embodiment, firstly, a mask generator generates a causal mask and an interference mask based on the input data. Then, a dual-branch graph convolutional network uses these two sets of masks to convolve and update the graph data, and aggregates them through a readout function to obtain graph-level causal feature vectors and interference feature vectors. The extracted causal feature vectors are input into a classifier to output a causal classification prediction probability distribution, used to calculate the classification loss; the extracted interference feature vectors are output as an interference prediction probability distribution, used to calculate the uniform distribution loss. Based on a backdoor adjustment strategy, the causal features of the current sample are fused with interference features from other samples randomly sampled from the current batch or feature library (e.g., feature addition) to obtain a synthetic feature. This synthetic feature is input into a classifier, outputting an intervention-predicted probability distribution, used to calculate the causal intervention loss.

[0107] (2) Loss calculation: Based on the defined combined loss function, calculate the supervised classification loss, interference suppression loss and causal intervention loss respectively, and apply them according to the preset hyperparameters. and The weighted sum is used to obtain the total loss for the current batch.

[0108] (3) Backpropagation and parameter update: Using the Adam optimizer, the gradient of the total loss with respect to all trainable parameters of the model is calculated through the backpropagation algorithm. Then, the model parameters are iteratively updated according to the optimizer's update rule (combining the first and second moments of the gradient) and the preset learning rate to minimize the total loss.

[0109] Verification and early cessation: After each training epoch, the model is switched to evaluation mode (without parameter updates) and forward propagation is performed using the validation set of the current epoch. Performance metrics such as classification accuracy and loss on the validation set are calculated and compared to the historical best validation performance. If the validation set performance no longer improves over several consecutive training epochs (e.g., with a patience value of 20 epochs), an early stopping mechanism is triggered, prematurely terminating the training process of the current epoch and rolling back to the model parameter state with the best validation performance. This effectively prevents the model from overfitting on the training set.

[0110] Test evaluation: After all five-fold cross-validation training is completed, the optimal model parameters saved during each training round are loaded. A one-time forward propagation test is then performed on the model using the test set corresponding to each round, which has not been seen during the entire training process. Evaluation metrics (such as accuracy, precision, recall, and F1 score) on the test set for each round are recorded, and the average of these five test results is calculated as the objective and final evaluation metric for the overall performance of the method of this invention, thereby measuring the model's generalization ability and stability.

[0111] S4, Psychological State Analysis Output This step aims to use the causal graph learning model with causal decoupling capability trained in step S3 to perform fully automatic analysis on the facial video data of the new target object, and finally output classification information to characterize its psychological state.

[0112] S401. Construct a dynamic facial image of the sample to be analyzed. First, data acquisition and preprocessing are performed. This process follows the S1 standard, acquiring facial videos of the target object and performing standardized preprocessing (including frame extraction, key point detection, frame alignment, and noise reduction) to obtain a standardized sequence of 200 facial images and their corresponding 68 key point coordinate sequences.

[0113] Based on the standardized data after collection and preprocessing, a dynamic facial map of the target object is constructed. Specifically, this includes: determining the rectangular boundaries of each ROI in each frame according to the same 23 ROI definitions; generating a node feature matrix using a feature extraction process (pre-trained ResNet-50 and dimensionality reduction); and constructing a weighted adjacency matrix based on sparse connectivity priors and the DTW algorithm.

[0114] S402, Model Reasoning and Probability Calculation The dynamic facial image samples constructed in step S401 are input into the pre-trained causal graph learning model. The model performs forward propagation (inference) operations, and its classifier module ultimately outputs a multi-dimensional probability vector. .

[0115] In one specific embodiment, if the model is used for analysis of a four-level depressive state, it outputs a four-dimensional probability vector: ; In the formula: The representative model predicts the probability that the target subject has no significant depressive symptoms. The representative model predicts the probability that the target subject has no significant depressive symptoms. This represents the predicted probability that the model determines the target subject to be in a state of moderate depression. The representative model predicts the probability that the target subject is in a state of severe depression, and satisfies the following conditions: + + + =1.

[0116] S403. Determine the final classification result. The final classification is determined using the maximum probability decision criterion. That is, the probability vector... The category index corresponding to the highest probability value is used as the final psychological state classification result output by the system.

[0117] S404, Generate an interpretable visualization report To enhance the transparency and credibility of the model's decision-making, this embodiment generates a visual analysis report.

[0118] In one specific embodiment, the report is presented in the form of a causal region of interest (ROI) heatmap: the weight values ​​of the causal node masks generated by the model during inference are extracted. The weight value corresponding to each ROI is then overlaid on the corresponding area of ​​the standard face template in the form of a heatmap (e.g., darker / brighter colors represent higher weight). This heatmap visually demonstrates which areas of the face the model assigns higher causal importance to during the current judgment, thus providing analysts with interpretable decision-making support.

[0119] Through the above steps, this invention achieves automated acquisition, processing, analysis, and status assessment of facial video data of target objects, and ultimately outputs structured classification information and optional decision-making basis.

[0120] like Figure 5 As shown, this is a visualization report generated along with the classification results. The weight values ​​of the node causal mask are superimposed on the 23 ROIs of the face image in the form of a heatmap. The higher the weight value, the darker or brighter the color of the region, indicating that the model pays more attention to the dynamic features of the facial region when making this judgment.

[0121] S5. Verification of Experimental Results This step aims to objectively verify the effectiveness of the proposed method for dynamic facial feature analysis based on causal graph learning through systematic comparative experiments. The experimental design follows a rigorous machine learning evaluation paradigm, conducting five-fold cross-validation experiments on multiple representative datasets to compare the performance of the proposed method with current mainstream baseline models.

[0122] To comprehensively evaluate performance, experiments were conducted on three datasets: AVEC2013, a classic public dataset used for depressive state analysis; AVEC2014, a public dataset providing different data distributions; and a clinical dataset (CDD) reflecting real-world clinical applications.

[0123] The experiment used four common metrics for evaluation: accuracy, precision, recall, and F1 score. Five-fold cross-validation was employed for all methods to ensure the statistical stability of the evaluation results.

[0124] The comparison model selected several representative baseline models for comparison, including: Traditional deep models: Deep Neural Networks (DNN), Recurrent Neural Networks (RNN), and Spatiotemporal Attention Networks (STA); General graph neural networks: Graph Convolutional Networks (GCN) and Graph Attention Networks (GAT).

[0125] Specifically, the experimental results are shown in Table 2. On the AVEC2013 dataset, the GAT-CGL model of this invention achieved the best performance across all metrics, with an accuracy of 68.71%. Compared to the base model GAT, the accuracy improved by 10.35 percentage points. This significantly demonstrates that the causal masking and intervention mechanism proposed in this invention can effectively enhance the model's ability to extract discriminative features from dynamic facial images. Similarly, the GCN-CGL model of this invention also improved the accuracy by 8.2 percentage points compared to the baseline model GCN. This proves that the causal masking mechanism and backdoor adjustment strategy proposed in this invention can effectively eliminate interference noise in the graph structure, significantly enhancing the discriminative ability of the base graph neural network. Compared to traditional DNNs and RNNs, the GAT-CGL of this invention achieved a leap of more than 25% in accuracy. This indicates that relying solely on temporal information or simple fully connected structures is insufficient to capture the complex heterogeneous collaborative features in facial micro-expressions, while the dynamic facial graph structure (DFGraph) of this invention has a significant advantage in this regard. In summary, on the AVEC2013 dataset, the method proposed in this invention outperforms existing state-of-the-art methods (SOTA) on all evaluation metrics. In particular, the dual improvement in F1 score and accuracy indicates that the model of this invention not only predicts more accurately but also achieves a more balanced identification of positive and negative samples.

[0126] Table 2 shows the results of the five-fold cross-validation experiment on the AVEC2013 dataset ("-" indicates that the results are unavailable).

[0127] As shown in Table 3, the GAT-CGL model of this invention still maintains its leading position in accuracy on the AVEC2014 dataset. Although the benchmark GAT model has a slightly higher F1 score, which may be due to the specific class distribution characteristics of this dataset, the method of this invention still has a clear advantage in accuracy, which is the core indicator for measuring overall performance, demonstrating its robustness.

[0128] The GAT-CGL model of this invention achieved an accuracy of 66.63%, the best performance among all compared models. Compared to the baseline GAT model (61.84% accuracy), this invention achieved a significant improvement of 4.79 percentage points. This fully verifies that by introducing a causal masking mechanism, the model can more effectively eliminate environmental noise, thereby making more accurate comprehensive judgments in complex testing environments. It is worth noting that in terms of the F1 score, the baseline GAT model (59.34 points) is slightly higher than the GAT-CGL model of this invention (55.62 points). This phenomenon is mainly attributed to the inherent class imbalance problem of the AVEC2014 dataset. Because the CGL framework of this invention introduces causal intervention and interference suppression mechanisms, the model tends to be more conservative and robust in decision-making, reducing overfitting to certain noisy features. Although this leads to a trade-off in recall for specific classes, from the perspective of overall accuracy, the metric that best reflects diagnostic reliability, this invention still establishes a significant leading position, proving its effectiveness in global discrimination.

[0129] Table 3 shows the results of five-fold cross-validation on the AVEC2014 dataset.

[0130] As shown in Table 4, the advantages of the method of this invention are more prominent on clinical datasets with higher data quality and closer resemblance to real-world scenarios. The GAT-CGL model achieved the highest accuracy (85.85%) and F1 score (85.63), with an accuracy improvement of nearly 4 percentage points compared to the basic GAT model. This result strongly demonstrates that the features extracted through the causal decoupling mechanism have a higher consistency with the pathological features of interest in clinical diagnosis.

[0131] Table 4. Results of five-fold cross-validation on the Clinical Dataset (CDD)

[0132] Based on the experimental results of the three datasets above, the following conclusions can be drawn: the proposed methods (GCN-CGL, GAT-CGL) outperform many existing baseline models on various evaluation metrics, especially the core accuracy metric. This scheme effectively addresses the problems of insufficient capture of local dynamic relationships and confusion between interfering features and causal features by introducing a dynamic face map and causal graph learning framework. DTW edge weight construction captures asynchronous dynamic associations, while causal masking and intervention loss achieve feature decoupling, thereby improving the model's discriminative ability and robustness.

[0133] The following specific embodiment further illustrates the implementation process of the present invention in detail. This embodiment demonstrates the complete process from data acquisition to final state classification output. Data acquisition and standardization preprocessing: Using a high-definition camera with 4K resolution (3840×2160 pixels) and a frame rate of 30 frames per second (fps), capture a frontal facial video of the target subject in a standard environment with uniform lighting and a clean background. Ensure that the angle between the center line of the target subject's face and the camera's optical axis is less than 15 degrees, the deviation between the line connecting the eyes and the horizontal axis of the image is less than 5 degrees, and maintain a distance of 0.5 meters to 1.0 meter.

[0134] The acquired video was sampled at equal intervals to extract an ordered sequence of 200 facial images.

[0135] For each frame in the sequence, the MTCNN algorithm is applied for high-precision face detection and bounding box cropping. Subsequently, all cropped face images are uniformly scaled to a fixed size of 282×282 pixels to achieve spatial normalization.

[0136] The OpenFace toolkit (or the equivalent dlib shape_predictor_68_face_landmarks model) is used to process each frame of the normalized face image, detect and output the two-dimensional pixel coordinates of 68 facial key points conforming to the iBUG 300-W standard.

[0137] Based on 68 detected key points, the outer corners of the left and right eyes and the tip of the nose were selected as references. A weighted least squares method was used to calculate the affine transformation matrix, and a geometric transformation was performed on each frame to achieve precise alignment with the reference frame (e.g., the intermediate frame) to eliminate rigid head movements. Finally, a Gaussian filter with a kernel size of (3, 3) was applied to smooth the aligned image sequence and suppress noise. The final output is a standardized 200-frame face image sequence and its corresponding 200-frame key point coordinate dataset.

[0138] Constructing a dynamic facial graph (DFGraph): This step transforms the obtained standardized data into structured graph data.

[0139] Based on the predefined geometric rules in Table 1, each frame of facial image is precisely divided into 23 regions of interest (ROIs), and these 23 ROIs are defined as 23 nodes of the dynamic facial map. The rectangular boundary of each ROI is calculated using the coordinates of its associated anchor keypoints, preset offsets, and preset width and height.

[0140] A ResNet-50 deep convolutional neural network pre-trained on the VGGFace2 large face recognition dataset was used as the feature extractor (classification layer removed).

[0141] For each ROI image patch (23×200 in total) in the 200-frame image sequence, the ResNet-50 model is input frame by frame, the initial features are extracted from its global average pooling layer, and then the dimensionality is reduced by a trainable fully connected layer to obtain a 10-dimensional deep feature vector of the ROI in the current frame.

[0142] For each ROI node, the 200 10-dimensional feature vectors obtained in 200 frames are concatenated strictly in chronological order to form a 2000-dimensional composite feature vector, which is used as the feature vector of that node.

[0143] The 2000-dimensional feature vectors of all 23 nodes are stacked vertically to form a node feature matrix.

[0144] Based on the list of "adjacent regions" defined in Table 1, determine the connection relationships between nodes and construct a binary initial adjacency matrix.

[0145] For each connection in the initial adjacency matrix, the Dynamic Time Warping (DTW) algorithm is used to calculate the minimum cumulative distance between the time series corresponding to the feature vectors of the two nodes. Subsequently, this distance is mapped to a similarity weight in the interval (0, 1) using a Gaussian kernel function.

[0146] Perform the Hadamard product (element-wise multiplication) between the similarity weight matrix and the initial adjacency matrix to obtain the final weighted adjacency matrix.

[0147] At this point, a dynamic facial image sample has been constructed.

[0148] Training and optimization of causal graph learning (CGL) models: A five-fold cross-validation strategy was adopted. All dynamic facial image data of the samples were randomly and uniformly divided into five non-overlapping subsets (preserving the class distribution).

[0149] Five independent training and evaluation rounds are conducted. In each round, one subset is selected as the test set, and the remaining four subsets are used as the initial training set. A portion (e.g., 10%) is then divided from the initial training set as the validation set, and the remainder is used as the training set.

[0150] On the training set, the Adam optimizer is used to iteratively update all learnable parameters of the CGL model (including the weights of the mask generator MLP, the bi-branch GCN, and the classifier MLP) with the combined loss function that includes supervised classification loss, interference suppression loss, and causal intervention loss as the optimization objective.

[0151] Set the maximum number of training epochs to 200. After each epoch, evaluate the model performance (e.g., accuracy) on the validation set.

[0152] If the validation set performance does not improve within several consecutive epochs (e.g., 20), an early stopping mechanism is triggered to terminate the current epoch training and save the model parameters with the best validation set performance.

[0153] After five rounds of training, the best model saved from each round is evaluated on the corresponding test set, and the average values ​​of metrics such as accuracy, precision, recall, and F1 score are calculated as the final generalization performance metrics of the model.

[0154] Output of the classification of depressive states: For a brand new facial video to be analyzed, the entire process of data acquisition, standardized preprocessing, and construction of a dynamic facial image is performed to transform it into a dynamic facial image sample.

[0155] This dynamic facial image sample is input into the best CGL model trained in the training and optimization steps of the causal graph learning (CGL) model, and forward propagation is performed.

[0156] The model's classifier module outputs a four-dimensional probability vector. ,in The representative model predicts the probability that the target subject has no significant depressive symptoms. The representative model predicts the probability that the target subject has no significant depressive symptoms. This represents the predicted probability that the model determines the target subject to be in a state of moderate depression. The representative model predicts the probability that the target subject is in a state of severe depression, and satisfies the following conditions: + + + =1.

[0157] Using the maximum probability method, The category corresponding to the element with the highest probability value is used as the final classification result of the depressive state of the target object by the system.

[0158] Simultaneously generate a visualization report, overlaying the weight values ​​of the causal node masks generated during model inference onto a standard face image in the form of a heatmap, intuitively displaying the key facial regions that the model focuses on in its decisions, and enhancing the interpretability of the analysis results.

[0159] Through the steps described above, this embodiment clearly demonstrates how to apply the theoretical framework of this invention to practical depressive state analysis tasks. From standardized data collection and structured graph construction to model training and inference with causal decoupling capabilities, and finally outputting interpretable classification results, a complete and closed-loop technical solution is formed.

[0160] The invention and its embodiments have been described above illustratively. This description is not restrictive, and the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. The accompanying drawings are only one embodiment of the invention, and the actual structure is not limited thereto. No reference numerals in the claims should limit the scope of the claims. Therefore, if a person skilled in the art is inspired by this description and designs a similar structure and embodiment without departing from the spirit of the invention, such design should fall within the scope of protection of this patent. Furthermore, the word "comprising" does not exclude other elements or steps, and the word "a" preceding an element does not exclude the inclusion of "a plurality" of that element. Multiple elements stated in the product claims may also be implemented by a single element through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A method for analyzing dynamic facial features of psychological states based on causal graph learning, comprising the following steps: Step 1: Acquire facial video of the target object, extract facial image frames, and form a facial image sequence; Step 2: Perform facial key point localization and frame alignment on each facial image frame in the facial image sequence, and output a standardized facial image sequence and key point coordinate sequence. Step 3: Based on the standardized facial image sequence and key point coordinate sequence, construct a dynamic facial map; the dynamic facial map includes a node feature matrix for characterizing the temporal dynamics of the appearance of different anatomical regions of the face and a weighted adjacency matrix for characterizing the asynchronous motion cooperative relationship between different anatomical regions of the face. Step 4: Input the dynamic facial image into the trained causal graph learning model and output the psychological state classification result of the target object; the causal graph learning model is trained based on a training dataset composed of multiple dynamic facial image samples, using a five-fold cross-validation strategy. During the training process, the causal feature decoupling module and classifier integrated in the model are jointly optimized with the goal of minimizing the combined loss function. The causal feature decoupling module is used to decouple causal features and interference features from the input dynamic face image, and the classifier module outputs the predicted classification result based on the causal features.

2. The method for dynamic facial feature analysis of psychological states based on causal graph learning according to claim 1, characterized in that, Step two involves locating facial key points and aligning frames, specifically including: Facial key points were located in each facial image frame using a standard of 68 facial key points. The outer corners of the left and right eyes and the tip of the nose were selected as reference points from facial key points, and a weight proportional to the detection confidence was assigned to each reference point to construct a weighted diagonal matrix. ; Based on the benchmark point and the weight diagonal matrix The affine transformation matrix that aligns the current frame to the reference frame is calculated using the weighted least squares method. The calculation formula is as follows: ； Let be the affine transformation matrix. This is the homogeneous coordinate matrix of the reference point in the current frame. This is the coordinate matrix of the corresponding point in the reference frame. This is a weighted diagonal matrix; The key point coordinate sequence is based on the affine transformation matrix. Perform synchronized updates; Based on the affine transformation matrix After updating all facial image frames and facial key point coordinates, the standardized facial image sequence and key point coordinate sequence are output.

3. The method for dynamic facial feature analysis of psychological states based on causal graph learning according to claim 1, characterized in that, Step three specifically includes: Based on the facial motion coding system, multiple regions of interest are defined as nodes of the dynamic facial map. The geometric boundary of each region is determined by the coordinates of the associated facial key points, the preset offset, the width, and the height. The depth features of each node in each frame of the facial image sequence are extracted, reduced in dimension, and then concatenated in time to form the dynamic feature vector of the node; the dynamic feature vectors of all nodes constitute a node feature matrix used to characterize the temporal dynamics of the appearance of different anatomical regions of the face. Based on prior knowledge of facial anatomy, the connection relationships between nodes are defined. For each pair of nodes that are connected, the dynamic time warping algorithm is used to calculate the distance between the time series corresponding to their dynamic feature vectors, and the obtained dynamic time warping distance is converted into edge weights representing the degree of similarity. All edge weights constitute a weighted adjacency matrix to characterize the asynchronous motion cooperative relationship between different anatomical regions of the face.

4. The method for dynamic facial feature analysis of psychological states based on causal graph learning according to claim 3, characterized in that, The extraction of depth features specifically includes: Using a ResNet-50 convolutional neural network pre-trained on the VGGFace2 dataset, the initial depth features of each node in the image region of each frame are extracted. The initial deep features are reduced to 10 dimensions using a trainable fully connected layer; The dimensionality-reduced features of each node across all frames of the image sequence are concatenated in time to form the final dynamic feature vector of that node.

5. The method for dynamic facial feature analysis of psychological states based on causal graph learning according to claim 3, characterized in that, The conversion of dynamic time-warped distance into edge weights is specifically achieved using a Gaussian kernel function, and the edge weight calculation formula is as follows: ； in, and For node indexing, For the first Each node and Edge weights between them The normalized DTW distance. This is the scale parameter.

6. The method for dynamic facial feature analysis of psychological states based on causal graph learning according to claim 1, characterized in that, The causal feature decoupling module includes a mask generator and a dual-branch graph convolutional network; The mask generator is configured to perform graph convolution operation on the node feature matrix of the dynamic face map with a graph convolution layer, and generate causal node masks, causal edge masks, and interference node masks and interference edge masks that are complementary to the causal node masks and causal edge masks, respectively, based on the node hiding features output after the operation. The dual-branch graph convolutional network is configured to use the causal node mask and causal edge mask, as well as the interference node mask and interference edge mask, to weight the input node feature matrix and weighted adjacency matrix respectively, and then perform graph convolution operation to output causal hidden features and interference hidden features. The causal hidden features are input into the classifier.

7. The method for dynamic facial feature analysis of psychological states based on causal graph learning according to claim 6, characterized in that, The mask generator is a multilayer perceptron based on an attention mechanism, wherein the generation formulas for the causal node mask and the causal edge mask are as follows: ； ； in, and For node indexing, For the first Each node's causal node mask To connect the first The node and the first Causal edge mask of edges of each node. For the first Hidden features of each node, It represents the Hadamah accumulation. Indicates passage And the normalization operation implemented by the Softmax function.

8. The method for dynamic facial feature analysis of psychological states based on causal graph learning according to claim 1, characterized in that, The combined loss function includes supervised classification loss. Interference loss and causal intervention loss , is represented as: ； in, , is a hyperparameter; supervised classification loss is the cross-entropy loss between the predicted output of the causal path and the true label; interference loss is the KL divergence loss between the predicted output of the interference path and the uniform distribution; causal intervention loss is the classification loss after fusing the causal features of the current sample with the interference features of other samples.

9. A dynamic facial feature analysis system for psychological states based on causal graph learning, characterized in that, For implementing the method as described in any one of claims 1-8, comprising: The data acquisition module is used to acquire facial videos of the target object, extract facial image frames, and form a facial image sequence; The preprocessing module is used to locate facial key points and align frames in each facial image frame of the facial image sequence, and output a standardized facial image sequence and key point coordinate sequence. A dynamic facial image construction module is used to construct a dynamic facial image based on the standardized facial image sequence and key point coordinate sequence. The analysis module is used to input the dynamic facial image into the trained causal graph learning model and output the psychological state classification result of the target object.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-8.

Images