Micro-expression analysis method based on facial key point recognition

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a micro-expression analysis method based on facial key point recognition, and the method comprises the steps: positioning a facial region through a high-precision face detection model; acquiring a face key point sequence by using a lightweight key point extraction network; constructing a micro-expression feature vector based on the dynamic change of the time sequence key points; and micro expressions are classified and identified by combining an attention mechanism and a multi-layer perceptron. The system comprises an image acquisition module, a key point extraction module, a time sequence feature modeling module and a micro expression classification module. According to the method, the accuracy and the environmental adaptability of micro-expression recognition are remarkably improved by fusing the space-time key point dynamic information and the attention mechanism.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of artificial intelligence, and particularly relates to a micro-expression analysis method based on facial key point recognition. BACKGROUND

[0002] With the rapid development of artificial intelligence and computer vision technology, micro-expression analysis shows important application value in the fields of psychological evaluation, security monitoring, human-computer interaction and clinical diagnosis. Micro-expression is a facial muscle movement with extremely short duration (usually 1 / 25 to 1 / 5 seconds) and weak amplitude, but can truly reflect the emotional state of individuals. Its automatic recognition relies on high-precision facial key point positioning and dynamic trajectory modeling. The existing technology generally uses a key point detection model (such as HRNet, MediaPipeFaceMesh) based on deep learning combined with optical flow or Kalman filtering for inter-frame tracking to capture the spatio-temporal evolution features of micro-expression.

[0003] The micro-expression analysis method based on facial key point recognition aims to infer the activation state of a specific facial action unit through the subtle displacement of key point coordinates in consecutive video frames, and then realize emotion classification. The core of this method lies in maintaining the spatio-temporal consistency of key points under high-speed and small movements, ensuring the stability and reliability of the input of the subsequent feature extraction and classification modules.

[0004] The existing technology has significant defects in processing rapidly changing micro-expression: on the one hand, traditional inter-frame tracking algorithms are sensitive to motion mutations, and are prone to key point coordinate drift due to local texture blur, light interference or head micro-movement at the start or peak of micro-expression; on the other hand, most methods lack explicit modeling of key point motion trends in the time dimension, and only rely on current frame or adjacent two frames information for correction, making it difficult to effectively compensate for positioning errors caused by too fast instantaneous motion.

[0005] Such drift will directly distort the timing pattern features of micro-expression, causing action unit misjudgment and ultimately leading to emotion classification errors. Especially in real scenes, affected by shooting distance, resolution limitations and non-cooperative acquisition conditions, the stability of key points is more prominent, which seriously restricts the robustness and practicality level of the micro-expression analysis system. SUMMARY

[0006] To solve the above problems, the application provides a micro-expression analysis method based on facial key point recognition, comprising: adaptively extracting a micro-expression key frame sequence from an input video stream; robustly tracking facial key points of the micro-expression key frame sequence to generate a facial key point timing sequence; spatio-temporal feature encoding the facial key point timing sequence to obtain a micro-expression spatio-temporal feature representation; Based on the spatiotemporal features of micro-expressions, a fusion analysis of activation patterns of multi-scale local action units is performed to generate micro-expression intensity maps. Based on the micro-expression intensity mapping, micro-expression category discrimination is performed and a classification result calibrated with confidence is output.

[0007] Preferably, the adaptive extraction of micro-expression keyframe sequences from the input video stream includes: The optical flow field is calculated for consecutive frames to obtain the global motion energy value, and a local binary mode operator is applied to each frame to calculate the local texture change metric. The global motion energy value and the local texture change metric are weighted and fused to form a frame-level saliency score; When the salience score of a frame first exceeds a preset threshold, it is marked as the micro-expression start frame; when the salience score is lower than another lower threshold for several consecutive frames, it is marked as the micro-expression end frame. The frame with the highest salience score between the starting frame and the ending frame is selected as the peak frame, and a fixed number of frames are extended forward and backward from the peak frame to form a micro-expression key frame sequence. If the length of the micro-expression keyframe sequence is less than 32 frames, it is completed by linear interpolation; if it exceeds 32 frames, it is downsampled to 32 frames evenly over time.

[0008] Preferably, robust tracking of facial key points is performed on the micro-expression keyframe sequence to generate a temporal sequence of facial key points, including: Initial keypoint detection is performed on a single frame image using a convolutional neural network, and the two-dimensional coordinates of sixty-eight standard anatomical landmarks are output. Construct an energy minimization model for keypoint trajectories, where the total energy is composed of a weighted sum of a data term and a smoothing term. The data term is based on the confidence map of the initial detection network, and the smoothing term is defined as the L2 norm of the second-order difference of the keypoint displacements between adjacent frames. The corrected temporal sequence of facial key points is obtained by iteratively optimizing and solving the energy minimization model.

[0009] Preferably, the temporal sequence of the facial key points is encoded with spatiotemporal features to obtain a spatiotemporal representation of micro-expressions, including: The coordinate sequence of each key point is centered to eliminate the influence of the overall head translation. The centered coordinate sequences of all 68 key points are concatenated into a 136×T dimensional matrix, where T is the length of the keyframe sequence; The matrix is ​​input into a temporal convolutional network containing four levels of dilated causal convolutional layers, with dilation factors of 1, 2, 4, and 8 for each level, kernel size of 3 for each level, and channel numbers of 64, 128, 256, and 512 respectively. Global average pooling is performed along the time dimension on the 512×T dimensional feature map output by the temporal convolutional network to obtain a 512-dimensional spatiotemporal feature representation of micro-expressions.

[0010] Preferably, based on the spatiotemporal feature representation of micro-expressions, a fusion analysis of multi-scale local action unit activation patterns is performed to generate a micro-expression intensity map, including: The face is divided into five semantic regions: left eyebrow region, right eyebrow region, periorbital region, nasolabial region, and mandibular region. For each region, extract a subset of key points and calculate the region-level deformation features based on the principal components of the relative displacement covariance matrix; The deformation features of each region are stacked over time to form a region-time feature tensor, and each region-level action unit activation intensity sequence is output through an independent lightweight convolutional encoder. The activation intensity sequences of all regional action units are fused based on a cross-regional attention mechanism, and the attention weight is determined by the cosine similarity between the regional activation sequence and the global mean sequence. Max pooling is performed on the fused action unit activation intensity sequence along the time dimension to obtain a 12-dimensional micro-expression intensity mapping.

[0011] Preferably, the deformation features of each region are stacked over time to form a region-time feature tensor, and each region-level action unit activation intensity sequence is output through an independent lightweight convolutional encoder, including: For the first The region is Deformation features in frames Stack them to form ; Will The input is an independent three-layer convolutional encoder, each layer containing a 3×3 convolutional kernel, batch normalization, and ReLU activation function, and the output is a 12×T dimensional sequence of action unit activation intensities. .

[0012] Preferably, the activation intensity sequences of all regional action units are fused based on a cross-regional attention mechanism, including: The global activation sequence is obtained by calculating the mean of the activation sequences of all regions. ; For each region Calculate its relationship with cosine similarity ; The cosine similarity is converted into attention weights using the softmax function: ; Calculate the weighted fusion sequence .

[0013] Preferably, based on the micro-expression intensity mapping, performing micro-expression category discrimination and outputting a confidence-calibrated classification result includes: The micro-expression intensity mapping is input into a fully connected neural network, which outputs the probability distribution of seven basic emotions. Calculate the feature entropy of the current sample ,in This is a normalized version of the intensity mapping; Query the historical database for the 100 samples most similar to the current intensity mapping, and calculate the consistency ratio C of their true labels; Calculate the final confidence level ,in It represents the theoretical maximum entropy; like If the score is below 0.7, it is considered an atypical micro-expression; otherwise, output the emotion category with the highest probability and its confidence level. .

[0014] Preferably, the seven basic emotions include happiness, sadness, anger, fear, surprise, disgust, and contempt.

[0015] Preferably, the frame rate of the input video stream is not less than 120 frames per second.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Effectively suppresses keypoint drift: By constructing an energy minimization model of data terms + smoothing terms, the continuity of keypoint displacement between adjacent frames is constrained by the second-order difference L2 norm, forcing the trajectory to present a physical motion mode of continuous acceleration. This significantly reduces coordinate jumps caused by local texture blurring, lighting interference, head micro-movements or occlusion during the initial / peak stages of micro-expressions, ensuring the spatiotemporal consistency of 68 standard anatomical landmarks on the face in consecutive frames, providing a reliable data foundation for subsequent feature extraction and classification. 2. Adapts to high-speed dynamic micro-expressions: Targeting the extremely short duration of micro-expressions (1 / 25-1 / 5 second), the input video stream frame rate is required to be no less than 120 frames / second. Combined with an adaptive keyframe extraction mechanism, the entire process of micro-expression initiation, peak, and termination is fully captured, solving the shortcomings of traditional low frame rate acquisition or static frame analysis in not being able to cover the dynamic process of micro-expressions.

[0017] 2. Multi-scale dynamic feature full coverage: A four-level dilated causal convolutional network is adopted, with dilation factors of 1, 2, 4, and 8 respectively, which exponentially expands the receptive field to the entire sequence of 32 frames. At the same time, it captures the short-term local jitters and long-term overall trends of micro-expressions, making up for the shortcomings of existing technologies that only rely on information from adjacent frames and lack temporal global modeling. Elimination of irrelevant interference factors: The influence of overall head translation is eliminated by centering the coordinates of key points, focusing on the local micro-movements of facial muscles, ensuring that the feature representation only reflects the muscle deformation related to micro-expressions, and improving feature purity and discriminative power.

[0018] 3. Refined analysis of action unit activation patterns: The face is divided into 5 semantic regions. Through region-level deformation feature extraction, independent convolutional coding, and cross-regional attention fusion, 12 core action units related to micro-expressions are accurately located. This solves the problem that single-scale analysis cannot distinguish local muscle coordinated movements, making the feature representation more consistent with the muscle movement essence of micro-expressions. Confidence calibration avoids misjudgment: Combining feature entropy and historical sample label consistency ratio, a two-factor confidence evaluation model is constructed. It cannot classify ambiguous or atypical micro-expression labels, effectively avoiding the overconfidence of traditional algorithms in judging abnormal samples and improving the credibility of classification results. Comprehensive and accurate emotion classification: Covering seven basic emotions: happiness, sadness, anger, fear, surprise, disgust, and contempt, the synchronous output of emotion category and intensity is achieved through a fully connected neural network, meeting the needs of emotion recognition in multiple scenarios. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall technical solution architecture of a micro-expression analysis method based on facial key point recognition proposed in this invention; Figure 2 This is a schematic diagram of the core principle framework of the micro-expression keyframe adaptive extraction mechanism based on dynamic temporal modeling in this invention; Figure 3 This is a logical flowchart of the spatiotemporal feature encoding of facial key point sequences and the mapping of micro-expression intensity in this invention. Figure 4 This is a logical flowchart of the multi-scale local action unit activation mode fusion analysis in this invention; Figure 5 This is a flowchart of the logic flow of the micro-expression category discrimination and confidence calibration module in this invention; Figure 6 This is a schematic diagram of the multi-level interaction relationship and data flow between the terminal device and the cloud analysis platform in this invention. Detailed Implementation

[0020] Please refer to Figures 1 to 6This invention proposes a micro-expression analysis method based on facial keypoint recognition, aiming to solve the problem of keypoint coordinate drift caused by abrupt changes in motion between frames during rapid micro-expression changes, thereby improving the accuracy and robustness of micro-expression classification. Micro-expressions, as involuntary facial movements with extremely short duration and subtle amplitude but high emotional indicative significance, are of great value in fields such as psychology research, security monitoring, and human-computer interaction. However, existing methods based on static keypoint detection or simple inter-frame differencing struggle to effectively maintain the spatial consistency of keypoints across consecutive frames when processing high-speed dynamic micro-expression sequences. Especially under interference conditions such as changes in lighting, head posture shifts, or partial occlusion, keypoint trajectories are prone to drastic jumps, leading to misjudgments in subsequent feature extraction and classification modules.

[0021] To overcome the aforementioned shortcomings, this invention constructs a full-process micro-expression analysis method that integrates dynamic temporal modeling, spatiotemporal feature encoding, multi-scale local action unit activation pattern analysis, and confidence calibration mechanism. This method uses high frame rate video streams as input. First, it performs adaptive extraction of micro-expression keyframes. Then, it performs robust tracking and spatiotemporal encoding of facial key points on the extracted frame sequence. Next, it generates a micro-expression intensity map through the fusion analysis of multi-scale local action unit activation patterns. Finally, it determines the micro-expression category and outputs a confidence-calibrated classification result.

[0022] The method includes the following steps: S1, adaptively extracting micro-expression keyframe sequences from the input video stream; S2, performing robust tracking of facial key points on the micro-expression keyframe sequences to generate a temporal sequence of facial key points; S3, performing spatiotemporal feature encoding on the temporal sequence of facial key points to obtain a micro-expression spatiotemporal feature representation; S4, performing multi-scale local action unit activation pattern fusion analysis based on the micro-expression spatiotemporal feature representation to generate a micro-expression intensity map; S5, performing micro-expression category discrimination based on the micro-expression intensity map and outputting a classification result calibrated with confidence.

[0023] In step S1, a sequence of micro-expression keyframes is adaptively extracted from the input video stream. The input video stream is captured by a high frame rate camera, with a frame rate of at least 120 frames per second, to ensure complete capture of the start, peak, and end phases of micro-expressions. Micro-expressions typically last between a quarter and half a second, corresponding to approximately thirty to sixty frames. To avoid redundant calculations and focus on the most discriminative frames, keyframes are filtered from the original video stream. The keyframe extraction mechanism is based on a joint criterion of overall facial motion energy and local texture changes.

[0024] First, the optical flow field is calculated for consecutive frames to obtain the pixel displacement vector field of each frame relative to the previous frame. Second, the optical flow field is spatially integrated to obtain the global motion energy value. Simultaneously, a local binary mode operator is applied to each frame to extract the local texture response, and its Hamming distance with the previous frame is calculated as a measure of local texture change. The global motion energy and the local texture change measure are weighted and fused to form a frame-level saliency score. When the saliency score of a frame first exceeds a preset threshold, it is marked as the micro-expression start frame. Subsequently, monitoring continues until the saliency score is below another lower threshold for several consecutive frames, at which point it is marked as the micro-expression end frame. Between the start and end frames, the frame with the highest saliency score is selected as the peak frame, and a fixed number of frames are extended forward and backward from the peak frame to form a micro-expression keyframe sequence. The length of this sequence is fixed at 32 frames. If it is insufficient, it is padded by linear interpolation; if it exceeds the limit, it is downsampled to 32 frames uniformly over time. This mechanism ensures that the extracted keyframe sequence completely covers the dynamic evolution of the micro-expression and excludes interference from irrelevant background frames.

[0025] In step S2, robust tracking of facial keypoints is performed on the micro-expression keyframe sequence to generate a temporal sequence of facial keypoints. Facial keypoints are defined as 68 standard anatomical landmarks, covering the eyebrows, eyes, nose, mouth, and jawline. Keypoint tracking employs a two-stage strategy: the first stage is initial keypoint detection, and the second stage is temporal consistency optimization. Initial keypoint detection is implemented using a convolutional neural network, which takes a single frame image as input and outputs the two-dimensional coordinates of 68 keypoints. To improve detection accuracy, the network structure includes a multi-scale feature pyramid and an attention-guided mechanism, enabling it to capture global structural information on low-resolution feature maps and accurately locate local details on high-resolution feature maps. After initial detection, the temporal consistency optimization stage begins.

[0026] This stage introduces inter-frame motion constraints and deformation smoothing priors to construct an energy minimization model for keypoint trajectories. Let the set of keypoint coordinates in frame t be... ,in Indicates the first The key point is in the first Frame position. Define total energy. ,in This is a data item that measures the consistency between the detection result of the current frame and the observed image; As a smoothing term, it constrains the continuity of keypoint displacements between adjacent frames; This is the balance coefficient, with a value of 0.5. (Data item) The smoothing term is derived from the confidence graph of the initial detection network. Defined as The L2 norm of the second-order difference is used to force the keypoint trajectory to exhibit a continuous acceleration motion pattern. The corrected keypoint sequence is obtained by iteratively optimizing this energy function. ,in This represents the length of the keyframe sequence. This optimization process effectively suppresses coordinate jumps caused by instantaneous noise or local occlusion, ensuring the physical plausibility of the keypoint trajectory in the time dimension.

[0027] In step S3, the temporal sequence of facial key points is spatiotemporally encoded to obtain a spatiotemporal representation of micro-expressions. The goal of spatiotemporal feature encoding is to transform the discrete key point coordinate sequence into a compact vector representation that contains dynamic evolution laws.

[0028] First, for each key point , and put it in Coordinate sequence in the frame Centering is performed by subtracting the mean position of the keypoint across the entire sequence to eliminate the influence of overall head translation. Then, the centered coordinate sequences of all 68 keypoints are concatenated into a 136×T matrix X, where each column corresponds to a frame and each row corresponds to a coordinate component (x or y). To capture dynamic patterns at different time scales, a temporal convolutional network is used to encode X. This network contains four levels of dilated causal convolutional layers with dilation factors of 1, 2, 4, and 8, all with a kernel size of 3 and channel numbers of 64, 128, 256, and 512 respectively. The dilated convolutional structure allows the receptive field to grow exponentially with the number of layers; the receptive field of the fourth layer covers all 32 frames, thus simultaneously capturing both short-term local jitter and long-term overall trends.

[0029] After four levels of convolution, the output feature map is obtained. To obtain a fixed-length sequence representation, global average pooling is performed on F along the time dimension, resulting in a 512-dimensional spatiotemporal feature vector. This vector represents the spatiotemporal features of micro-expressions, with each dimension corresponding to a specific spatiotemporal motion pattern, such as the initial speed of eyebrow raising and the duration of mouth corner stretching.

[0030] In step S4, based on the spatiotemporal feature representation of micro-expressions, a fusion analysis of multi-scale local action unit activation patterns is performed to generate a micro-expression intensity map. Action units are the basic building blocks of a facial motion coding system, with each action unit corresponding to a specific set of muscle movements. Micro-expressions are typically composed of brief activations of a few action units. To analyze these activation patterns, this invention designs a multi-scale local action unit analysis module. This module first divides the face into five semantic regions: left eyebrow region, right eyebrow region, periorbital region, nasolabial region, and mandibular region. For each region, a subset of its internal key points is extracted, and region-level deformation features are calculated based on these subsets.

[0031] Deformation characteristics are obtained by calculating the principal components of the covariance matrix of the relative displacements of key points. Let the first... The region in the first The deformation characteristics of the frame are , The region feature dimension has a value of eight. The deformation features of all regions are stacked over time to form a region-time feature tensor. For each Each region-level action unit activation intensity sequence is output through an independent lightweight convolutional encoder. ,in The predefined number of action units is set to 12, covering common micro-expression-related action units such as AU1 (inner brow lift), AU4 (browlowerer), and AU12 (lipcornerpuller). Subsequently, all regions... Cross-regional attention fusion is performed. Attention weights are determined by the spatiotemporal correlation between regions, and the calculation formula is as follows:

[0032] in, The mean of the activation sequences across all regions. Let be the cosine similarity function. The fused action unit activation intensity sequence is: Finally, regarding Max pooling along the time dimension yields a 12-dimensional micro-expression intensity mapping vector s, where each dimension represents the maximum activation intensity of the corresponding action unit throughout the entire micro-expression process. This intensity mapping vector s serves as a semantic description of the micro-expression, directly reflecting its muscle movement composition.

[0033] In step S5, based on the micro-expression intensity mapping, micro-expression category discrimination is performed, and a classification result calibrated with confidence is output. Micro-expression categories include seven basic emotions: happiness, sadness, anger, fear, surprise, disgust, and contempt. The discrimination module is implemented using a fully connected neural network, with the micro-expression intensity mapping vector as input. The output is a seven-dimensional probability distribution. , To improve classification reliability, a confidence calibration mechanism is introduced. This calibration is based on the statistical consistency of historical classification results and the feature uncertainty of the current sample.

[0034] First, calculate the feature entropy of the current sample. ,in for The normalized version. Higher feature entropy indicates more dispersed activation of action units, less atypical micro-expressions, and therefore a lower confidence level. Secondly, query the historical database and compare it with the current... Calculate the consistency ratio of the K most similar samples (K is 100) with their true labels. Final confidence level ,in This is the weighting coefficient, with a value of 0.6. This represents the theoretical maximum entropy. If... If the expression is below a preset confidence threshold (e.g., 0.7), it is judged as "atypical micro-expression" or "unclassifiable". Otherwise, the category with the highest probability is output as the final result, along with a confidence level. This mechanism effectively avoids overconfident judgment of fuzzy or anomalous samples, improving the system's robustness in practical applications.

[0035] The above method relies on a complete hardware and software system. This system includes a front-end acquisition module, a cloud analysis platform, and a terminal interaction interface. The front-end acquisition module consists of a high frame rate camera and an embedded processor, responsible for real-time capture and initial compression of the video stream. The cloud analysis platform deploys all the above algorithm modules, receives the keyframe sequences uploaded by the front end, performs the full-process analysis from S1 to S5, and returns the classification results and confidence scores to the terminal. The terminal interaction interface displays the analysis results in a graphical interface, allowing users to view micro-expression categories, intensity distributions, and confidence scores. The system adopts a distributed architecture, with the front end performing only lightweight initial keyframe screening, while complex computational tasks are completed in the cloud, ensuring a balance between low latency and high accuracy.

[0036] In the data processing workflow, all intermediate data is stored in a standardized format. Facial keypoint time-series sequences are saved as floating-point arrays, and spatiotemporal feature representations and intensity mapping vectors are encapsulated in a JSON structure for easy cross-platform transmission. The system has a built-in exception handling mechanism: if the keypoint detection failure rate exceeds 10%, a resampling instruction is triggered, requiring the front-end to re-upload the video segment; if the network is interrupted, the most recent valid result is cached locally and synchronized after the connection is restored. Furthermore, the system regularly updates the action unit activation model, absorbing new labeled data through an online learning mechanism to continuously optimize classification performance.

[0037] In summary, this invention constructs a high-precision and robust micro-expression analysis method through five core steps: adaptive keyframe extraction, robust keypoint tracking, spatiotemporal feature encoding, multi-scale action unit fusion, and confidence calibration. This method effectively solves the keypoint drift problem of traditional algorithms under inter-frame motion abrupt changes, achieving fine-grained analysis and reliable classification of the dynamic process of micro-expressions.

Claims

1. A micro-expression analysis method based on facial landmark recognition, characterized in that, The method comprises the following steps: Adaptively extracting a micro-expression key frame sequence from an input video stream; Robustly tracking facial key points of the micro-expression key frame sequence to generate a facial key point time sequence; Encoding the facial key point time sequence in space-time features to obtain a micro-expression space-time feature representation; Based on the micro-expression space-time feature representation, performing fusion analysis of multi-scale local action unit activation patterns to generate a micro-expression intensity map; Based on the micro-expression intensity map, performing micro-expression category discrimination and outputting a classification result calibrated by confidence.

2. The micro-expression analysis method based on facial landmark recognition according to claim 1, characterized in that, Adaptively extracting a micro-expression key frame sequence from an input video stream, comprising: Calculating an optical flow field for consecutive frames to obtain a global motion energy value, and applying a local binary pattern operator to each frame of image to calculate a local texture change metric; Weighted fusion of the global motion energy value and the local texture change metric to form a frame-level saliency score; When the saliency score of a certain frame first exceeds a preset threshold, mark it as a micro-expression starting frame, and when the saliency score is lower than another lower threshold for a number of consecutive frames, mark it as a micro-expression ending frame; Selecting the frame with the highest saliency score between the starting frame and the ending frame as the peak frame, and expanding a fixed number of frames forward and backward from the peak frame to form a micro-expression key frame sequence; If the length of the micro-expression key frame sequence is less than 32 frames, complete it by linear interpolation, and if it exceeds, uniformly downsample it in time to 32 frames.

3. The micro-expression analysis method based on facial landmark recognition according to claim 2, characterized in that, Robustly tracking facial key points of the micro-expression key frame sequence to generate a facial key point time sequence, comprising: Performing initial key point detection on a single frame of image through a convolutional neural network to output two-dimensional coordinates of sixty-eight standard anatomical landmark points; Constructing an energy minimization model of key point trajectory, wherein the total energy is composed of a data term and a smoothing term, the data term is based on the confidence map of the initial detection network, and the smoothing term is defined as the L2 norm of the second-order difference of key point displacement between adjacent frames; Solving the energy minimization model through iterative optimization to obtain a corrected facial key point time sequence.

4. The micro-expression analysis method based on facial landmark recognition according to claim 3, characterized in that, Encoding the facial key point time sequence in space-time features to obtain a micro-expression space-time feature representation, comprising: Performing centering processing on the coordinate sequence of each key point to eliminate the overall translation effect of the head; Concatenating the centered coordinate sequences of all 68 key points into a 136×T matrix, where T is the length of the key frame sequence; Inputting the matrix into a temporal convolution network containing four levels of dilated causal convolution layers, the dilation factors of each level are 1, 2, 4, and 8 respectively, the convolution kernel size is 3, and the channel number is 64, 128, 256, and 512 respectively; Performing global average pooling on the 512×T feature map output by the temporal convolution network along the time dimension to obtain a 512-dimensional micro-expression space-time feature representation.

5. The micro-expression analysis method based on facial landmark recognition according to claim 4, characterized in that, Based on the micro-expression space-time feature representation, performing fusion analysis of multi-scale local action unit activation patterns to generate a micro-expression intensity map, comprising: Dividing the face into five semantic regions: left eyebrow region, right eyebrow region, eye region, nose-lip region, and chin region; Extracting the internal key point subset of each region, and calculating the regional deformation feature based on the principal components of the relative displacement covariance matrix; The deformation features of each region are stacked by time to form a region-time feature tensor, and the region-level action unit activation intensity sequence is output respectively through an independent lightweight convolutional encoder; The cross-region attention mechanism is used to fuse all region-level action unit activation intensity sequences, and the attention weight is determined by the cosine similarity between the region activation sequence and the global mean sequence; The action unit activation intensity sequence after fusion is maximum-pooled along the time dimension to obtain a 12-dimensional micro-expression intensity mapping.

6. The micro-expression analysis method based on facial landmark recognition according to claim 5, characterized in that, The deformation features of each region are stacked by time to form a region-time feature tensor, and the region-level action unit activation intensity sequence is output respectively through an independent lightweight convolutional encoder, including: Stacking the morphing features of the first region in the second frame forms a stack ; Will An independent three-layer convolutional encoder is input, each layer containing a 3x3 convolution kernel, batch normalization and ReLU activation function, and outputting a 12xT-dimensional action unit activation intensity sequence .

7. The micro-expression analysis method based on facial landmark recognition according to claim 6, characterized in that, The cross-region attention mechanism is used to fuse all region-level action unit activation intensity sequences, including: Computing the mean of all regional activation sequences results in a global activation sequence ; For each region , the cosine similarity of its to is calculated The cosine similarity is converted into attention weight through a softmax function: ； Computing a weighted fused sequence .

8. The micro-expression analysis method based on facial landmark recognition according to claim 7, characterized in that, Based on the micro-expression intensity mapping, micro-expression category discrimination is performed and a classification result calibrated by confidence is output, including: The micro-expression intensity mapping is input into a fully connected neural network to output the probability distribution of seven basic emotions; calculating a feature entropy of the current sample wherein is a normalized version of the intensity map; 100 samples most similar to the current intensity mapping in the historical database are queried, and the consistency ratio C of their true labels is counted; calculating a final confidence wherein is the theoretical maximum entropy; If Below 0.7, it is determined as atypical micro-expression, otherwise output the most probable emotion category and confidence .

9. The micro-expression analysis method based on facial landmark recognition according to claim 8, characterized in that, The 7 basic emotions include happy, sad, angry, afraid, surprised, disgusted, and contemptuous.

10. The micro-expression analysis method based on facial landmark recognition according to claim 1, characterized in that, The frame rate of the input video stream is not less than 120 frames per second.

Images