A micro-expression analysis method based on facial key point recognition

Abstract

The application relates to the technical field of artificial intelligence, and discloses a micro-expression analysis method based on facial key point recognition, which comprises the following steps: positioning a facial region through a high-precision face detection model; acquiring a facial key point sequence by using a light key point extraction network; constructing a micro-expression feature vector based on dynamic changes of time-series key points; and classifying and identifying the micro-expression by combining an attention mechanism and a multilayer perception machine. The system comprises an image acquisition module, a key point extraction module, a time-series feature modeling module and a micro-expression classification module. By fusing dynamic information of space-time key points and an attention mechanism, the application significantly improves the accuracy and environmental adaptability of micro-expression recognition.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence, specifically relating to a micro-expression analysis method based on facial key point recognition. Background Technology

[0002] With the rapid development of artificial intelligence and computer vision technologies, micro-expression analysis has demonstrated significant application value in fields such as psychological assessment, security monitoring, human-computer interaction, and clinical diagnosis. Micro-expressions are facial muscle movements that are extremely short-lived (typically 1 / 25 to 1 / 5 of a second) and subtle in amplitude, yet can accurately reflect an individual's emotional state. Automatic recognition relies on high-precision facial keypoint localization and dynamic trajectory modeling. Current technologies generally employ deep learning-based keypoint detection models (such as HRNet and MediaPipeFaceMesh) combined with optical flow or Kalman filtering for inter-frame tracking to capture the spatiotemporal evolution characteristics of micro-expressions.

[0003] Micro-expression analysis methods based on facial keypoint recognition aim to infer the activation state of specific facial action units by observing the subtle displacements of keypoint coordinates in consecutive video frames, thereby achieving emotion classification. The core of this type of method lies in maintaining the spatiotemporal consistency of keypoints under high-speed, minute movements, ensuring the stability and reliability of inputs to subsequent feature extraction and classification modules.

[0004] Existing technologies have significant shortcomings when dealing with rapidly changing micro-expressions: on the one hand, traditional inter-frame tracking algorithms are sensitive to sudden changes in motion, and key point coordinates are prone to drift at the beginning or peak of micro-expressions due to local texture blurring, lighting interference, or slight head movements; on the other hand, most methods lack explicit modeling of the movement trend of key points in the time dimension, and only rely on the information of the current frame or two adjacent frames for correction, which is difficult to effectively compensate for the positioning error caused by the instantaneous movement.

[0005] Such drift directly distorts the temporal and morphological characteristics of micro-expressions, causing misjudgment of action units and ultimately leading to incorrect emotion classification. Especially in real-world scenarios, the stability problem of key points is more prominent due to limitations in shooting distance, resolution, and non-cooperative acquisition conditions, severely restricting the robustness and practicality of micro-expression analysis systems. Summary of the Invention

[0006] To address the aforementioned problems, this invention provides a micro-expression analysis method based on facial key point recognition, comprising: Adaptively extract micro-expression keyframe sequences from the input video stream; Robust tracking of facial key points is performed on the micro-expression keyframe sequence to generate a temporal sequence of facial key points; Spatiotemporal feature encoding is performed on the temporal sequence of the facial key points to obtain a spatiotemporal feature representation of micro-expressions; Based on the spatiotemporal features of micro-expressions, a fusion analysis of activation patterns of multi-scale local action units is performed to generate micro-expression intensity maps. Based on the micro-expression intensity mapping, micro-expression category discrimination is performed and a classification result calibrated with confidence is output.

[0007] Preferably, the adaptive extraction of micro-expression keyframe sequences from the input video stream includes: The optical flow field is calculated for consecutive frames to obtain the global motion energy value, and a local binary mode operator is applied to each frame to calculate the local texture change metric. The global motion energy value and the local texture change metric are weighted and fused to form a frame-level saliency score; When the salience score of a frame first exceeds a preset threshold, it is marked as the micro-expression start frame; when the salience score is lower than another lower threshold for several consecutive frames, it is marked as the micro-expression end frame. The frame with the highest salience score between the starting frame and the ending frame is selected as the peak frame, and a fixed number of frames are extended forward and backward from the peak frame to form a micro-expression key frame sequence. If the length of the micro-expression keyframe sequence is less than 32 frames, it is completed by linear interpolation; if it exceeds 32 frames, it is downsampled to 32 frames evenly over time.

[0008] Preferably, robust tracking of facial key points is performed on the micro-expression keyframe sequence to generate a temporal sequence of facial key points, including: Initial keypoint detection is performed on a single frame image using a convolutional neural network, and the two-dimensional coordinates of sixty-eight standard anatomical landmarks are output. Construct an energy minimization model for keypoint trajectories, where the total energy is composed of a weighted sum of a data term and a smoothing term. The data term is based on the confidence map of the initial detection network, and the smoothing term is defined as the L2 norm of the second-order difference of the keypoint displacements between adjacent frames. The corrected temporal sequence of facial key points is obtained by iteratively optimizing and solving the energy minimization model.

[0009] Preferably, the temporal sequence of the facial key points is encoded with spatiotemporal features to obtain a spatiotemporal representation of micro-expressions, including: The coordinate sequence of each key point is centered to eliminate the influence of the overall head translation. The centered coordinate sequences of all 68 key points are concatenated into a 136×T dimensional matrix, where T is the length of the keyframe sequence; The matrix is ​​input into a temporal convolutional network containing four levels of dilated causal convolutional layers, with dilation factors of 1, 2, 4, and 8 for each level, kernel size of 3 for each level, and channel numbers of 64, 128, 256, and 512 respectively. Global average pooling is performed along the time dimension on the 512×T dimensional feature map output by the temporal convolutional network to obtain a 512-dimensional spatiotemporal feature representation of micro-expressions.

[0010] Preferably, based on the spatiotemporal feature representation of micro-expressions, a fusion analysis of multi-scale local action unit activation patterns is performed to generate a micro-expression intensity map, including: The face is divided into five semantic regions: left eyebrow region, right eyebrow region, periorbital region, nasolabial region, and mandibular region. For each region, extract a subset of key points and calculate the region-level deformation features based on the principal components of the relative displacement covariance matrix; The deformation features of each region are stacked over time to form a region-time feature tensor, and each region-level action unit activation intensity sequence is output through an independent lightweight convolutional encoder. The activation intensity sequences of all regional action units are fused based on a cross-regional attention mechanism, and the attention weight is determined by the cosine similarity between the regional activation sequence and the global mean sequence. Max pooling is performed on the fused action unit activation intensity sequence along the time dimension to obtain a 12-dimensional micro-expression intensity mapping.

[0011] Preferably, the deformation features of each region are stacked over time to form a region-time feature tensor, and each region-level action unit activation intensity sequence is output through an independent lightweight convolutional encoder, including: For the In each region Deformation features in frames Stack them to form ; Will The input is an independent three-layer convolutional encoder, each layer containing a 3×3 convolutional kernel, batch normalization, and ReLU activation function, and the output is a 12×T dimensional sequence of action unit activation intensities. .

[0012] Preferably, the activation intensity sequences of all regional action units are fused based on a cross-regional attention mechanism, including: The global activation sequence is obtained by calculating the mean of the activation sequences of all regions. ; For each region Calculate its relationship with cosine similarity ; The cosine similarity is converted into attention weights using the softmax function: ; Calculate the weighted fusion sequence .

[0013] Preferably, based on the micro-expression intensity mapping, performing micro-expression category discrimination and outputting a confidence-calibrated classification result includes: The micro-expression intensity mapping is input into a fully connected neural network, which outputs the probability distribution of seven basic emotions. Calculate the feature entropy of the current sample ,in This is a normalized version of the intensity mapping; Query the historical database for the 100 samples most similar to the current intensity mapping, and calculate the consistency ratio C of their true labels; Calculate the final confidence level ,in It represents the theoretical maximum entropy; like If the score is below 0.7, it is considered an atypical micro-expression; otherwise, output the emotion category with the highest probability and its confidence level. .

[0014] Preferably, the seven basic emotions include happiness, sadness, anger, fear, surprise, disgust, and contempt.

[0015] Preferably, the frame rate of the input video stream is not less than 120 frames per second.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Effectively suppresses keypoint drift: By constructing an energy minimization model of data terms + smoothing terms, the continuity of keypoint displacement between adjacent frames is constrained by the second-order difference L2 norm, forcing the trajectory to present a physical motion mode of continuous acceleration. This significantly reduces coordinate jumps caused by local texture blurring, lighting interference, head micro-movements or occlusion during the initial / peak stages of micro-expressions, ensuring the spatiotemporal consistency of 68 standard anatomical landmarks on the face in consecutive frames, providing a reliable data foundation for subsequent feature extraction and classification. 2. Adapts to high-speed dynamic micro-expressions: Targeting the extremely short duration of micro-expressions (1 / 25-1 / 5 second), the input video stream frame rate is required to be no less than 120 frames / second. Combined with an adaptive keyframe extraction mechanism, the entire process of micro-expression initiation, peak, and termination is fully captured, solving the shortcomings of traditional low frame rate acquisition or static frame analysis in not being able to cover the dynamic process of micro-expressions.

[0017] 2. Multi-scale dynamic feature full coverage: A four-level dilated causal convolutional network is adopted, with dilation factors of 1, 2, 4, and 8 respectively, which exponentially expands the receptive field to the entire sequence of 32 frames. At the same time, it captures the short-term local jitters and long-term overall trends of micro-expressions, making up for the shortcomings of existing technologies that only rely on information from adjacent frames and lack temporal global modeling. Elimination of irrelevant interference factors: The influence of overall head translation is eliminated by centering the coordinates of key points, focusing on the local micro-movements of facial muscles, ensuring that the feature representation only reflects the muscle deformation related to micro-expressions, and improving feature purity and discriminative power.

[0018] 3. Refined analysis of action unit activation patterns: The face is divided into 5 semantic regions. Through region-level deformation feature extraction, independent convolutional coding, and cross-regional attention fusion, 12 core action units related to micro-expressions are accurately located. This solves the problem that single-scale analysis cannot distinguish local muscle coordinated movements, making the feature representation more consistent with the muscle movement essence of micro-expressions. Confidence calibration avoids misjudgment: Combining feature entropy and historical sample label consistency ratio, a two-factor confidence evaluation model is constructed. It cannot classify ambiguous or atypical micro-expression labels, effectively avoiding the overconfidence of traditional algorithms in judging abnormal samples and improving the credibility of classification results. Comprehensive and accurate emotion classification: Covering seven basic emotions: happiness, sadness, anger, fear, surprise, disgust, and contempt, the synchronous output of emotion category and intensity is achieved through a fully connected neural network, meeting the needs of emotion recognition in multiple scenarios. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall technical solution architecture of a micro-expression analysis method based on facial key point recognition proposed in this invention; Figure 2 This is a schematic diagram of the core principle framework of the micro-expression keyframe adaptive extraction mechanism based on dynamic temporal modeling in this invention; Figure 3 This is a logical flowchart of the spatiotemporal feature encoding of facial key point sequences and the mapping of micro-expression intensity in this invention. Figure 4 This is a logical flowchart of the multi-scale local action unit activation mode fusion analysis in this invention; Figure 5 This is a flowchart of the logic flow of the micro-expression category discrimination and confidence calibration module in this invention; Figure 6 This is a schematic diagram of the multi-level interaction relationship and data flow between the terminal device and the cloud analysis platform in this invention. Detailed Implementation

[0020] Please refer to Figures 1 to 6This invention proposes a micro-expression analysis method based on facial keypoint recognition, aiming to solve the problem of keypoint coordinate drift caused by abrupt changes in motion between frames during rapid micro-expression changes, thereby improving the accuracy and robustness of micro-expression classification. Micro-expressions, as involuntary facial movements with extremely short duration and subtle amplitude but high emotional indicative significance, are of great value in fields such as psychology research, security monitoring, and human-computer interaction. However, existing methods based on static keypoint detection or simple inter-frame differencing struggle to effectively maintain the spatial consistency of keypoints across consecutive frames when processing high-speed dynamic micro-expression sequences. Especially under interference conditions such as changes in lighting, head posture shifts, or partial occlusion, keypoint trajectories are prone to drastic jumps, leading to misjudgments in subsequent feature extraction and classification modules.

[0021] To overcome the aforementioned shortcomings, this invention constructs a full-process micro-expression analysis method that integrates dynamic temporal modeling, spatiotemporal feature encoding, multi-scale local action unit activation pattern analysis, and confidence calibration mechanism. This method uses high frame rate video streams as input. First, it performs adaptive extraction of micro-expression keyframes. Then, it performs robust tracking and spatiotemporal encoding of facial key points on the extracted frame sequence. Next, it generates a micro-expression intensity map through the fusion analysis of multi-scale local action unit activation patterns. Finally, it determines the micro-expression category and outputs a confidence-calibrated classification result.

[0022] The method includes the following steps: S1, adaptively extracting micro-expression keyframe sequences from the input video stream; S2, performing robust tracking of facial key points on the micro-expression keyframe sequences to generate a temporal sequence of facial key points; S3, performing spatiotemporal feature encoding on the temporal sequence of facial key points to obtain a micro-expression spatiotemporal feature representation; S4, performing multi-scale local action unit activation pattern fusion analysis based on the micro-expression spatiotemporal feature representation to generate a micro-expression intensity map; S5, performing micro-expression category discrimination based on the micro-expression intensity map and outputting a classification result calibrated with confidence.

[0023] In step S1, a sequence of micro-expression keyframes is adaptively extracted from the input video stream. The input video stream is captured by a high frame rate camera, with a frame rate of at least 120 frames per second, to ensure complete capture of the start, peak, and end phases of micro-expressions. Micro-expressions typically last between a quarter and half a second, corresponding to approximately thirty to sixty frames. To avoid redundant calculations and focus on the most discriminative frames, keyframes are filtered from the original video stream. The keyframe extraction mechanism is based on a joint criterion of overall facial motion energy and local texture changes.

[0024] First, the optical flow field is calculated for consecutive frames to obtain the pixel displacement vector field of each frame relative to the previous frame. Second, the optical flow field is spatially integrated to obtain the global motion energy value. Simultaneously, a local binary mode operator is applied to each frame to extract the local texture response, and its Hamming distance with the previous frame is calculated as a measure of local texture change. The global motion energy and the local texture change measure are weighted and fused to form a frame-level saliency score. When the saliency score of a frame first exceeds a preset threshold, it is marked as the micro-expression start frame. Subsequently, monitoring continues until the saliency score is below another lower threshold for several consecutive frames, at which point it is marked as the micro-expression end frame. Between the start and end frames, the frame with the highest saliency score is selected as the peak frame, and a fixed number of frames are extended forward and backward from the peak frame to form a micro-expression keyframe sequence. The length of this sequence is fixed at 32 frames. If it is insufficient, it is padded by linear interpolation; if it exceeds the limit, it is downsampled to 32 frames uniformly over time. This mechanism ensures that the extracted keyframe sequence completely covers the dynamic evolution of the micro-expression and excludes interference from irrelevant background frames.

[0025] In step S2, robust tracking of facial keypoints is performed on the micro-expression keyframe sequence to generate a temporal sequence of facial keypoints. Facial keypoints are defined as 68 standard anatomical landmarks, covering the eyebrows, eyes, nose, mouth, and jawline. Keypoint tracking employs a two-stage strategy: the first stage is initial keypoint detection, and the second stage is temporal consistency optimization. Initial keypoint detection is implemented using a convolutional neural network, which takes a single frame image as input and outputs the two-dimensional coordinates of 68 keypoints. To improve detection accuracy, the network structure includes a multi-scale feature pyramid and an attention-guided mechanism, enabling it to capture global structural information on low-resolution feature maps and accurately locate local details on high-resolution feature maps. After initial detection, the temporal consistency optimization stage begins.

[0026] This stage introduces inter-frame motion constraints and deformation smoothing priors to construct an energy minimization model for keypoint trajectories. Let the set of keypoint coordinates in frame t be... ,in Indicates the first The key point is in the first Frame position. Define total energy. ,in This is a data item that measures the consistency between the detection result of the current frame and the observed image; As a smoothing term, it constrains the continuity of keypoint displacements between adjacent frames; This is the balance coefficient, with a value of 0.5. (Data item) The smoothing term is derived from the confidence graph of the initial detection network. Defined as The L2 norm of the second-order difference is used to force the keypoint trajectory to exhibit a continuous acceleration motion pattern. The corrected keypoint sequence is obtained by iteratively optimizing this energy function. ,in This represents the length of the keyframe sequence. This optimization process effectively suppresses coordinate jumps caused by instantaneous noise or local occlusion, ensuring the physical plausibility of the keypoint trajectory in the time dimension.

[0027] In step S3, the temporal sequence of facial key points is spatiotemporally encoded to obtain a spatiotemporal representation of micro-expressions. The goal of spatiotemporal feature encoding is to transform the discrete key point coordinate sequence into a compact vector representation that contains dynamic evolution laws.

[0028] First, for each key point , and put it in Coordinate sequence in the frame Centering is performed by subtracting the mean position of the keypoint across the entire sequence to eliminate the influence of overall head translation. Then, the centered coordinate sequences of all 68 keypoints are concatenated into a 136×T matrix X, where each column corresponds to a frame and each row corresponds to a coordinate component (x or y). To capture dynamic patterns at different time scales, a temporal convolutional network is used to encode X. This network contains four levels of dilated causal convolutional layers with dilation factors of 1, 2, 4, and 8, all with a kernel size of 3 and channel numbers of 64, 128, 256, and 512 respectively. The dilated convolutional structure allows the receptive field to grow exponentially with the number of layers; the receptive field of the fourth layer covers all 32 frames, thus simultaneously capturing both short-term local jitter and long-term overall trends.

[0029] After four levels of convolution, the output feature map is obtained. To obtain a fixed-length sequence representation, global average pooling is performed on F along the time dimension, resulting in a 512-dimensional spatiotemporal feature vector. This vector represents the spatiotemporal features of micro-expressions, with each dimension corresponding to a specific spatiotemporal motion pattern, such as the initial speed of eyebrow raising and the duration of mouth corner stretching.

[0030] In step S4, based on the spatiotemporal feature representation of micro-expressions, a fusion analysis of multi-scale local action unit activation patterns is performed to generate a micro-expression intensity map. Action units are the basic building blocks of a facial motion coding system, with each action unit corresponding to a specific set of muscle movements. Micro-expressions are typically composed of brief activations of a few action units. To analyze these activation patterns, this invention designs a multi-scale local action unit analysis module. This module first divides the face into five semantic regions: left eyebrow region, right eyebrow region, periorbital region, nasolabial region, and mandibular region. For each region, a subset of its internal key points is extracted, and region-level deformation features are calculated based on these subsets.

[0031] Deformation characteristics are obtained by calculating the principal components of the covariance matrix of the relative displacements of key points. Let the first... The region in the first The deformation characteristics of the frame are , The region feature dimension has a value of eight. The deformation features of all regions are stacked over time to form a region-time feature tensor. For each Each region-level action unit activation intensity sequence is output through an independent lightweight convolutional encoder. ,in The predefined number of action units is set to 12, covering common micro-expression-related action units such as AU1 (inner brow lift), AU4 (browlowerer), and AU12 (lipcornerpuller). Subsequently, all regions... Cross-regional attention fusion is performed. Attention weights are determined by the spatiotemporal correlation between regions, and the calculation formula is as follows:

[0032] in, The mean of the activation sequences across all regions. Let be the cosine similarity function. The fused action unit activation intensity sequence is: Finally, regarding Max pooling along the time dimension yields a 12-dimensional micro-expression intensity mapping vector s, where each dimension represents the maximum activation intensity of the corresponding action unit throughout the entire micro-expression process. This intensity mapping vector s serves as a semantic description of the micro-expression, directly reflecting its muscle movement composition.

[0033] In step S5, based on the micro-expression intensity mapping, micro-expression category discrimination is performed, and a classification result calibrated with confidence is output. Micro-expression categories include seven basic emotions: happiness, sadness, anger, fear, surprise, disgust, and contempt. The discrimination module is implemented using a fully connected neural network, with the micro-expression intensity mapping vector as input. The output is a seven-dimensional probability distribution. , To improve classification reliability, a confidence calibration mechanism is introduced. This calibration is based on the statistical consistency of historical classification results and the feature uncertainty of the current sample.

[0034] First, calculate the feature entropy of the current sample. ,in for The normalized version. Higher feature entropy indicates more dispersed activation of action units, less atypical micro-expressions, and therefore a lower confidence level. Secondly, query the historical database and compare it with the current... Calculate the consistency ratio of the K most similar samples (K is 100) with their true labels. Final confidence level ,in This is the weighting coefficient, with a value of 0.6. This represents the theoretical maximum entropy. If... If the expression is below a preset confidence threshold (e.g., 0.7), it is judged as "atypical micro-expression" or "unclassifiable". Otherwise, the category with the highest probability is output as the final result, along with a confidence level. This mechanism effectively avoids overconfident judgment of fuzzy or anomalous samples, improving the system's robustness in practical applications.

[0035] The above method relies on a complete hardware and software system. This system includes a front-end acquisition module, a cloud analysis platform, and a terminal interaction interface. The front-end acquisition module consists of a high frame rate camera and an embedded processor, responsible for real-time capture and initial compression of the video stream. The cloud analysis platform deploys all the above algorithm modules, receives the keyframe sequences uploaded by the front end, performs the full-process analysis from S1 to S5, and returns the classification results and confidence scores to the terminal. The terminal interaction interface displays the analysis results in a graphical interface, allowing users to view micro-expression categories, intensity distributions, and confidence scores. The system adopts a distributed architecture, with the front end performing only lightweight initial keyframe screening, while complex computational tasks are completed in the cloud, ensuring a balance between low latency and high accuracy.

[0036] In the data processing workflow, all intermediate data is stored in a standardized format. Facial keypoint time-series sequences are saved as floating-point arrays, and spatiotemporal feature representations and intensity mapping vectors are encapsulated in a JSON structure for easy cross-platform transmission. The system has a built-in exception handling mechanism: if the keypoint detection failure rate exceeds 10%, a resampling instruction is triggered, requiring the front-end to re-upload the video segment; if the network is interrupted, the most recent valid result is cached locally and synchronized after the connection is restored. Furthermore, the system regularly updates the action unit activation model, absorbing new labeled data through an online learning mechanism to continuously optimize classification performance.

[0037] In summary, this invention constructs a high-precision and robust micro-expression analysis method through five core steps: adaptive keyframe extraction, robust keypoint tracking, spatiotemporal feature encoding, multi-scale action unit fusion, and confidence calibration. This method effectively solves the keypoint drift problem of traditional algorithms under inter-frame motion abrupt changes, achieving fine-grained analysis and reliable classification of the dynamic process of micro-expressions.

Claims

1. A micro-expression analysis method based on facial key point recognition, characterized in that, include: Adaptively extract micro-expression keyframe sequences from the input video stream; The adaptive extraction of micro-expression keyframe sequences from an input video stream includes: calculating the optical flow field for consecutive frames to obtain global motion energy values, and applying a local binary mode operator to each frame to calculate a local texture change metric; weightedly fusing the global motion energy values ​​and the local texture change metric to form a frame-level saliency score; marking a frame as the micro-expression start frame when its saliency score first exceeds a preset threshold, and marking a frame as the micro-expression end frame when its saliency score is below another lower threshold for several consecutive frames; selecting the frame with the highest saliency score between the start frame and the end frame as the peak frame, and extending a fixed number of frames forward and backward from the peak frame to form a micro-expression keyframe sequence; if the length of the micro-expression keyframe sequence is less than 32 frames, it is completed by linear interpolation; if it exceeds 32 frames, it is downsampled to 32 frames uniformly over time. Robust tracking of facial keypoints is performed on the micro-expression keyframe sequence to generate a temporal sequence of facial keypoints. This process includes: performing initial keypoint detection on a single frame image using a convolutional neural network, outputting the two-dimensional coordinates of sixty-eight standard anatomical landmarks; constructing an energy minimization model for the keypoint trajectory, where the total energy is a weighted sum of a data term and a smoothing term. The data term is based on the confidence map of the initial detection network, and the smoothing term is defined as the L2 norm of the second-order difference of the keypoint displacements between adjacent frames; and iteratively optimizing the energy minimization model to obtain a corrected temporal sequence of facial keypoints. Spatiotemporal feature encoding is performed on the temporal sequence of the facial key points to obtain a spatiotemporal feature representation of micro-expressions; Based on the spatiotemporal features of micro-expressions, a fusion analysis of activation patterns of multi-scale local action units is performed to generate micro-expression intensity maps. Based on the micro-expression intensity mapping, micro-expression category discrimination is performed and a classification result calibrated with confidence is output.

2. The micro-expression analysis method based on facial key point recognition according to claim 1, characterized in that, Spatiotemporal feature encoding is performed on the temporal sequence of the facial key points to obtain a spatiotemporal feature representation of micro-expressions, including: The coordinate sequence of each key point is centered to eliminate the influence of the overall head translation. The centered coordinate sequences of all 68 key points are concatenated into a 136×T dimensional matrix, where T is the length of the keyframe sequence; The matrix is ​​input into a temporal convolutional network containing four levels of dilated causal convolutional layers, with dilation factors of 1, 2, 4, and 8 for each level, kernel size of 3 for each level, and channel numbers of 64, 128, 256, and 512 respectively. Global average pooling is performed along the time dimension on the 512×T dimensional feature map output by the temporal convolutional network to obtain a 512-dimensional spatiotemporal feature representation of micro-expressions.

3. The micro-expression analysis method based on facial key point recognition according to claim 2, characterized in that, Based on the aforementioned spatiotemporal features of micro-expressions, a fusion analysis of multi-scale local action unit activation patterns is performed to generate a micro-expression intensity map, including: The face is divided into five semantic regions: left eyebrow region, right eyebrow region, periorbital region, nasolabial region, and mandibular region. For each region, extract a subset of key points and calculate the region-level deformation features based on the principal components of the relative displacement covariance matrix; The deformation features of each region are stacked over time to form a region-time feature tensor, and each region-level action unit activation intensity sequence is output through an independent lightweight convolutional encoder. The activation intensity sequences of all regional action units are fused based on a cross-regional attention mechanism, and the attention weight is determined by the cosine similarity between the regional activation sequence and the global mean sequence. Max pooling is performed on the fused action unit activation intensity sequence along the time dimension to obtain a 12-dimensional micro-expression intensity mapping.

4. The micro-expression analysis method based on facial key point recognition according to claim 3, characterized in that, The deformation features of each region are stacked temporally to form a region-time feature tensor, and each region-level action unit activation intensity sequence is output through an independent lightweight convolutional encoder, including: For the The region is Deformation features in frames Stack them to form ; Will The input is an independent three-layer convolutional encoder, each layer containing a 3×3 convolutional kernel, batch normalization, and ReLU activation function, and the output is a 12×T dimensional sequence of action unit activation intensities. .

5. The micro-expression analysis method based on facial key point recognition according to claim 4, characterized in that, The activation intensity sequences of all regional action units are fused based on a cross-regional attention mechanism, including: The global activation sequence is obtained by calculating the mean of the activation sequences of all regions. ; For each region Calculate its relationship with cosine similarity ; The cosine similarity is converted into attention weights using the softmax function: ； Calculate the weighted fusion sequence .

6. The micro-expression analysis method based on facial key point recognition according to claim 5, characterized in that, Based on the micro-expression intensity mapping, micro-expression category discrimination is performed and a confidence-calibrated classification result is output, including: The micro-expression intensity mapping is input into a fully connected neural network, which outputs the probability distribution of seven basic emotions. Calculate the feature entropy of the current sample ,in This is a normalized version of the intensity mapping; Query the historical database for the 100 samples most similar to the current intensity mapping, and calculate the consistency ratio C of their true labels; Calculate the final confidence level ,in It represents the theoretical maximum entropy; like If the score is below 0.7, it is considered an atypical micro-expression; otherwise, output the emotion category with the highest probability and its confidence level. .

7. The micro-expression analysis method based on facial key point recognition according to claim 6, characterized in that, The seven basic emotions include happiness, sadness, anger, fear, surprise, disgust, and contempt.

8. The micro-expression analysis method based on facial key point recognition according to claim 1, characterized in that, The frame rate of the input video stream is not less than 120 frames per second.

Images