Psychological scientific analysis method based on facial micro-expression recognition

Abstract

The invention discloses a psychological scientific analysis method based on facial micro-expression recognition, and relates to the technical field of facial micro-expression recognition, and the method comprises the following steps: an acquisition device starts to dynamically capture the facial expression of a person at a preset frame rate, obtains continuous facial image data, and provides image input for subsequent micro-expression analysis; facial features of each frame of image are recognized, the obtained features are converted into micro-expression change data information, and a data set is established. When a transient micro-expression is recognized, the frame rate of the acquisition equipment can be automatically improved to ensure that key face changes are captured at the moment when the micro-expression occurs. The dynamic adjustment improves the data precision, avoids missing capture of details due to a low frame rate, optimizes the frame rate according to the intensity and duration of the micro-expression, and ensures efficient use of resources, thereby improving the accuracy of emotion analysis and psychological state evaluation.

Description

Technical Field

[0001] The present invention relates to the technical field of facial micro-expression recognition, and in particular to a psychological science analysis method based on facial micro-expression recognition. Background Art

[0002] Psychological science analysis based on facial micro-expression recognition studies the individual's psychological state, emotional response, and potential psychological activities by capturing and interpreting human facial micro-expressions (i.e., facial changes in a very short period of time, such as the slight rise and fall of eyebrows, the instantaneous change of eyes, etc.). These micro-expressions usually occur unconsciously and can reveal the individual's true emotional response, even if they may try to conceal it on the surface. Through high-precision facial recognition technology, combined with psychological theories and analysis models, researchers can analyze and infer an individual's emotions, stress, anxiety, anger, fear and other psychological states. This technology is widely used in psychological research, criminal investigation, medical diagnosis, education, etc., which can help to more accurately understand human emotions and psychological changes, and thus provide a scientific basis for emotional regulation, psychotherapy and other aspects.

[0003] The existing technology has the following deficiencies: the existing technology usually uses a fixed frame rate to capture micro-expressions in the process of dynamic capture of micro-expressions through acquisition equipment, but some micro-expressions are extremely short-lived and usually have low intensity, and may only appear in a few milliseconds. Errors may occur in the dynamic capture of micro-expressions. Such errors may lead to misjudgment of micro-expressions, thereby affecting the accurate analysis of psychological states. Incorrect emotion recognition will not only interfere with the conclusions of psychological analysis, such as misjudging anxiety as anger, thereby triggering unnecessary adversarial interactions or improper interventions; in the field of criminal investigation, it may also lead to erroneous emotional speculation, which in turn affects the judgment of suspects and even leads to wrongful convictions, endangering the fair judgment of the case.

[0004] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0005] The purpose of the present invention is to provide a psychological science analysis method based on facial micro-expression recognition. When a short micro-expression is recognized, the frame rate of the acquisition device is automatically increased to ensure that key facial changes are captured at the moment when the micro-expression occurs. This dynamic adjustment improves data accuracy, avoids missing details due to low frame rate, and optimizes the frame rate according to the intensity and duration of micro-expressions to ensure efficient use of resources, thereby improving the accuracy of emotion analysis and psychological state assessment, so as to solve the problems in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: A psychological science analysis method based on facial micro-expression recognition, comprising the following steps:

[0007] The acquisition device starts dynamically capturing the facial expressions of a person at a preset frame rate, obtains continuous facial image data, and provides image input for subsequent micro-expression analysis;

[0008] By recognizing the facial features of each frame of image, the obtained features are converted into micro-expression change data information and a data set is established;

[0009] Extract the key features reflecting that the micro-expression is short-lived and of low intensity from the data set, analyze the extracted key features under the detection window, and quantify the changes of the micro-expression features in the time dimension;

[0010] Input the key features after quantitative analysis into a pre-trained deep learning model, and use the deep learning model to intelligently evaluate the micro-expression changes;

[0011] According to the evaluation results of the deep learning model, classify the current micro-expression into two categories: "short micro-expression" and "normal micro-expression";

[0012] For the "normal micro-expression", continue to perform dynamic capture and facial micro-expression recognition analysis at the original acquisition frame rate, and further perform emotion recognition, facial expression analysis, and psychological state evaluation based on the original acquisition data;

[0013] For the "short micro-expression", based on the evaluation results of the deep learning model on the micro-expression changes, the frame rate of the acquisition device will be automatically increased to ensure that the key data of the facial changes are captured at the moment when the micro-expression occurs, and the emotion analysis is accurately performed.

[0014] Preferably, extract the key features reflecting that the micro-expression is short-lived and of low intensity from the data set. The extracted features include the minute tension fluctuations of facial muscles and the change rate of facial geometric structure. Under the detection window, analyze the minute tension fluctuations of facial muscles and the change rate of facial geometric structure, respectively generate a local muscle tension fluctuation reference value and a facial geometric change rate reference value, and quantify the changes of the micro-expression features in the time dimension through the local muscle tension fluctuation reference value and the facial geometric change rate reference value, so as to accurately capture the minute tension fluctuations of facial muscles and the rapid changes of geometric structure.

[0015] Preferably, the specific steps for analyzing the minute tension fluctuations of facial muscles under the detection window to generate a local muscle tension fluctuation reference value are as follows:

[0016] First, the instantaneous tension fluctuations of facial muscle activities are captured by the displacement changes of key facial feature points. The instantaneous movement of each facial feature point is represented by the following formula:

[0017] ΔM i (k) = |M i (k) - M i (k - 1)|

[0018] , where M i (k) is the muscle tension value of facial feature point i at time frame k, and M i (k - 1) is the muscle tension value of facial feature point i at time frame k - 1. ΔM i (k) is the muscle tension fluctuation of facial feature point i at time frame k;

[0019] Then, by analyzing the instantaneous tension fluctuation values of each facial feature point in the time domain, the local muscle tension fluctuation rate is obtained. By calculating the fluctuation amplitude and change rate of each facial area within the detection window, the local muscle tension fluctuation rate is generated. The calculation expression is as follows:

[0020] ,

[0021] where R i is the local muscle tension fluctuation rate of facial feature point i, N is the total number of frames within the detection window, exp(-α·k) is the exponential decay factor used to assign different weights to the muscle tension fluctuations ΔM i (k), and α is the decay factor;

[0022] Finally, the obtained local muscle tension fluctuation rate R i is comprehensively analyzed to generate a local muscle tension fluctuation reference value to measure the intensity and duration of microexpressions. The generation formula is as follows:

[0023] ,

[0024] where LMTF is the local muscle tension fluctuation reference value, is the change amount of the local muscle tension fluctuation rate represented by facial feature point i at the j-th moment, exp(-β·j) is the exponential decay function, β is the decay factor, and M is the total number of time points within the detection window.

[0025] Preferably, the specific steps for analyzing the change rate of facial geometry under the detection window to generate a facial geometry change rate reference value are as follows:

[0026] First, the shape context method is used to accurately extract the facial geometric structure. The geometric structure features of the face are extracted from each frame of the facial image. By calculating the positional relationship between key points, the changes in facial expressions are transformed into a geometric model. The calculation expression is as follows:

[0027] ,

[0028] In the formula, Δd e is the sum of the geometric distances between e facial key points and all key points. x e and y e are the coordinates of the e-th key point in the facial image. x r and y r are the coordinates of the r-th remaining key point corresponding to the e-th key point. P is the total number of key points in the facial features;

[0029] Based on the extracted facial geometric features, the rate of change of the facial geometric structure over time is calculated. For microexpressions, their changes are manifested as subtle displacements in local areas. Therefore, the geometric deformation rate of facial key points is calculated. The calculation expression of the geometric change rate of facial key points is as follows:

[0030] ,

[0031] In the formula, V geom is the geometric change rate of facial key points. P e (t) is the position of the e-th facial key point at time point t. Δt is the time interval. ||P e (t)|| is the modulus of the position vector of the e-th key point, that is, the Euclidean distance from the key point to the origin. ||P e (t + Δt) - P e (t)|| is the Euclidean distance of the displacement of the e-th key point, that is, the actual moving distance of the key point between two time points. P e (t + Δt) is the position of the e-th facial key point at the next time point t + Δt;

[0032] Finally, through the geometric change rate V of facial key points geom , a reference value of the facial geometric change rate is generated. An exponential weighting function is used to smooth the change rate to obtain an accurate reference value of the facial geometric change rate. The calculation expression is as follows:

[0033] ,

[0034] In the formula, FGCR is the reference value of the facial geometric change rate. w t is the weight at the t-th time point, is an attenuation function used to control the attenuation effect at each time point, where t is the time point index, τ is the attenuation constant, and V geom (t) is the geometric change rate of the facial key points at the t-th time point.

[0035] Preferably, the locally muscle tension fluctuation reference value and the facial geometric change rate reference value after analysis are input into a pre-trained deep learning model. A micro-expression change coefficient is generated by the deep learning model, and the micro-expression change is intelligently evaluated through the micro-expression change coefficient.

[0036] Preferably, when the micro-expression change coefficient generated during the intelligent evaluation of the micro-expression change by the pre-trained deep learning model is compared and analyzed with the pre-set micro-expression change coefficient reference threshold, the current micro-expression is classified. The classification steps are as follows:

[0037] If the micro-expression change coefficient is greater than or equal to the pre-set micro-expression change coefficient reference threshold, the current micro-expression is classified as a normal micro-expression;

[0038] If the micro-expression change coefficient is less than the pre-set micro-expression change coefficient reference threshold, the current micro-expression is classified as a transient micro-expression.

[0039] Preferably, for the "transient micro-expression", based on the result of the micro-expression change evaluation by the deep learning model, the frame rate of the acquisition device is automatically increased to ensure capturing the key data of the facial changes at the moment when the micro-expression occurs, and the specific steps for accurate emotion analysis are as follows:

[0040] After the current micro-expression is classified as a transient micro-expression, the frame rate of the acquisition device is dynamically increased to ensure capturing the key facial changes within a short time window. The adjusted frame rate is calculated based on the degree of change of the micro-expression. The calculation expression is as follows:

[0041] ,

[0042] In the formula, F new is the adjusted frame rate, F initial is the initial frame rate, MEC is the micro-expression change coefficient, MEC ref The micro-expression change coefficient reference threshold θ is the sensitivity coefficient, is the smoothing coefficient,

[0043] Once the frame rate is increased, the acquisition device captures facial image data at the new frame rate. During the occurrence of a brief micro-expression, the acquisition device will perform in-depth analysis on each frame of the image to ensure that the key data of the subtle facial changes are captured. At this time, based on the accurate image data and continuous frame rate update, it is ensured that the analysis system can obtain complete and high-quality image data, thereby improving the accuracy of micro-expression recognition and the reliability of emotion analysis. The formula is as follows:

[0044] ,

[0045] In the formula, D captured is the image data set successfully captured, including each frame of data of the brief micro-expression, and ImageFrame y (t) represents the y-th frame of image data captured at time t, and H is the total number of image frames. is a time weighting function used to weight each frame of the image according to the time difference. e is the natural base, γ is the decay factor, and |t - t y | is the time difference between the previous time t and the acquisition time t y of the y-th frame, and t 0 and t 1 are the start and end times of the acquired data respectively.

[0046] In the above technical solution, the technical effects and advantages provided by the present invention are as follows:

[0047] The present invention ensures that the key data of facial changes are captured at the moment of the occurrence of a micro-expression and emotion analysis is accurately performed. Specifically, when the system recognizes a "brief micro-expression", it will dynamically adjust the frame rate of the acquisition device, automatically increasing the frame rate from the original frame rate to a higher frame rate (such as 120 frames per second or higher). This increase in the frame rate can increase the number of image frames captured per unit time, ensuring that the details of facial changes within a shorter time window are captured. Through this dynamic adjustment process, the acquisition device can capture sufficiently clear image data at the critical moment of the occurrence of a brief micro-expression, avoiding missing the changes in the micro-expression due to a low frame rate. In addition, the system can also automatically adjust the frame rate according to the change intensity and duration of the micro-expression, making the use of resources more efficient and avoiding unnecessary over-acquisition. With the increase in the frame rate, the accuracy of the data is guaranteed, and subsequent emotion analysis and psychological state assessment can be based on more accurate micro-expression recognition results, thereby greatly improving the accuracy of emotion recognition and the reliability of psychological state analysis. Description of the Drawings

[0048] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the accompanying drawings required in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0049] Figure 1 This is a method flow chart of a psychological science analysis method based on facial micro-expression recognition according to the present invention. Detailed implementation manners

[0050] Now, example embodiments will be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the example embodiments to those skilled in the art.

[0051] The present invention provides a Figure 1 psychological science analysis method based on facial micro-expression recognition as shown below, including the following steps:

[0052] The acquisition device starts dynamically capturing the facial expressions of a person at a preset frame rate, obtains continuous facial image data, and provides image input for subsequent micro-expression analysis.

[0053] The acquisition device capturing at a preset frame rate means capturing a certain number of image frames per second, and these frames will contain the minute movement information of facial muscles. At this stage, the data collected by the device will include all possible micro-expressions, including short and more obvious expression changes. Although a lower frame rate at this stage may not be able to fully capture extremely short and low-intensity micro-expressions, it provides a full-process data record for subsequent processing.

[0054] The acquisition device (such as a camera or other visual perception device) will start capturing the facial expression changes of the observed person at a preset frame rate. The frame rate refers to the number of image frames collected per second. Generally, the higher the frame rate, the more details can be captured, especially for quickly occurring micro-expressions. By continuously capturing these facial image data, the device can form a series of temporally continuous image sequences as the basis for subsequent analysis. These images will be used to extract and analyze the features of micro-expressions, such as the minute movements of facial muscles, emotional expressions, etc., and then conduct in-depth psychological analysis and emotional inference. In short, the acquisition device provides an original data stream through continuous image capture for the subsequent analysis system to recognize and understand micro-expressions.

[0055] By identifying the facial features of each frame of the image (such as the movement trajectories of key points like eyes, eyebrows, and the corners of the mouth), the obtained features are converted into micro-expression change data information and a data set is established;

[0056] These data include the change amount of facial muscles, the movement amplitude of the expression area, the duration of the expression, etc. In this process, facial recognition technology and image processing algorithms (such as facial landmark detection based on feature point extraction or deep learning models) are used to extract effective feature information. The data set will contain key features under different time points and different micro-expression states, serving as the basis for subsequent analysis.

[0057] Extract the key features reflecting that the micro-expression is short-lived and of low intensity from the data set, and analyze the extracted key features under the detection window to quantify the changes of micro-expression features in the time dimension;

[0058] Extract the key features reflecting that the micro-expression is short-lived and of low intensity from the data set. The extracted features include the tiny tension fluctuations of facial muscles and the change rate of facial geometric structure. Under the detection window, analyze the tiny tension fluctuations of facial muscles and the change rate of facial geometric structure, respectively generate the local muscle tension fluctuation reference value and the facial geometric change rate reference value, and quantify the changes of micro-expression features in the time dimension through the local muscle tension fluctuation reference value and the facial geometric change rate reference value, so as to accurately capture the tiny tension fluctuations of facial muscles and the rapid changes of geometric structure.

[0059] The tiny tension fluctuations of facial muscles can indicate that the micro-expression is in a short-lived and low-intensity state. Micro-expressions usually appear rapidly at the initial stage of an emotion or when emotion control has not been fully manifested, with a short duration and low intensity. The tiny tension fluctuations of facial muscles usually occur before the expression is fully manifested, and are the weak reactions of facial muscles and the nervous system at the beginning of an emotional response. These fluctuations do not immediately cause obvious facial expression changes, but rather occur subtle and slight muscle tension changes within an extremely short time, which are often not easily detectable by the naked eye. This kind of tension fluctuation is the initial sign of a micro-expression, usually disappearing within a few milliseconds, so it has transience. And its low intensity is reflected in the small amplitude of these fluctuations, only involving the slight tension or relaxation of facial muscles, without causing significant expression changes (such as squinting eyes, raising the corners of the mouth, etc.). By analyzing these tiny tension fluctuations, the initial stage of micro-expressions can be effectively identified, helping to distinguish low-intensity micro-expressions and avoiding missing this key emotional change signal due to lack of fine detection.

[0060] The specific steps for analyzing the tiny tension fluctuations of facial muscles under the detection window to generate the local muscle tension fluctuation reference value are as follows:

[0061] First, the instantaneous tension fluctuations of facial muscle activities are captured through the displacement changes of key facial feature points (such as the corners of the eyes, mouth, eyebrows, etc.). The instantaneous movement of each facial feature point is represented by the following formula:

[0062] ΔM i (k) = |M i (k) - M i (k - 1)|

[0063] , where M i (k) is the muscle tension value of facial feature point i at time frame k, and M i (k - 1) is the muscle tension value of facial feature point i at time frame k - 1. ΔM i (k) is the muscle tension fluctuation of facial feature point i at time frame k;

[0064] This step calculates the displacement difference between adjacent moments of feature points, reflecting the instantaneous fluctuations of muscle tension. For microexpressions, the tension fluctuations of facial muscles are usually small but sufficient to affect the changes in facial expressions. Therefore, it is necessary to accurately capture the small changes of each feature point. These instantaneous fluctuations are the signs of the short duration and low intensity of microexpressions, providing the basic data for subsequent analysis.

[0065] Then, by analyzing the instantaneous tension fluctuation values of each facial feature point in the time domain, the local muscle tension fluctuation rate is obtained. By calculating the fluctuation amplitude and change rate of each facial area within the detection window, the local muscle tension fluctuation rate is generated. The calculation expression is as follows:

[0066] ,

[0067] where R i is the local muscle tension fluctuation rate of facial feature point i, N is the total number of frames within the detection window, exp(-α·k) is the exponential decay factor, which is used to assign different weights to the muscle tension fluctuations ΔM i (k). α is the decay factor, which is a parameter controlling the decay rate and determines the speed of decay;

[0068] This step captures the dynamic changes of microexpressions, especially the characteristics of short - duration and low - intensity microexpressions, by calculating the weighted fluctuations of facial feature points in the time dimension.

[0069] Finally, the obtained local muscle tension fluctuation rate R i is comprehensively analyzed to generate a local muscle tension fluctuation reference value, which measures the intensity and duration of microexpressions. The generation formula is as follows:

[0070] ,

[0071] In the formula, LMTF is the local muscle tension fluctuation reference value, is the change amount of the facial local muscle tension fluctuation rate represented by the facial feature point i at the j-th moment, exp(-β·j) is an exponential decay function used to assign weights to the muscle tension fluctuations at each time point, β is the decay factor that controls the influence degree of the time factor on the calculation of the local muscle tension fluctuation reference value, and M is the total number of time points in the detection window.

[0072] This step comprehensively reflects the minute tension changes of the facial muscles by performing weighted averaging on the fluctuation characteristics of each facial area. As the local muscle tension fluctuation reference value LMTF becomes smaller, it indicates that the micro-expression is shorter and of lower intensity, while a larger LMTF value represents a more obvious expression change. Through this quantification method, the intensity of the micro-expression can be accurately evaluated, and the state of the micro-expression can be judged based on the changes in the fluctuation index.

[0073] The smaller the local muscle tension fluctuation reference value generated by analyzing the minute tension fluctuations of the facial muscles under the detection window usually means that the amplitude of the muscle tension fluctuations of the facial muscles is small and the duration is extremely short. Such minute fluctuations are exactly the characteristics of low-intensity and short micro-expressions because only weak muscle tension or relaxation occurs within a short time, without triggering obvious facial expression changes, indicating that the micro-expression has not fully appeared or is under control. On the contrary, if the local muscle tension fluctuation reference value is large, it means that the amplitude of the muscle tension fluctuations of the facial muscles is large, possibly accompanied by micro-expression changes over a longer period or stronger emotional responses, which usually indicates that the micro-expression has entered a normal or more obvious state.

[0074] The short and slight change rate of the facial geometric structure can indeed indicate that the micro-expression is short and of low intensity. Micro-expressions are caused by extremely small and rapid movements of the facial muscles, usually with an extremely short duration, a small change amplitude, and these minute changes often occur within a short time. For example, when a person feels slightly anxious or uneasy, there may be a slight twitch at the corners of the eyes or a slight upward curve of the mouth unconsciously. Although these movements are very short, they can reflect their true inner emotions. The change rate of the facial geometric structure is one of the key indicators for capturing such changes. When the change rate of the micro-expression is low and the duration is extremely short, it usually means that the movement amplitude of the facial muscles is small, the expression change is not significant, and the emotional fluctuation is also relatively weak. Such short and low-intensity changes are often difficult to capture by traditional facial expression recognition technologies but can be identified through fine analysis of the change rate of the geometric structure. Therefore, the short and slight change rate of the facial geometric structure can effectively reflect the shortness and low-intensity characteristics of the micro-expression, thus helping the analysis system to capture more accurate emotional signals in subtle emotional fluctuations.

[0075] The specific steps for analyzing the change rate of facial geometry under the detection window to generate a reference value for facial geometry change rate are as follows:

[0076] First, use the Shape Context Method to accurately locate and extract the facial geometry. Extract the geometric structure features of the face from each frame of the facial image. The features include the positions of facial key points (such as the edges of eyes, eyebrows, and mouth). By calculating the positional relationships between the key points, the change in facial expression is transformed into a geometric model. The calculation expression is as follows:

[0077] ,

[0078] In the formula, Δd e is the sum of the geometric distances between e facial key points and all key points. x e and y e are the coordinates of the e-th key point in the facial image. x r and y r are the coordinates of the remaining r-th key point corresponding to the e-th key point. P is the total number of key points in the facial features;

[0079] Based on the extracted facial geometric features, calculate the rate of change of the facial geometry over time. For microexpressions, their changes are manifested as subtle displacements in local areas. Therefore, calculate the geometric deformation rate of facial key points, which represents the relative change speed of the facial geometry between two frames. The calculation expression for the geometric change rate of facial key points is as follows:

[0080] ,

[0081] In the formula, V geom is the geometric change rate of facial key points. P e (t) is the position of the e-th facial key point at time point t. Δt is the time interval. ||P e (t)|| is the modulus of the position vector of the e-th key point, that is, the Euclidean distance from the key point to the origin. By calculating the distance of each facial key point to the origin, the overall position change of the facial structure can be reflected. ||P e (t + Δt) - P e (t)|| is the Euclidean distance of the displacement of the e-th key point, that is, the actual moving distance of the key point between two time points. This distance measures the degree of movement of a certain facial key point in a short time and is a core indicator of microexpression change. P e (t + Δt) is the position of the e-th facial key point at the next time point t + Δt;

[0082] This step quantifies the rate of overall facial geometry change by summing the displacement amounts of all key points and dividing by the facial geometry feature value at the current moment.

[0083] Finally, through the facial key point geometry change rate V geom , a facial geometry change rate reference value is generated. The facial geometry change rate reference value aims to reflect the transience and low intensity of microexpressions. The lower the rate, the smaller the facial geometry change rate reference value, indicating that the microexpression is shorter and of lower intensity. An exponential weighting function is used to smooth the change rate to obtain an accurate facial geometry change rate reference value. The calculation expression is as follows:

[0084] ,

[0085] In the formula, FGCR is the facial geometry change rate reference value, w t is the weight at the t-th time point, controlling the contribution of each time point to the facial geometry change rate reference value, is the decay function, used to control the decay effect at each time point, t is the time point index, τ is the decay constant, and V geom (t) is the facial key point geometry change rate at the t-th time point.

[0086] By weighting and decaying the change rate, it is possible to more accurately capture the transient changes of microexpressions and avoid the interference of noise, thereby providing an effective quantitative index for the classification of microexpressions.

[0087] The smaller the facial geometry change rate reference value generated after analyzing the change rate of the facial geometry structure under the detection window means that the change of the facial geometry structure is very slight and transient, thus reflecting the low intensity and transience of microexpressions. Microexpressions are essentially caused by subtle movements of facial muscles and usually manifest as changes with a short duration and small amplitude. When the facial geometry change rate reference value is low, it indicates that the geometric features of the facial expression (such as movements around the corners of the mouth, eyebrows, and eyes) change slowly and slightly, which is consistent with the characteristics of low-intensity and short microexpressions. On the contrary, if the facial geometry change rate reference value is high, it indicates that the facial geometry structure has changed greatly and quickly, which usually corresponds to normal and long-lasting expression changes, so it indicates that the microexpression is of normal intensity and duration.

[0088] Input the key features after quantitative analysis into a pre-trained deep learning model, and use the deep learning model to intelligently evaluate the microexpression changes;

[0089] Input the reference values of local muscle tension fluctuations and the reference values of facial geometric change rates after analysis into a pre-trained deep learning model. Generate micro-expression change coefficients through the deep learning model, and use the micro-expression change coefficients to intelligently evaluate micro-expression changes.

[0090] The pre-trained deep learning model refers to a neural network model trained in the micro-expression analysis task based on a large amount of labeled data (including real facial expressions and corresponding emotion or psychological state labels). The goal of this model is to automatically extract relevant features from the input facial image data, learn and identify the relationship between micro-expressions and emotional states. In the training stage, the deep learning model continuously adjusts its internal parameters through backpropagation and gradient optimization, gradually improving the accuracy in identifying different emotion categories (such as anger, anxiety, joy, surprise, etc.). Through multiple iterations, the model can learn the mapping relationship between these micro-expressions and specific emotions or psychological states from the subtle differences in pixels, facial expression features, muscle movements, etc. in the image.

[0091] Once the model is trained and can achieve good performance on the validation set, it becomes a pre-trained deep learning model. This model usually consists of several layers of neural networks (such as convolutional neural network CNN), which are specifically used to process visual information in facial images, and generate micro-expression change coefficients through the input reference values of local muscle tension fluctuations and the reference values of facial geometric change rates. The micro-expression change coefficient is the output result of this model, used to measure the change amplitude and emotional expression of facial micro-expressions over a period of time. The training of the model relies on a large amount of labeled data sets to ensure that it can identify short-term and low-intensity micro-expression features and intelligently conduct emotion evaluation. Through this method, the system can accurately evaluate the current facial expression based on the input micro-expression features and infer the emotion or psychological state of the individual. The advantage of this model is that it can handle complex non-linear relationships, automatically identify complex facial dynamics, make accurate judgments, avoid artificial subjective interference, and improve the accuracy and efficiency of micro-expression analysis.

[0092] The deep learning model is not limited here. Any deep learning model that can comprehensively analyze the reference value of local muscle tension fluctuations LMTF and the reference value of facial geometric change rate FGCR to generate micro-expression change coefficient MEC can be used. To implement the technical solution of the present invention, the present invention provides a specific implementation method;

[0093] The formula for generating the micro-expression change coefficient MEC is as follows: MEC = q 1 ·LMTF + q 2 ·FGCR, where q 1 、q 2They are respectively the preset proportionality coefficients for the local muscle tension fluctuation reference value LMTF and the facial geometry change rate reference value FGCR, and q 1 and q 2 are both greater than 0.

[0094] The preset proportionality coefficient refers to a constant or weight used in the model to adjust the contribution of different parameters to the final result. In this passage, the proportionality coefficients q 1 and q 2 are used to weight the contributions of two parameters (i.e., local muscle tension fluctuation LMTF and surface muscle movement rate FGCR) to generate the micro-expression change coefficient MEC. The role of these proportionality coefficients is to determine the importance and influence degree of each parameter in the final calculation.

[0095] The preset proportionality coefficients q 1 and q 2 respectively control the contributions of local muscle tension fluctuation and facial muscle movement rate to the micro-expression change coefficient. By adjusting these two coefficients, the model can be flexibly adjusted according to the relative importance of various factors in different situations to ensure that the influence of each factor meets the actual requirements. The condition here is that q 1 and q 2 must be greater than zero, indicating that both of these two factors play a positive role in the calculation, and their magnitudes will directly affect the evaluation result of micro-expression changes.

[0096] From the micro-expression change coefficient, it can be seen that the smaller the local muscle tension fluctuation reference value generated by analyzing the tiny tension fluctuation of facial muscles under the detection window, and the smaller the facial geometry change rate reference value generated by analyzing the change rate of facial geometry structure under the detection window, the smaller the micro-expression change coefficient generated when the micro-expression is intelligently evaluated by a pre-trained deep learning model, indicating that the micro-expression is in a transient and low-intensity state. On the contrary, it indicates a normal emotional expression.

[0097] According to the evaluation result of the deep learning model, the current micro-expression is classified into two categories: "transient micro-expression" and "normal micro-expression";

[0098] The micro-expression change coefficient generated when the micro-expression is intelligently evaluated by a pre-trained deep learning model is compared and analyzed with the preset micro-expression change coefficient reference threshold to divide the current micro-expression. The division steps are as follows:

[0099] If the micro-expression change coefficient is greater than or equal to the preset micro-expression change coefficient reference threshold, the current micro-expression is classified as a normal micro-expression;

[0100] If the micro-expression change coefficient is less than the preset micro-expression change coefficient reference threshold, the current micro-expression is classified as a transient micro-expression;

[0101] Brief microexpressions refer to the subtle changes that occur quickly and briefly in facial expressions, usually lasting for an extremely short time, often completed within dozens of milliseconds. Normal microexpressions, on the other hand, refer to the facial expression changes that last for a certain period of time and have a relatively large amplitude, usually manifested as obvious emotional reactions such as anger, happiness, sadness, etc. Different from brief microexpressions, normal microexpressions have a longer duration and are usually manifested as the coordinated movement of multiple areas of the face, such as the combined actions of the mouth, eyebrows, and eyes.

[0102] For "normal microexpressions", continue to capture dynamically and analyze facial microexpressions using the original acquisition frame rate, and further perform emotion recognition, facial expression analysis, and psychological state assessment based on the original acquisition data;

[0103] The purpose is to ensure continuous tracking and accurate analysis of normal microexpressions. Since normal microexpressions are usually accompanied by obvious and continuous facial expression changes, maintaining the original acquisition frame rate helps to comprehensively capture the dynamic activities of facial muscles, ensuring that during the entire emotional expression process, whether it is subtle changes or relatively intense emotional fluctuations, they can be fully recognized and analyzed. Through further processing of the original acquisition data, the system can accurately perform emotion recognition, facial expression analysis, and psychological state assessment, and then provide a more accurate judgment of an individual's emotional state. This continuous and full-process tracking analysis method helps to accurately evaluate an individual's emotional and psychological reactions when normal microexpressions occur, thus providing reliable data support for subsequent decision-making or intervention.

[0104] For "brief microexpressions", based on the results of the evaluation of microexpression changes by the deep learning model, the frame rate of the acquisition device will be automatically increased to ensure that key data on facial changes are captured at the moment when the microexpression occurs and accurate emotion analysis is performed;

[0105] For "brief microexpressions", based on the results of the evaluation of microexpression changes by the deep learning model, the specific steps to automatically increase the frame rate of the acquisition device to ensure that key data on facial changes are captured at the moment when the microexpression occurs and accurate emotion analysis is performed are as follows:

[0106] After the current microexpression is classified as a brief microexpression, the frame rate of the acquisition device is dynamically increased to ensure that key facial changes are captured within a short time window, and the adjusted frame rate is calculated based on the degree of change of the microexpression. The calculation formula is as follows:

[0107] ,

[0108] In the formula, F new is the adjusted frame rate, F initialis the initial frame rate, representing the acquisition frequency of the device under normal conditions. MEC is the micro-expression change coefficient, MEC ref The reference threshold θ of the micro-expression change coefficient is the sensitivity coefficient, which is used to control the influence of the degree of micro-expression change on the frame rate adjustment. is the smoothing coefficient, which is used to adjust the influence of the amplitude of micro-expression change on the frame rate increase.

[0109] By increasing the frame rate, the system can capture the details of short micro-expressions more efficiently. In the formula, the part emphasizes the non-linear influence of the difference in micro-expression change coefficients on the frame rate adjustment, enabling the system to perform more refined frame rate increases when the micro-expression changes are small or instantaneous.

[0110] Once the frame rate is increased, the acquisition device acquires facial image data at the new frame rate. During the occurrence of short micro-expressions, the acquisition device will perform in-depth analysis on each frame of the image to ensure capturing the key data of the subtle facial changes. At this time, based on the accurate image data and continuous frame rate updates, it is ensured that the analysis system can obtain complete and high-quality image data, thereby improving the accuracy of micro-expression recognition and the reliability of emotion analysis. The formula is as follows:

[0111] ,

[0112] In the formula, D captured is the image data set successfully captured, which contains each frame of data of short micro-expressions, ImageFrame y (t) represents the y-th frame of image data captured at time t. H is the total number of image frames. is the time weighting function, which is used to weight each frame of the image according to the time difference. e is the natural base, γ is the decay factor, which is a key parameter affecting the weighting function and determines the decay rate in time, controlling how fast the weight of the image frame decreases over time, |t - t y | is the time difference between the previous time t and the acquisition time t y of the y-th frame, t 0 and t 1 are the start and end times of the acquired data respectively.

[0113] This step ensures that the device can accurately capture each frame of the image at the critical moment of micro-expression occurrence, especially within the time window of short micro-expressions. The time weighting function strengthens the priority acquisition of key frame images, making the acquired image data more concentrated in time at the moment of micro-expression change, thereby improving the accuracy and reliability of emotion analysis.

[0114] By dynamically adjusting the frame rate of the acquisition device, it is ensured that short and low-intensity micro-expression changes can be accurately captured, thereby improving the accuracy of micro-expression recognition and the reliability of emotion analysis. Since the duration of micro-expressions is extremely short and the change amplitude is small, traditional fixed frame rates are prone to lag or distortion when capturing these subtle changes, resulting in missed captures or misinterpretations of micro-expressions. The deep learning model can effectively identify which expressions belong to short micro-expressions through intelligent evaluation of the current micro-expression changes, and automatically trigger an increase in the device frame rate based on this recognition result.

[0115] Specifically, when the system determines that a certain micro-expression is a short micro-expression, by increasing the frame rate of the camera device, more detailed facial image data can be obtained, which is crucial for capturing instantaneous facial changes in micro-expressions. High-frame-rate acquisition can capture more facial change details within a shorter time window, enabling the system to more accurately identify micro-expression features and avoid missing key emotional signals due to insufficient frame rate.

[0116] This dynamic adjustment process not only ensures the high precision of the acquired data but also reduces the waste of system resources. When the micro-expression change amplitude is large and the duration is long, the system will continue to use the standard frame rate and will not unnecessarily increase the frame rate, optimizing the overall data acquisition efficiency. Therefore, the role of this step is to ensure the accurate capture of short micro-expressions by improving device performance and provide more accurate data support for subsequent emotion analysis.

[0117] This invention ensures the capture of key data on facial changes at the moment of micro-expression occurrence and accurate emotion analysis. Specifically, when the system recognizes a "short micro-expression", it will dynamically adjust the frame rate of the acquisition device, automatically increasing from the original frame rate to a higher frame rate (such as 120 frames per second or higher). This increase in frame rate can increase the number of image frames acquired per unit time, ensuring the capture of facial change details within a shorter time window. Through this dynamic adjustment process, the acquisition device can capture sufficiently clear image data at the critical moment when short micro-expressions occur, avoiding missing micro-expression changes due to low frame rate. In addition, the system can also automatically adjust the frame rate according to the change intensity and duration of micro-expressions, making the use of resources more efficient and avoiding unnecessary over-acquisition. With the increase in frame rate, the accuracy of the data is guaranteed, and subsequent emotion analysis and psychological state assessment can be based on more accurate micro-expression recognition results, thus greatly improving the accuracy of emotion recognition and the reliability of psychological state analysis.

[0118] The above formulas are all dimensionless and take their numerical values for calculation. The formulas are obtained by collecting a large amount of data for software simulation to get a formula closest to the real situation. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0119] The above only describes certain exemplary embodiments of the present invention by way of illustration. Without doubt, for those of ordinary skill in the art, various modifications can be made to the described embodiments in various different ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and description are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

[0120] It should be noted that in this text, if there are relational terms such as first and second, etc., they are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0121] It should be understood that in various embodiments of the present application, the magnitudes of the serial numbers of the above processes do not mean the sequence of execution is prior or posterior. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0122] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in this text can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0123] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0124] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0125] In addition, in each embodiment of the present application, each functional unit may be integrated into one processing unit, may exist separately as individual physical units, or two or more units may be integrated into one unit.

[0126] As described above, only the specific embodiments of the present application are provided, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed in the present application can easily think of changes or substitutions, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

[0127] Only some exemplary embodiments of the present invention have been described by way of illustration above. Without doubt, for those of ordinary skill in the art, the described embodiments can be modified in various different ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the protection scope of the claims of the present invention.

Claims

1. A psychological science analysis method based on facial micro-expression recognition, characterized in that: The following steps are involved: The acquisition device starts to dynamically capture the facial expressions of people at a preset frame rate, obtains continuous facial image data, and provides image input for subsequent micro-expression analysis; By identifying the facial features of each frame of the image, the acquired features are converted into micro-expression change data information and a data set is established; Extract key features that reflect the short-term and low-intensity micro-expressions from the data set, analyze the extracted key features under the detection window, and quantify the changes of micro-expression features in the time dimension; The key features after quantitative analysis are input into the pre-trained deep learning model, and the deep learning model is used to intelligently evaluate the changes in micro-expressions; According to the evaluation results of the deep learning model, the current micro-expressions are classified into two categories: "short-term micro-expressions" and "normal micro-expressions"; For "normal micro-expressions", the original acquisition frame rate is maintained for dynamic capture and facial micro-expression recognition and analysis, and emotion recognition, facial expression analysis, and psychological state assessment are further performed based on the original acquisition data; For "brief micro-expressions", the results of the evaluation of micro-expression changes based on the deep learning model will automatically increase the frame rate of the acquisition device to ensure that the key data of facial changes are captured at the moment when the micro-expression occurs, and accurate emotional analysis can be performed.

2. A psychological science analysis method based on facial micro-expression recognition according to claim 1, characterized in that: Key features reflecting that micro-expressions are short-term and of low intensity are extracted from the data set. The extracted features include tiny tension fluctuations of facial muscles and the rate of change of facial geometric structure. Within the detection window, the tiny tension fluctuations of facial muscles and the rate of change of facial geometric structure are analyzed to generate local muscle tension fluctuation reference values ​​and facial geometry change rate reference values, respectively. The changes of micro-expression features are quantified in the time dimension through the local muscle tension fluctuation reference values ​​and the facial geometry change rate reference values, thereby accurately capturing the tiny tension fluctuations of facial muscles and the rapid changes of geometric structure.

3. A psychological science analysis method based on facial micro-expression recognition according to claim 2, characterized in that: The specific steps for analyzing the tiny tension fluctuations of facial muscles in the detection window to generate the reference value of local muscle tension fluctuations are as follows: First, the instantaneous tension fluctuations of facial muscle activity are captured by the displacement changes of key facial feature points. The instantaneous movement of each facial feature point is expressed by the following formula: ΔM i (k)=|M i (k)-M i (k-1)|, Where M i (k) is the muscle tension value of facial feature point i in time frame k, M i (k-1) is the muscle tension value of facial feature point i in time frame k-1, ΔM i (k) is the muscle tension fluctuation of facial feature point i in time frame k; Then, by analyzing the instantaneous tension fluctuation value of each facial feature point in the time domain, the local muscle tension fluctuation rate is obtained. By calculating the fluctuation amplitude and change rate of each facial area in the detection window, the local muscle tension fluctuation rate is generated. The calculation expression is as follows: ， In the formula, R i is the local muscle tension fluctuation rate of facial feature point i, N is the total number of frames in the detection window, and exp(-α·k) is the exponential decay factor used to give the muscle tension fluctuation ΔM at different time frames. i (k) Assign different weights, α is the attenuation factor; Finally, the local muscle tension fluctuation rate R i Comprehensive analysis is performed to generate a reference value for local muscle tension fluctuations to measure the intensity and duration of micro-expressions. The generation formula is as follows: ， Where LMTF is the reference value of local muscle tension fluctuation, is the change in the fluctuation rate of the local facial muscle tension represented by the facial feature point i at the jth moment, exp(-β·j) is the exponential decay function, β is the decay factor, and M is the total number of detection window time points.

4. A psychological science analysis method based on facial micro-expression recognition according to claim 2, characterized in that: The specific steps of analyzing the rate of change of facial geometry in the detection window to generate a reference value of the rate of change of facial geometry are as follows: First, the facial geometric structure is accurately extracted through the shape context method. The geometric structural features of the face are extracted from each frame of the facial image. By calculating the positional relationship between key points, the changes in facial expressions are converted into a geometric model. The calculation expression is as follows: ， In the formula, Δd e is the sum of the geometric distances between e facial key points and all key points, x e and e is the coordinate of the e-th key point in the facial image, x r and r are the coordinates of the remaining r-th key points corresponding to the e-th key point, and P is the total number of key points in the facial features; Based on the extracted facial geometric features, the rate at which the facial geometric structure changes over time is calculated. For micro-expressions, the changes are manifested as slight displacements in local areas, so the geometric deformation rate of facial key points is calculated. The expression for calculating the geometric change rate of facial key points is as follows: ， Where V geom is the geometric change rate of facial key points, P e (t) is the position of the e-th facial key point at time t, Δt is the time interval, ∥P e (t)∥ is the modulus of the position vector of the e-th key point, that is, the Euclidean distance of the key point from the origin, ∥P e (t+Δt)-P e (t)∥ is the Euclidean distance of the displacement of the e-th key point, that is, the actual movement distance of the key point between two time points, P e (t+Δt) is the position of the e-th facial keypoint at the next time point t+Δt; Finally, the geometric change rate V of the facial key points is geom ,Generate a reference value of facial geometry change rate, use an exponential weighting function to smooth the change rate, and obtain an accurate reference value of facial geometry change rate. The calculation expression is as follows: ， Where FGCR is the reference value of facial geometry change rate, w t is the weight at the tth time point, is the attenuation function, which is used to control the attenuation effect at each time point. t is the time point index, τ is the attenuation constant, and V geom (t) is the geometric change rate of the tth facial keypoint at the tth time point.

5. A psychological science analysis method based on facial micro-expression recognition according to claim 2, characterized in that: The analyzed local muscle tension fluctuation reference value and facial geometry change rate reference value are input into the pre-trained deep learning model, and the micro-expression change coefficient is generated by the deep learning model, and the micro-expression change is intelligently evaluated through the micro-expression change coefficient.

6. A psychological science analysis method based on facial micro-expression recognition according to claim 5, characterized in that: The micro-expression change coefficient generated by the pre-trained deep learning model when intelligently evaluating the micro-expression change is compared with the pre-set micro-expression change coefficient reference threshold, and the current micro-expression is divided. The division steps are as follows: If the micro-expression variation coefficient is greater than or equal to a preset micro-expression variation coefficient reference threshold, the current micro-expression is classified as a normal micro-expression; If the micro-expression variation coefficient is less than a preset micro-expression variation coefficient reference threshold, the current micro-expression is classified as a short-term micro-expression.

7. A psychological science analysis method based on facial micro-expression recognition according to claim 6, characterized in that: For "short-term micro-expressions", the results of the deep learning model's evaluation of micro-expression changes will automatically increase the frame rate of the acquisition device to ensure that the key data of facial changes are captured at the moment when micro-expressions occur, and the specific steps for accurate emotion analysis are as follows: After the current micro-expression is divided into short-term micro-expressions, the frame rate of the acquisition device is dynamically increased to ensure that key facial changes are captured within a short time window. The adjusted frame rate is calculated based on the degree of change of the micro-expression. The calculation expression is as follows: ， In the formula, F new is the adjusted frame rate, F initial is the initial frame rate, MEC is the micro-expression variation coefficient, and MEC ref The reference threshold value of the micro-expression change coefficient θ is the sensitivity coefficient, is the smoothing coefficient, Once the frame rate is increased, the acquisition device collects facial image data at the new frame rate. During the occurrence of short micro-expressions, the acquisition device will conduct in-depth analysis of each frame of the image to ensure that the key data of subtle facial changes are captured. At this time, based on accurate image data and continuous frame rate updates, the analysis system can obtain complete and high-quality image data, thereby improving the accuracy of micro-expression recognition and the reliability of emotion analysis. The formula is as follows: ， Where D captured is a successfully captured image dataset, containing each frame of short micro-expressions, ImageFrame y (t) represents the yth frame of image data captured at time t, H is the total number of image frames, is a time weighting function, which is used to weight each frame of the image according to the time difference, e is the natural base, γ is the attenuation factor, |tt y | is the previous time t and the yth frame acquisition time t y The time difference between them, t0 and t1 are the start and end time of data collection respectively.

Images