Psychological risk early warning method and system for micro-expression image sequence

Abstract

The invention discloses a psychological risk early warning method and system for a micro-expression image sequence, and relates to the technical field of computer vision and psychological health monitoring, and the method comprises the steps: extracting a continuous face image sequence from a video stream, and calculating an eyebrow displacement feature and a mouth corner deformation feature through face key point tracking; comparing with a preset threshold value to mark an eyebrow event and a mouth corner event; counting the occurrence frequency of the event in a preset monitoring window, and calculating an event coupling coefficient according to the time relevance of the event; performing weighted fusion on the interbrow event frequency, the mouth corner event frequency and the coupling coefficient to obtain a basic risk value; if the basic risk value is smaller than a preset risk threshold value, outputting a low-risk early warning signal; if yes, entropy value characteristics are calculated according to the probability distribution of the event types, and a high-risk or medium-risk early warning signal is judged to be output according to the entropy value characteristics; according to the method, the emotion change of an individual under the synergistic effect of multiple micro expressions can be judged, and personalized psychological risk early warning is provided.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and mental health monitoring, in particular to a mental risk early warning method and system for micro-expression image sequences. BACKGROUND

[0002] Objective and timely identification of mental health status is crucial for early intervention and risk prevention. As facial muscle activities with extremely short duration and difficult to suppress autonomously, micro-expressions have been proven to effectively reveal the true internal emotional state of individuals, showing important potential in clinical diagnosis, security screening and human-computer interaction, and becoming one of the key technical directions for non-invasive mental state monitoring. Automatically capturing and analyzing micro-expression features from continuous image sequences using computer vision technology is a core approach to realize efficient mental risk early warning.

[0003] Current mental risk assessment technology solutions based on micro-expressions mainly focus on extracting single action features of specific facial regions from video streams, and judging the occurrence frequency of single events based on pre-set static thresholds, and then calculating the risk level. This kind of method has three main technical defects: First, micro-expression events in the glabella region and the corner of the mouth are regarded as independent phenomena, ignoring the inherent synergistic mechanism of facial muscle groups when expressing emotions. Second, fixed time windows are used for event statistics, which cannot adapt to physiological differences in micro-expression duration of different individuals. Third, existing systems lack quantitative indicators for the pattern disorder degree when multiple micro-expression events occur mixedly, making it difficult to capture subtle features of mental state fluctuations.

[0004] Therefore, the core problem to be solved by the prior art is how to effectively model and quantify the dynamic temporal coupling relationship between different facial micro-expression events, and how to accurately depict the pattern disorder degree when multiple micro-expression events occur mixedly within a monitoring window, so as to significantly improve the reliability and individualization level of mental risk early warning.

[0005] In view of the above problems, the prior art needs to be improved. SUMMARY

[0006] In view of the deficiencies in the prior art, the present application provides a mental risk early warning method and system for micro-expression image sequences.

[0007] In order to achieve the above purpose, the technical scheme of the present application is as follows:

[0008] In a first aspect, the present application discloses a mental risk early warning method for micro-expression image sequences, comprising the following steps:

[0009] Obtaining a continuous facial image sequence, the image sequence is extracted from a video stream at a pre-set frame interval;

[0010] Based on the facial image sequence, the eyebrow displacement feature and mouth corner deformation feature between adjacent frames are calculated by facial key point tracking;

[0011] The eyebrow event is marked based on the comparison result between the eyebrow displacement feature and the first preset threshold, and the mouth event is marked based on the comparison result between the mouth corner deformation feature and the second preset threshold.

[0012] Within a preset monitoring window, the number of occurrences of the eyebrow-between event and the corner-of-mouth event is counted, and the event coupling coefficient is calculated based on the temporal correlation between the eyebrow-between event and the corner-of-mouth event.

[0013] The basic risk value is obtained by weighted and fused calculation of the number of eyebrow-related events, the number of mouth-related events, and the event coupling coefficient.

[0014] If the basic risk value is less than a preset risk threshold, a low-risk warning signal is output and the facial image sequence is reacquired; otherwise, the entropy feature is calculated based on the probability distribution of micro-expression event types within the monitoring window, and if the entropy feature is greater than a preset entropy threshold, a high-risk warning signal is output; otherwise, a medium-risk warning signal is output.

[0015] Secondly, this invention discloses a psychological risk early warning system for micro-expression image sequences, comprising:

[0016] An image acquisition module is used to acquire a continuous sequence of facial images, the image sequence being extracted from a video stream at preset frame intervals;

[0017] The feature extraction module is used to calculate the eyebrow displacement features and mouth corner deformation features between adjacent frames based on the facial image sequence using a facial key point tracking algorithm;

[0018] The event marking module is used to mark eyebrow events based on the comparison result of the eyebrow displacement feature and a first preset threshold, and to mark mouth events based on the comparison result of the mouth corner deformation feature and a second preset threshold.

[0019] The event statistics module is used to count the number of occurrences of the eyebrow-between event and the corner of the mouth event within a preset monitoring window, and to calculate the event coupling coefficient based on the temporal correlation between the eyebrow-between event and the corner of the mouth event;

[0020] The risk calculation module is used to perform weighted fusion calculations on the number of eyebrow events, the number of mouth events, and the event coupling coefficient to obtain a basic risk value.

[0021] The risk assessment module is used to determine whether the basic risk value is less than a preset risk threshold. If it is, a low-risk warning signal is output and the facial image sequence is reacquired; otherwise, the entropy calculation process is initiated.

[0022] The entropy calculation module is used to calculate the entropy feature based on the probability distribution of micro-expression event types within the monitoring window, and to determine whether the entropy feature is greater than a preset entropy threshold. If it is, a high-risk warning signal is output; otherwise, a medium-risk warning signal is output.

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0024] 1. By introducing a coupled detection mechanism of eyebrow displacement features and mouth corner deformation features, the problem of isolating micro-expression events in different facial regions in traditional methods is effectively solved. By capturing and quantifying the dynamic temporal coupling relationship between eyebrow and mouth corner micro-expression events, the ability to recognize complex micro-expression patterns is improved, enabling the system to more accurately judge an individual's emotional changes, especially in the case of multiple micro-expressions working together;

[0025] 2. By introducing information entropy theory and combining it with the probability distribution of micro-expression event types within the monitoring window, the entropy value characteristics can be calculated, which can quantify the degree of pattern disorder when various micro-expression events occur. This method not only improves the monitoring of common emotional changes, but also enhances the analytical accuracy of the evolution process of complex psychological states, and improves the system's sensitivity and accuracy to psychological risks.

[0026] 3. A dynamic adjustment monitoring window scheme is adopted, generating an adaptive time window based on individual baseline data. This dynamic adaptation mechanism automatically optimizes the event detection time window according to individual physiological differences, improving the accuracy of micro-expression analysis and reducing errors caused by individual differences, thereby providing more personalized psychological risk warnings. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is an overall block diagram of the method in Embodiment 1 of the present invention;

[0029] Figure 2 This is a flowchart of the method according to Embodiment 1 of the present invention;

[0030] Figure 3 This is an overall block diagram of the system in Embodiment 2 of the present invention. Detailed Implementation

[0031] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] Application Overview: In traditional psychological risk assessment systems, facial micro-expression monitoring methods typically employ discrete facial region motion feature extraction strategies. After sampling the video stream at fixed frame intervals, facial keypoint tracking algorithms are used to calculate the displacement of the brow bone region and the deformation of the corner of the mouth region. When the feature of a single region exceeds a preset static threshold, an isolated event is triggered. The system uses the linear superposition of the number of events occurring in each region within a fixed time window as the basis for risk assessment, without establishing a temporal correlation model for cross-regional motion events and ignoring the statistical characteristics analysis of event type distribution. Consequently, in real-world applications, when multiple facial micro-expression events occur in a temporally coupled or alternating manner, the system cannot effectively distinguish between coordinated action patterns and random noise interference, and lacks a quantitative representation of the dynamic evolution of complex events.

[0033] To address the aforementioned issues, this application first considers the lack of temporal correlation in cross-regional micro-expression events in traditional methods. Analysis reveals that isolated statistical analysis of the frequency of events between the eyebrows and the corners of the mouth fails to capture coordinated action patterns. To resolve this, this application proposes establishing an event coupling detection mechanism. Within a preset time window, it verifies whether events between the eyebrows and the corners of the mouth have a temporal correlation, using the proportion of correlated events as a coupling coefficient to enhance the recognition of complex micro-expression patterns. Simultaneously, addressing the deficiency of insufficient quantification of disordered event type distribution, this application introduces information entropy theory. By calculating the entropy value characteristics of the probability of occurrence of various events within the monitoring window, it effectively characterizes the complexity and randomness of micro-expression patterns, improving the accuracy of analyzing dynamic changes in psychological states. Furthermore, to solve the problem of poor individual adaptability caused by fixed parameters, this application adopts a dynamic risk threshold judgment mechanism. When the basic risk value exceeds the threshold, it triggers the entropy value analysis process, forming a multi-level evaluation architecture to ensure the accurate capture of high-risk signals.

[0034] Example 1:

[0035] like Figures 1-2As shown, the psychological risk warning method for micro-expression image sequences includes the following steps: acquiring a continuous facial image sequence, the image sequence being extracted from a video stream at preset frame intervals; calculating the eyebrow displacement feature and mouth corner deformation feature between adjacent frames based on the facial image sequence by tracking facial key points; marking eyebrow events based on the comparison result of the eyebrow displacement feature with a first preset threshold, and marking mouth corner events based on the comparison result of the mouth corner deformation feature with a second preset threshold; counting the occurrence times of eyebrow events and mouth corner events within a preset monitoring window, and calculating the event coupling coefficient based on the temporal correlation between eyebrow events and mouth corner events; performing weighted fusion calculation on the occurrence times of eyebrow events, mouth corner events, and event coupling coefficient to obtain a basic risk value; determining whether the basic risk value is less than a preset risk threshold, if so, outputting a low-risk warning signal and reacquiring the facial image sequence, otherwise calculating the entropy feature based on the probability distribution of micro-expression event types within the monitoring window, determining whether the entropy feature is greater than a preset entropy threshold, if so, outputting a high-risk warning signal, otherwise outputting a medium-risk warning signal.

[0036] The features include: Eyebrow displacement feature, which calculates the displacement change of the brow arch region between adjacent frames by tracking facial key points (specifically, a facial key point tracking algorithm based on a convolutional neural network), used to capture the subtle movement characteristics of the eyebrow muscle group in micro-expressions; Mouth corner deformation feature, which calculates the distance change of the mouth corner shape between adjacent frames by tracking key points at the corner of the mouth (specifically, optical flow or geometric deformation models), used to quantify the dynamic characteristics of muscle contraction or relaxation in the mouth corner region; Event coupling coefficient, which refers to the statistical correlation ratio between eyebrow events and mouth corner events in the time dimension (specifically, a sliding time window is used to detect related events and calculate the ratio), used to characterize the collaborative change patterns of micro-expressions in different facial regions; and Base risk value, a comprehensive index generated by weighted fusion of eyebrow event frequency, mouth corner event frequency, and coupling coefficient (specifically, a linear weighting or nonlinear fusion algorithm), used to integrate multi-dimensional features to form a unified risk assessment metric. Among them, the entropy feature refers to the Shannon entropy of the distribution of micro-expression event types within the monitoring window, which can be achieved by using probability statistics and information entropy formulas to quantify the degree of disorder in the occurrence pattern of micro-expression events.

[0037] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0038] First, a video stream is captured using a camera device, and a continuous sequence of facial images is extracted from the video stream at a frame interval of 30 frames per second.

[0039] Next, a deep learning-based facial landmark detection algorithm was used to process each frame of the image, extracting the coordinates of 68 facial landmarks. Among them, four landmarks in the brow bone region and four landmarks in the corner of the mouth region were selected as the basis for feature calculation.

[0040] The average displacement of the brow arch keypoints between adjacent frames is calculated as the brow displacement feature, and the change in distance between the corner of the mouth keypoints is calculated as the corner of the mouth deformation feature. The brow displacement feature is compared with a preset threshold of 2mm; if it exceeds the threshold, it is marked as a brow event. The corner of the mouth deformation feature is compared with a preset threshold of 1.5mm; if it exceeds the threshold, it is marked as a corner of the mouth event.

[0041] Within a 5-second monitoring window, the number of occurrences of eyebrow-related events and mouth-related events is counted. Simultaneously, for each eyebrow-related event, it is checked whether a mouth-related event occurs within the following 200ms; if so, the two are considered to be temporally correlated. The proportion of temporally correlated eyebrow-related events to the total number of eyebrow-related events is calculated as the event coupling coefficient.

[0042] The frequency of events between the eyebrows, the frequency of events at the corners of the mouth, and the event coupling coefficient are assigned weights of 0.3, 0.3, and 0.4 respectively, and then weighted and summed to obtain the basic risk value. If the basic risk value is less than the preset threshold of 0.5, a low-risk warning signal is output, and the next round of monitoring begins.

[0043] If the base risk value is greater than or equal to 0.5, the probability distribution of independent eyebrow events, independent corner-of-the-mouth events, and coupled events within the monitoring window is calculated, and the entropy feature is calculated according to the information entropy formula. If the entropy feature is greater than the preset threshold of 1.2, a high-risk warning signal is output; otherwise, a medium-risk warning signal is output.

[0044] Through the above-described scheme, this application can effectively capture and quantify the dynamic temporal coupling relationship between micro-expression events in different facial regions, improving the ability to recognize complex micro-expression patterns. The introduction of entropy feature analysis can accurately characterize the degree of pattern disorder when multiple micro-expression events occur simultaneously within the monitoring window, enhancing the system's ability to analyze the evolution of complex psychological states. The multi-level risk assessment architecture improves the sensitivity and accuracy of early warnings and reduces the false alarm rate. These improvements significantly enhance the reliability and personalization of psychological risk early warnings, providing a more objective and timely basis for early intervention and risk prevention.

[0045] This application further proposes that the eyebrow displacement feature is obtained by calculating the change in the coordinates of the key points of the eyebrow arch, and the mouth corner deformation feature is obtained by calculating the change in the distance between the key points of the mouth corner; the calculation process of the event coupling coefficient includes: for each eyebrow event, detecting whether there is a related mouth corner event within a preset time window; and using the ratio of the number of eyebrow events with related events to the total number of eyebrow events as the event coupling coefficient.

[0046] Specifically, the change in the coordinates of the brow arch key points is calculated by tracking the positional offset of key points in the brow arch region within the facial image sequence, with the horizontal displacement difference between two symmetrical key points in the glabella region selected as the feature value. The change in the distance between the corners of the mouth key points is obtained by measuring the rate of change of the length of the line connecting the two corners of the mouth key points, which is calculated as the ratio of the distance difference between adjacent frames to the initial distance. In the calculation of the event coupling coefficient, the length of the preset time window is set to 1.2 times the average duration of the individual baseline action. The detection of associated events uses a timestamp matching algorithm; when the time interval between the glabella event and the corner of the mouth event is less than the length of the time window, it is determined to be an associated event.

[0047] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0048] The eyebrow displacement feature is calculated by measuring the coordinate changes of key points on the brow ridge. Specifically, the inner endpoints of the left and right eyebrows can be selected as key points on the brow ridge, and the coordinate changes of these points between adjacent frames can be tracked. For example, the Euclidean distance change between the left and right key points on the brow ridge can be calculated as the eyebrow displacement feature.

[0049] Mouth corner deformation features are calculated by measuring the change in distance between key points at the corners of the mouth. Furthermore, the left and right corners of the mouth can be selected as key points, and the distance change between these two points in adjacent frames can be calculated. For example, the relative rate of change in distance between the left and right corner key points can be used as the mouth corner deformation feature.

[0050] The calculation process for the event coupling coefficient includes: for each eyebrow-related event, detecting whether there is a related mouth-related event within a preset time window. The preset time window can be set to 500 milliseconds. The ratio of the number of eyebrow-related events to the total number of eyebrow-related events is used as the event coupling coefficient. Therefore, the event coupling coefficient reflects the temporal correlation strength between the micro-expressions of the eyebrow and mouth.

[0051] Through the above technical solutions, this application achieves the accurate extraction and quantification of key features of facial micro-expressions. The eyebrow displacement feature and mouth corner deformation feature capture subtle muscle activities in the forehead and lower half of the face, respectively, providing a reliable basis for the recognition of micro-expression events. The introduction of the event coupling coefficient effectively characterizes the temporal correlation patterns between micro-expressions in different facial regions, overcoming the limitations of analyzing micro-expressions in isolation. This multi-feature fusion method significantly improves the comprehensiveness and accuracy of micro-expression analysis, laying a solid foundation for subsequent psychological risk assessment.

[0052] This application further proposes to use the ratio of the number of eyebrow events to the duration of the monitoring window as the eyebrow event frequency, and the ratio of the number of mouth corner events to the duration of the monitoring window as the mouth corner event frequency. The basic risk value is obtained by integrating the eyebrow event frequency, mouth corner event frequency and event coupling coefficient through a weighting coefficient.

[0053] The calculation of the frequency of events between the eyebrows is standardized by dividing the number of events by the window duration, eliminating the impact of differences in monitoring duration on the statistical results. The frequency of events at the corners of the mouth is calculated using the same logic to ensure that the two event frequencies have the same dimensions. The weighting coefficients are configured differently according to the contribution of different characteristics to psychological risk, and the event coupling coefficient reflects the degree of coordination between the two micro-expression movements. The three are fused together through a linear weighting method to form a basic risk value.

[0054] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0055] The weighted fusion calculation process includes the following steps:

[0056] First, calculate the frequency of eyebrow events by dividing the number of eyebrow events by the monitoring window duration. For example, if the monitoring window is 60 seconds and eyebrow events occur 10 times, then the frequency of eyebrow events is 10 / 60 = 0.167 times / second.

[0057] Secondly, calculate the frequency of mouth corner events by dividing the number of mouth corner events by the monitoring window duration. For example, if the monitoring window is 60 seconds and the mouth corner events occur 15 times, then the frequency of mouth corner events is 15 / 60 = 0.25 times / second.

[0058] Finally, the basic risk value is obtained by integrating the frequency of events between the eyebrows, the frequency of events at the corner of the mouth, and the event coupling coefficient through weighting coefficients. The specific integration method is as follows:

[0059] Basic risk value = w1 * frequency of events between the eyebrows + w2 * frequency of events at the corners of the mouth + w3 * event coupling coefficient

[0060] w1, w2, and w3 are preset weighting coefficients that can be adjusted according to the actual application scenario; for example, w1=0.3, w2=0.3, and w3=0.4 can be set.

[0061] Through the above technical solutions, this application achieves a comprehensive quantitative assessment of multiple micro-expression features. By standardizing the frequency of events between the eyebrows and corners of the mouth to their occurrence rate per unit time, the influence of different monitoring window lengths is eliminated, improving the consistency of risk assessment. Simultaneously, the introduction of an event coupling coefficient effectively captures the temporal correlation between micro-expression events, enhancing the comprehensiveness and accuracy of risk assessment. Furthermore, the use of weighted coefficients for feature fusion provides a flexible means of adjusting personalized risk assessment for different application scenarios, improving the system's adaptability.

[0062] This application further proposes the proportion probability of independent eyebrow events, corner-of-the-mouth events, and coupled events within a statistical monitoring window. A coupled event is defined as a corner-of-the-mouth event occurring within a preset time window when each independent eyebrow event occurs. The entropy value feature is obtained through probability distribution disorder calculation, and its calculation method is as follows:

[0063]

[0064] in, The entropy value is a characteristic, and K represents the set of event types. This indicates the probability of the corresponding event type occurring.

[0065] The event type set includes three types: independent eyebrow-related events, independent mouth-related events, and coupled events. The probability of each event is determined by the ratio of its occurrence frequency to the total number of events. The disorder of the probability distribution is calculated using the information entropy formula; a higher information entropy value indicates a more disordered event distribution. For example, the entropy value reaches its maximum when the three types of events are evenly distributed within the monitoring window; the entropy value is minimum when only a single event type exists. The entropy value feature, together with the basic risk value, participates in the risk level judgment. When the basic risk value exceeds a preset risk threshold, the entropy value feature is further used to distinguish between high-risk and medium-risk warning signals.

[0066] Specifically, in the risk assessment process, the total number of all micro-expression events within the monitoring window is first counted, and the proportions of independent eyebrow-shaped events, independent mouth-shaped events, and coupled events are calculated. The probabilities of each event type are substituted into the information entropy formula to obtain the entropy value characteristic, which characterizes the degree of disorder in the event distribution. If the entropy value characteristic exceeds a preset threshold, it indicates that multiple micro-expression events occur disorderly within the monitoring window, reflecting a high risk of psychological disorder and triggering a high-risk warning signal; conversely, a medium-risk warning signal is output. By introducing information entropy to quantify the regularity of event distribution, the problem of existing technologies lacking objective measurement of the disorder of mixed event patterns is effectively solved, significantly improving the accuracy of psychological risk assessment.

[0067] Through the above technical solution, this application can accurately quantify the degree of disorder in the distribution of micro-expression event types within the monitoring window. This effectively distinguishes between micro-expression changes under normal circumstances and disordered manifestations under abnormal psychological states, improving the accuracy of psychological risk warnings. Furthermore, by introducing entropy features, this solution can capture complex micro-expression patterns, rather than relying solely on the frequency statistics of single events, thereby enhancing the system's ability to identify potential psychological risks.

[0068] This application further proposes a scheme for dynamically adjusting the monitoring window, including: obtaining the average duration of eyebrow movements and the average duration of mouth movements from an individual baseline database; generating an adaptive time window by scaling the maximum value of the average duration; and using the adaptive time window instead of the preset monitoring window when calculating the event coupling coefficient.

[0069] The individual baseline database is obtained through historical data statistics. The average duration of eyebrow-related movements is determined by calculating the average time interval from the start to the end of eyebrow-related events, and the average duration of mouth-related movements is determined by calculating the average time span of mouth-related events. The scaling factor k ranges from 1.2 to 1.5 to cover the fluctuation range of movement durations. The adaptive time window generation logic selects the maximum duration of eyebrow-related and mouth-related movements and extends the time window length through linear scaling.

[0070] Specifically, during the individual baseline data acquisition phase, at least 50 sets of complete action cycle data for eyebrow and mouth corner events are continuously monitored, and the arithmetic mean of their durations is calculated. During real-time monitoring, the average action duration of the current individual is input into the time window calculation module to generate an adaptive time window that matches the individual's physiological characteristics. For example, if the average duration of an individual's eyebrow movement is 200 milliseconds and the mouth corner movement is 180 milliseconds, the maximum value of 200 milliseconds is selected and multiplied by a coefficient of 1.2, resulting in an adaptive time window of 240 milliseconds. This time window is used to detect the correlation between eyebrow and mouth corner events, ensuring that the time window covers the individual's complete micro-expression action cycle, thereby accurately identifying event coupling relationships. Thus, the system can dynamically optimize the time window parameters based on individual differences, improving the accuracy of event correlation detection and enhancing the personalization level of psychological risk assessment.

[0071] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0072] Obtain the average duration of eyebrow and mouth movements from the individual baseline database. Generate an adaptive time window by scaling the average duration to the maximum value. Use the adaptive time window instead of the preset monitoring window when calculating the event coupling coefficient.

[0073] The adaptive time window is calculated as follows:

[0074]

[0075] in, Indicates an adaptive time window. This indicates the average duration of the eyebrow movement. Average duration of mouth corner movements The preset scaling factor, It is a function for maximizing the value.

[0076] Specifically, the average duration data of eyebrow and mouth movements are first extracted from the individual baseline database. For example, for a given individual, the average duration of eyebrow movements... The average duration of the mouth corner movement is 0.5 seconds. The first value is 0.3 seconds. Then, the maximum of these two times is taken, i.e., max(0.5, 0.3) = 0.5 seconds. Assuming the preset scaling factor k is 2, the calculated adaptive time window... =1 second.

[0077] When calculating the event coupling coefficient in the future, this 1-second adaptive time window will be used instead of a fixed preset monitoring window; thus, the size of the time window can be dynamically adjusted according to the individual's micro-expression features, improving the system's adaptability.

[0078] Through the above technical solution, this application achieves dynamic adaptive adjustment of the monitoring window. By introducing micro-expression duration data from the individual baseline database, the system can automatically adjust the time window size according to the facial expression characteristics of different individuals. This adaptive mechanism significantly improves the accuracy and personalization level of micro-expression event detection. Because the time window matches the actual duration of an individual's micro-expression, the probability of false positives and false negatives is reduced, thereby improving the overall reliability of psychological risk warning. In addition, the dynamic adjustment method also enhances the system's adaptability to differences between individuals, making the risk assessment results more targeted and practical.

[0079] This application further proposes to calculate the rate of change characteristic of the basic risk value between continuous monitoring windows, and dynamically adjust the frame interval based on the comparison result of the rate of change characteristic and the preset rate of change threshold: if the rate of change characteristic is less than the first rate of change threshold, the frame interval is increased; if the rate of change characteristic is greater than the second rate of change threshold, the frame interval is decreased; otherwise, the frame interval remains unchanged.

[0080] The rate of change characteristic is obtained by dividing the absolute difference between the baseline risk values ​​of adjacent monitoring windows by the monitoring window duration. Mathematically, the rate of change characteristic equals the absolute difference between the baseline risk value of the current window and the baseline risk value of the previous window divided by the monitoring window duration. The logic for dynamically adjusting the frame interval is implemented through two preset rate of change thresholds. When the rate of change is below the first threshold, it indicates that the risk status is stabilizing, and the computational load is reduced by increasing the frame interval. When the rate of change is above the second threshold, it indicates that risk fluctuations are intensifying, and the data acquisition density is increased by decreasing the frame interval to capture rapid changes.

[0081] Specifically, the system periodically compares the baseline risk values ​​of adjacent monitoring windows and quantifies the dynamic characteristics of the risk state by calculating their rate of change. When the rate of change is in a low range, extending the frame interval reduces the image processing frequency and lowers hardware resource consumption; when the rate of change enters a high range, shortening the frame interval improves temporal resolution and avoids the loss of key micro-expression features. This adaptive mechanism ensures that the data acquisition frequency matches the intensity of risk fluctuations in real time, optimizing system operating efficiency while maintaining early warning accuracy. For example, when the monitoring window duration is 10 seconds and the baseline risk value suddenly increases from 0.3 to 0.7, the rate of change reaches 0.04 / second, exceeding the preset second rate of change threshold of 0.03 / second. The system then adjusts the frame interval from 500 milliseconds to 200 milliseconds, thereby acquiring a higher density image sequence in subsequent monitoring to track rapid changes.

[0082] As a preferred embodiment, the solution of this application is specifically implemented as follows:

[0083] The rate of change characteristic of the baseline risk value between continuous monitoring windows is calculated, and the frame interval is dynamically adjusted based on the comparison result of the rate of change characteristic with the preset rate of change threshold. Specifically, if the rate of change characteristic is less than the first rate of change threshold, the frame interval is increased; if the rate of change characteristic is greater than the second rate of change threshold, the frame interval is decreased; otherwise, the frame interval remains unchanged.

[0084] For example, a first rate of change threshold can be set to 0.1, and a second rate of change threshold can be set to 0.5. When the calculated rate of change characteristic is less than 0.1, the frame interval is increased from 1 / 30 of a second to 1 / 15 of a second; when the rate of change characteristic is greater than 0.5, the frame interval is decreased from 1 / 30 of a second to 1 / 60 of a second; when the rate of change characteristic is between 0.1 and 0.5, the frame interval remains unchanged at 1 / 30 of a second.

[0085] Through the above technical solution, this application achieves adaptive adjustment of the frame interval. When the psychological state is relatively stable, the sampling frequency is reduced to decrease computational load; when the psychological state changes rapidly, the sampling frequency is increased to capture more details, thereby optimizing system resource utilization efficiency while ensuring monitoring accuracy. Furthermore, dynamic frame interval adjustment can adapt to the micro-expression changes of different individuals, improving the personalization and accuracy of psychological risk warnings.

[0086] This application further proposes a method for calculating the rate of change characteristic as follows:

[0087]

[0088] in, Indicates the rate of change characteristic. This indicates the baseline risk value for the current monitoring window. This indicates the baseline risk value for the previous monitoring window. Monitoring window duration.

[0089] The rate of change characteristic is normalized by dividing the absolute difference of the baseline risk values ​​of adjacent monitoring windows by the monitoring window duration, thus eliminating the influence of different window durations on the magnitude of change. The baseline risk value of the current monitoring window reflects the latest risk level, while the baseline risk value of the previous monitoring window represents the historical state; the difference between the two reflects the intensity of risk fluctuations. The monitoring window duration is used as the denominator to convert the absolute difference into the rate of change per unit time, ensuring the comparability of calculation results under different monitoring periods.

[0090] Specifically, the rate of change per unit time is calculated by continuously monitoring the difference in baseline risk values ​​between windows, thus quantifying the severity of risk level changes. When the rate of change characteristic is below a first rate of change threshold, it indicates that risk fluctuations are mild, and the frame interval is increased to reduce image processing overhead. When the rate of change characteristic is above a second rate of change threshold, it indicates that risk fluctuations are severe, and the frame interval is decreased to improve the real-time performance of feature capture. The monitoring window duration serves as a time benchmark, ensuring that the rate of change calculation results are decoupled from the time dimension, adapting to the dynamic adjustment needs of different monitoring cycles. Therefore, the system can adaptively optimize the image acquisition frequency based on the real-time speed of risk changes, balancing computational resource consumption and monitoring sensitivity.

[0091] Through the above technical solution, this application achieves precise quantification of changes in basic risk values ​​between continuous monitoring windows. The introduction of the rate of change feature enables the system to promptly capture the dynamic trend of risk level changes, providing an objective basis for subsequent frame interval adjustments. This adaptive mechanism improves the system's sensitivity to micro-expression changes, optimizing the utilization efficiency of computing resources while ensuring monitoring accuracy. Furthermore, the frame interval dynamic adjustment strategy based on the rate of change feature allows the system to appropriately reduce the sampling frequency to save resources when the risk level is relatively stable, while promptly increasing the sampling frequency to capture more details when the risk level changes rapidly, thus achieving a balance between the real-time performance and accuracy of psychological risk warning.

[0092] This application further proposes to extract the coupled event timestamps of all eyebrow-to-eyebrow events and mouth-to-mouth events; calculate the temporal phase ratio feature of mouth-to-mouth actions taking precedence over eyebrow-to-eyebrow actions; and when the temporal phase ratio feature is greater than a preset phase threshold, increase the output risk level.

[0093] The extraction of coupled event timestamps is achieved by recording the occurrence time of each eyebrow-between event and its associated corner-of-the-mouth event; the temporal phase ratio feature is defined as the ratio of the number of coupled events whose corner-of-the-mouth event timestamp is earlier than the eyebrow-between event timestamp to the total number of coupled events, calculated as follows:

[0094]

[0095] in, Indicates the characteristics of timing phase ratio, This indicates the number of coupled events whose timestamps are earlier than the timestamps of the events between the eyebrows. Indicates the total number of coupled events;

[0096] Risk level upgrade strategies include adjusting low-risk signals to medium-risk, or medium-risk signals to high-risk.

[0097] Specifically, within the monitoring window, when a coupled event is detected, the timestamps of the eyebrow-related event and the associated mouth-related event are recorded. By comparing the timestamp order of the two events in each coupled event, the number of cases where the mouth-related action precedes the eyebrow-related action is counted. The ratio of this number to the total number of coupled events is used as the temporal phase proportion feature. For example, if the total number of coupled events is 20, and 12 of them are mouth-related events preceding eyebrow-related events, then the temporal phase proportion feature is 0.6. When this ratio exceeds a preset phase threshold (e.g., 0.55), it indicates that the mouth-related action more frequently serves as the initial trigger signal for emotional reactions, potentially reflecting a stronger negative psychological state. In this case, the risk level corresponding to the base risk value is increased by one level. By introducing a quantitative indicator of event temporal priority, the dynamic triggering patterns in micro-expression coordination can be captured more accurately, thereby enhancing the sensitivity of identifying high-risk psychological states.

[0098] As a preferred embodiment, the solution of this application is implemented as follows: When a coupled event between the eyebrows and the corner of the mouth is detected within the monitoring window, the timestamp of each event is recorded in real time using a facial key point tracking algorithm. Specifically, when the start timestamp of the eyebrow event and the start timestamp of the corresponding corner of the mouth event satisfy a temporal relationship, a correlation is established between the two. Further, a sliding time window algorithm is used to perform temporal phase analysis on the coupled events, and the number of cases where the corner of the mouth event occurs earlier than the eyebrow event is counted. For example, when the timestamp of a detected eyebrow event is t=3250ms, if a corner of the mouth event exists within the range of t=3240ms to t=3250ms, it is determined that the corner of the mouth action occurs first in this coupled event. The temporal phase ratio feature is calculated accordingly, and when this feature exceeds a preset threshold of 0.65, the original medium-risk warning signal is automatically upgraded to a high-risk warning signal.

[0099] Through the above technical solution, this application effectively solves the technical deficiency of existing technologies that ignore the temporal correlation patterns between micro-expression events in different facial regions. By quantifying the temporal priority relationship between mouth corner movements and eyebrow movements, it is possible to more accurately capture the subconscious physiological reaction characteristics in emotional expression, thereby enhancing the ability to identify feigned expressions. Furthermore, dynamically adjusting the risk level based on the temporal phase ratio characteristics can significantly improve the accuracy of early warning for specific psychological states such as depressive tendencies.

[0100] Example 2:

[0101] like Figure 3 As shown, the psychological risk early warning system for micro-expression image sequences includes:

[0102] An image acquisition module is used to acquire a continuous sequence of facial images, the image sequence being extracted from a video stream at preset frame intervals;

[0103] The feature extraction module is used to calculate the eyebrow displacement features and mouth corner deformation features between adjacent frames based on the facial image sequence using a facial key point tracking algorithm;

[0104] The event marking module is used to mark eyebrow events based on the comparison result of the eyebrow displacement feature and a first preset threshold, and to mark mouth events based on the comparison result of the mouth corner deformation feature and a second preset threshold.

[0105] The event statistics module is used to count the number of occurrences of the eyebrow-between event and the corner of the mouth event within a preset monitoring window, and to calculate the event coupling coefficient based on the temporal correlation between the eyebrow-between event and the corner of the mouth event;

[0106] The risk calculation module is used to perform weighted fusion calculations on the number of eyebrow events, the number of mouth events, and the event coupling coefficient to obtain a basic risk value.

[0107] The risk assessment module is used to determine whether the basic risk value is less than a preset risk threshold. If it is, a low-risk warning signal is output and the facial image sequence is reacquired; otherwise, the entropy calculation process is initiated.

[0108] The entropy calculation module is used to calculate the entropy feature based on the probability distribution of micro-expression event types within the monitoring window, and to determine whether the entropy feature is greater than a preset entropy threshold. If it is, a high-risk warning signal is output; otherwise, a medium-risk warning signal is output.

[0109] The above description is merely an example and illustration of the structure of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the structure of the invention or exceed the scope defined in the claims, all of which should fall within the protection scope of the present invention.

[0110] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0111] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method of psychological risk alerting of a micro-expression image sequence, characterized in that, The method comprises the following steps: obtaining a continuous facial image sequence extracted from a video stream at a preset frame interval; calculating the inter-frame eyebrow displacement feature and the mouth corner deformation feature based on the facial image sequence through facial key point tracking; labeling the inter-brow event according to the comparison result of the inter-brow displacement feature and a first preset threshold, and labeling the mouth corner event according to the comparison result of the mouth corner deformation feature and a second preset threshold; within a preset monitoring window, counting the number of occurrences of the inter-brow event and the mouth corner event, and calculating the event coupling coefficient according to the time correlation of the inter-brow event and the mouth corner event; performing weighted fusion calculation on the number of occurrences of the inter-brow event, the number of occurrences of the mouth corner event, and the event coupling coefficient to obtain a basic risk value; judging whether the basic risk value is less than a preset risk threshold, and if so, outputting a low-risk warning signal and re-obtaining the facial image sequence, otherwise calculating an entropy value feature according to the probability distribution of the micro-expression event type within the monitoring window, judging whether the entropy value feature is greater than a preset entropy value threshold, and if so, outputting a high-risk warning signal, otherwise outputting a medium-risk warning signal.

2. The micro-expression image sequence psychological risk warning method according to claim 1, characterized in that: the inter-brow displacement feature is calculated by the change in eyebrow arch key point coordinates, and the mouth corner deformation feature is calculated by the change in distance between corner key points; the calculation process of the event coupling coefficient includes: for each inter-brow event, detecting whether there is an associated mouth corner event within a preset time window; the ratio of the number of inter-brow events with associated events to the total number of inter-brow events is taken as the event coupling coefficient.

3. The micro-expression image sequence psychological risk pre-warning method according to claim 1, characterized in that: the weighted fusion calculation process includes: taking the ratio of the number of inter-brow events to the length of the monitoring window as the inter-brow event frequency; taking the ratio of the number of mouth corner events to the length of the monitoring window as the mouth corner event frequency; fusing the inter-brow event frequency, the mouth corner event frequency, and the event coupling coefficient through a weight coefficient to obtain a basic risk value.

4. The micro-expression image sequence psychological risk pre-warning method according to claim 1, characterized in that: the calculation process of the entropy value feature includes: counting the probability of independent inter-brow events, mouth corner events, and coupled events within the monitoring window; the coupled event is a mouth corner event occurring within a preset time window when each independent inter-brow event occurs; the entropy value feature is calculated by the probability distribution disorder degree, and the calculation method is: wherein, is an entropy value feature, K represents a set of event types, represents a probability of occurrence of the corresponding event type.

5. The micro-expression image sequence psychological risk early warning method according to claim 1, characterized in that: further comprising dynamically adjusting the monitoring window: obtaining the average duration of inter-brow actions and the average duration of mouth corner actions in the individual baseline library; scaling the adaptive time window according to the maximum value of the average duration; when calculating the event coupling coefficient, the adaptive time window is used instead of the preset monitoring window. the calculation method of the adaptive time window is: wherein, denotes an adaptive time window, denotes an average duration of glabella action, average duration of mouth corner action, is a preset scaling coefficient, is a maximum function.

6. The micro-expression image sequence psychological risk early warning method according to claim 1, characterized in that: further comprising: calculating the change rate feature of the basic risk value between consecutive monitoring windows, and dynamically adjusting the frame interval according to the comparison result of the change rate feature and a preset change rate threshold: if the change rate feature is less than a first change rate threshold, increase the frame interval; if the change rate feature is greater than a second change rate threshold, decrease the frame interval; otherwise, keep the frame interval unchanged.

7. The micro-expression image sequence psychological risk pre-alarm method according to claim 6, characterized in that: the calculation method of the change rate feature is: wherein, represents a rate of change characteristic, represents a current monitoring window base risk value, represents a previous monitoring window base risk value, represents monitoring window duration.

8. The micro-expression image sequence psychological risk early warning method according to claim 1, characterized in that: further comprising: extracting the coupling event timestamps of all inter-brow events and mouth corner events; A time phase proportion feature of a mouth corner action being prior to a glabella action is calculated; When the time phase proportion feature is greater than a preset phase threshold, a risk level of an output is raised.

9. The micro-expression image sequence psychological risk pre-alarm method according to claim 8, characterized in that: The time phase proportion feature is calculated in the following way: wherein, represents a timing phase proportion bit feature, represents a number of coupling events where the mouth corner event timestamp is earlier than the inter eyebrow event timestamp, represents a total number of coupling events.

10. A psychological risk alert system for micro-expression image sequences, characterized by: The micro-expression image sequence is used in a psychological risk early warning method, and the method comprises the following steps: An image acquisition module is configured to acquire a continuous facial image sequence, and the image sequence is extracted from a video stream at a preset frame interval; A feature extraction module is configured to calculate a glabella displacement feature and a mouth corner deformation feature between adjacent frames based on the facial image sequence by using a facial key point tracking algorithm; An event marking module is configured to mark a glabella event according to a comparison result of the glabella displacement feature and a first preset threshold, and mark a mouth corner event according to a comparison result of the mouth corner deformation feature and a second preset threshold; An event statistics module is configured to count the number of occurrences of the glabella event and the mouth corner event within a preset monitoring window, and calculate an event coupling coefficient according to the time correlation between the glabella event and the mouth corner event; A risk calculation module is configured to perform weighted fusion calculation on the number of occurrences of the glabella event, the number of occurrences of the mouth corner event, and the event coupling coefficient to obtain a basic risk value; A risk judgment module is configured to judge whether the basic risk value is less than a preset risk threshold, and if so, output a low-risk early warning signal and reacquire the facial image sequence; otherwise, proceed to an entropy value calculation process; An entropy value calculation module is configured to calculate an entropy value feature according to the probability distribution of micro-expression event types within the monitoring window, and judge whether the entropy value feature is greater than a preset entropy value threshold, and if so, output a high-risk early warning signal; otherwise, output a medium-risk early warning signal.

Images