Physical and psychological health evaluation method based on facial micro-vibration image

Abstract

The invention discloses a physical and psychological health evaluation method based on a facial micro-vibration image, and belongs to the technical field of artificial intelligence emotion calculation. The method comprises the following steps: acquiring a facial video, analyzing and measuring to obtain the frequency and amplitude of pixel points in facial muscles, further calculating various psychological and physiological indexes, and displaying a light ring of a facial contour through a video frame; results based on frequency and amplitude changes of pixel points in facial muscles can be visually seen; and respectively calculating different parameter models based on the obtained frequencies and amplitudes of the pixel points in the facial muscles so as to calculate each psychological index and each physiological index, namely 12 psychological indexes and 2 physiological indexes in total. According to the physical and psychological health evaluation method based on the facial micro-vibration image, psychological indexes and partial physiological indexes related to the emotion can be calculated through the vibration of facial muscles, and the accuracy of facial emotion recognition is improved by integrating multiple indexes.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence emotion computing, and in particular to a method for evaluating physical and mental health based on facial micro-vibration images. Background Art

[0002] Facial expression recognition is one of the topics that combines emotional computing with computer vision. It has broad application prospects and unique practical significance in various scenarios such as medical care, social robots, communications, and security. With the enrichment of facial expression data sets and the enhancement of computer computing power, expression recognition algorithms have also been continuously developed.

[0003] At present, the commonly used methods for facial emotion detection are: text description-based methods: using CLIPER, a framework of contrastive text-image pair pre-training method (CLIP), to automatically learn a set of text descriptors and improve the interpretability of the model; methods based on the basic structure of the face: using the connection between face AU or semantic topology map and facial area and muscle unit, or using the exaggeration of facial actions in comics, so that the model can locate the specific facial area and its changes related to the expression; attention mechanism in expression recognition: using the attention mechanism to give different weights to different facial features, improve the recognition accuracy, and obtain the importance of different facial areas and features for model learning, as well as methods using multi-core support vector machines (MKSVM) and decision trees and deep forests. Under the current conditions where the datasets for expression recognition and datasets with interpretable additional annotations are relatively limited, these methods have some limitations. For example, methods based on facial action units (AU) may require complex annotation and detection processes, which increases the computational burden. And no more effective physiological indicators are obtained from the face to improve the accuracy of emotion recognition. Summary of the invention

[0004] The purpose of the present invention is to provide a method for evaluating physical and mental health based on facial micro-vibration images, which can calculate psychological indicators and some physiological indicators related to emotions through the vibration of facial muscles, and improve the accuracy of facial emotion recognition by integrating multiple indicators.

[0005] To achieve the above object, the present invention provides a method for evaluating physical and mental health based on facial micro-vibration images, comprising the following steps:

[0006] S1, collect facial video;

[0007] S2, analyzing and measuring the frequency and amplitude of the pixels in the facial muscles;

[0008] S3, the halo of the facial contour is displayed through the video frame to intuitively see the results of the frequency and amplitude changes based on the pixels in the facial muscles;

[0009] S4. Calculate different parameter models based on the frequency and amplitude of the pixels in the facial muscles, and then calculate the psychological index and the physiological index.

[0010] Preferably, the parameter models of the psychological indicators are respectively aggression model algorithm, stress model algorithm, suspicion model algorithm, balance model algorithm, charm model algorithm, vitality model algorithm, self-regulation model algorithm, inhibition model algorithm, neuroticism model algorithm, depression model algorithm, happiness model algorithm, and extroversion model algorithm.

[0011] Preferably, the algorithm model of the physiological index is a heart rate algorithm model and a respiratory rate algorithm model.

[0012] Preferably, in S3, the vibration halo of the facial contour is obtained by an algorithm based on the frequency and amplitude of the pixels in the facial muscles, and the amplitude formula and the frequency formula are as follows:

[0013] Amplitude formula:

[0014]

[0015] Among them, x, y are the coordinates of the pixel point, U x,y,i is the amplitude of the signal at (x, y) in the i-th frame; N is the number of frames in which the amplitude component of the vibration image is accumulated;

[0016] Frequency formula:

[0017]

[0018] Among them, Δ i is the inter-frame difference of the i-th point in the image, N is the number of frames accumulated by the amplitude component of the vibration image, and F m is the frequency of the maximum value in the density of the frequency distribution histogram.

[0019] Preferably, the model formula of the psychological index parameter includes a negative emotion model, a positive emotion model and a psychophysiological emotion model.

[0020] Preferably, the formula of the negative emotion model is as follows:

[0021] The aggressive model is as follows:

[0022]

[0023] Among them, the T1 parameter is determined by the frequency histogram; F m is the frequency of the maximum value in the frequency distribution histogram density; is the average frequency in the frequency density histogram, F i is the frequency obtained in N frames for the reference number of the i-th frequency in the distribution density histogram; F inis the vibration image processing frequency; n is the reference number of N frames with inter-frame differences greater than the threshold;

[0024] The pressure model is as follows:

[0025]

[0026] Among them, the T2 parameter is determined by the degree of asymmetry of the external vibration image, that is, the degree of facial asymmetry affected by micro-movements on the left and right sides of the head; the huge difference in the amplitude and frequency of left and right head movements is a characteristic of the increased level of the T2 parameter; among them, is the total amplitude of the vibration image of the i-th row on the left side of the object, is the total amplitude of the vibration image of the i-th row on the right side of the object, yes and The maximum value between is the total frequency of the vibration image of the i-th row on the left side of the object, is the total frequency of the vibration image of the i-th row on the right side of the object, yes and The maximum value between ; n is the number of rows occupied by an image;

[0027] The anxiety model is as follows:

[0028]

[0029] Among them, the T3 parameter is determined based on the relationship between the high-frequency part of the vibration spectrum and the total power of the human head micro-motion frequency spectrum; P i (f) is the frequency distribution spectral power of vibration imaging; f max is the maximum frequency of the vibration imaging frequency distribution spectrum;

[0030] The suspicion model is as follows:

[0031] T4=T1*30%+T2*30%+T3*40%;

[0032] Among them, the T4 parameter is defined as the average of the sum of the first three conditional negative emotions T1, T2, and T3, which describes the overall level of conditional negative emotions in a person's state;

[0033] The depression model is as follows:

[0034]

[0035] Among them, the T11 parameter is determined by the frequency histogram, σ is the standard deviation of the vibration frequency in the frequency histogram, and Mean is the average value of the vibration frequency in the frequency histogram.

[0036] Preferably, the formula of the positive emotion model is as follows:

[0037] The equilibrium model is as follows:

[0038]

[0039] Among them, the T5 parameter is defined by the frequency histogram, which indicates the similarity between the current frequency histogram and the normal distribution law; K is the allocation coefficient of the obtained frequency histogram, and y' is the normal distribution density;

[0040] The charm model is as follows:

[0041]

[0042] Among them, the T6 parameter is determined by the symmetry of facial micro-movements; W li is the average amplitude value on the left side of each row of external vibration images, W ri is the average amplitude value on the right side of each row of external vibration images, |W li -W ri | is the difference between the average amplitude values ​​on the left and right sides of each row of external vibration images; C li is the maximum frequency value on the left side of each row of external vibration images, C ri is the maximum frequency value on the right side of each row of external vibration images, |C li -C ri | is the difference between the maximum frequency values ​​on the left and right sides of each line of the external vibration image, and N is the number of video frames;

[0043] The vitality model is as follows:

[0044]

[0045] Among them, the T7 parameter is determined by the frequency histogram; M is the maximum value on the frequency histogram, σ is the mean square deviation determined by the frequency histogram; Fps is the maximum vibration input frequency;

[0046] The self-balancing model is as follows:

[0047] T8=T5*60%+T6*40%;

[0048] Among them, the T8 parameter is defined as the average of the sum of conditional positive emotions T5 and T6, which describes the overall level of conditional positive emotions of a person at a given time;

[0049] The happy model is as follows:

[0050]

[0051] Among them, I is the information validity of the psychological and physiological state, E is the energy reduction characteristic of the psychological and physiological state; dI is the change in the information validity of the psychological and physiological state, and dE is the change in the energy reduction characteristic of the psychological and physiological state.

[0052] Preferably, the formula of the psychological physiological emotion model is as follows:

[0053] The inhibition model is as follows:

[0054]

[0055] Among them, the T9 parameter is in seconds and has real physical dimensions; it describes the minimum reaction time of a person to a given event or stimulus; Mean_freq is the average value of the main frequency component of the frequency histogram;

[0056] The neuroticism model is as follows:

[0057] T10 = σ(F1);

[0058] Among them, the T10 parameter represents the variance of the measured horizontal value, and F1 is the average value of the sum of pixel differences between frames.

[0059] Preferably, the negative emotion ratio Neg is calculated according to the psychological parameters T1, T2, T3, T4, T5, T6, T7, T8, T9, T10, T11 and T12 to intuitively reflect the emotional results:

[0060]

[0061] Among them, N is the average value of negative emotions, P is the average value of positive emotions, and Phy is the average value of psychological and physiological parameter indicators.

[0062] Preferably, the calculation process of the algorithm model of the physiological indicators is: based on the periodic nature of heart rate and respiratory signals, they will appear as significant frequency components in the frequency domain, and the heart rate and respiratory rate have significantly different frequency ranges; through the physiological parameter algorithm, the frequency histogram of facial muscle vibration is obtained based on the vibration image, and then the heart rate and respiratory parameters are obtained by corresponding different physiological signals to different frequency ranges.

[0063] Therefore, the present invention adopts the above-mentioned physical and mental health evaluation method based on facial micro-vibration images, which can calculate the psychological indicators and some physiological indicators related to emotions through the vibration of facial muscles, and improve the accuracy of facial emotion recognition by integrating multiple indicators.

[0064] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a facial contour halo result diagram of an embodiment of a method for evaluating physical and mental health based on facial micro-vibration images of the present invention;

[0066] Figure 2It is a flow chart of a facial contour halo algorithm of an embodiment of a method for evaluating physical and mental health based on facial micro-vibration images of the present invention;

[0067] Figure 3 It is a parameter model flow chart of psychological indicators in an embodiment of a method for evaluating physical and mental health based on facial micro-vibration images of the present invention;

[0068] Figure 4 It is a flow chart of an algorithm model of physiological indicators of an embodiment of a method for evaluating physical and mental health based on facial micro-vibration images of the present invention;

[0069] Figure 5 It is an overall flow chart of an embodiment of a method for evaluating physical and mental health based on facial micro-vibration images of the present invention. DETAILED DESCRIPTION

[0070] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0071] Unless otherwise defined, technical or scientific terms used in the present invention shall have the common meanings understood by one having ordinary skills in the field to which the present invention belongs.

[0072] Embodiment 1

[0073] The present invention can make full use of the human vestibular system. Any changes in physiological or psychological state will affect the vestibular system, and then affect the verticality of the human head. Therefore, the micro-motion parameters (frequency, amplitude) of the human head can be used as an important indicator to measure the psychological state of the human body. From a physical perspective, this mechanical head vibration is a vibration process, and the vibration parameters reflect the quantitative relationship between energy and object movement. This is the principle of using vibration imaging technology to obtain complete information on head mobility parameters, and calculating various physiological and psychological indicator parameters of the face through vibration image algorithms to achieve better detection of emotion recognition effects, improve the accuracy of facial emotion recognition, and better judge a person's psychological state.

[0074] like Figure 1 to Figure 2 As shown in the figure, first, a video with a face is input. In particular, the head and face movement amplitude in the video must be small, and no large movement can occur, otherwise the accuracy of the model parameters will be affected. Then the face is detected by the YOLO model, and frame extraction is performed. The vibration frequency and amplitude of the facial muscles in each frame are calculated using the algorithm. The halo of the facial contour is calculated using the obtained amplitude frequency, and the corresponding frequency histogram and power spectrum density are calculated. The psychophysiological parameter model provided by the algorithm is used to calculate the various psychophysiological parameter indicators. Finally, the calculated indicators are used to obtain a summary conclusion of the proportion of negative emotions, which improves the accuracy of the assessment of people's psychological conditions.

[0075] After obtaining the vibration frequency and amplitude of the facial muscles, the color of each row of contour pixels is obtained according to the frequency range; the size of the facial halo contour is determined by the relative amplitude of each row of pixels. The human body contour is obtained through an algorithm, and the pixels on the left and right sides of each row of contour are obtained. Each row that makes up the halo is then determined by the color and relative amplitude, thereby determining the halo.

[0076] like Figure 3 to Figure 4 As shown in the figure, the face in each frame is detected by the YOLO model through the incoming video file, and the detected target area is further analyzed by the algorithm, including edge detection and contour extraction. The video is frame-processed, the signal is extracted from the image sequence to obtain the frequency and amplitude of facial muscle vibration, the frequency histogram and power spectrum density PSD are calculated, and then the psychological parameters are calculated according to the psychological parameter model. Through in-depth analysis of the power spectrum density, the system can calculate a series of parameters reflecting the psychological and physiological state of the individual. For example, the system can evaluate the individual's stress level (T2_Stress) by analyzing the symmetry error of the signal, evaluate the tension level (T3_Tension) by analyzing the maximum frequency in the PSD, and evaluate the suppression ability (T9_suppress) by analyzing the average period of the signal. In addition, the system can also calculate the individual's vitality (T7_Energy), emotional state (T12_Happy) and neuroticism level (T10_Neuroticism). Through in-depth analysis of the frequency histogram, the system can calculate parameter indicators such as aggression level (T1_Aggressive), depression level (T11_Depression), and balance ability (T5_Balance).

[0077] Heart rate and respiration signals are periodic in nature, so they will show significant frequency components in the frequency domain, and heart rate and respiration rate have significantly different frequency ranges. Through the physiological parameter algorithm, the frequency histogram of facial muscle vibration can be obtained based on the vibration image, and then the heart rate and respiration parameters can be obtained by corresponding different physiological signals to different frequency ranges.

[0078] In order to more intuitively reflect the results of emotions, the proportion of negative emotions is calculated based on the above 12 psychological parameters to give a conclusive data. In summary, the present invention uses facial muscle vibration images and psychological and physiological parameter models to calculate the amplitude frequency, 12 psychological parameters and 2 physiological parameters, and displays the facial halo contour through amplitude frequency calculation, and finally calculates the proportion of negative emotions to give a summary conclusion.

[0079] Principle of physiological parameter model: Since heart rate and respiration signals are periodic in nature, they will show significant frequency components in the frequency domain, and heart rate and respiration rate have significantly different frequency ranges. Through the physiological parameter algorithm, the frequency histogram of facial muscle vibration can be obtained based on the vibration image, and then the heart rate and respiration parameters can be obtained by corresponding different physiological signals to different frequency ranges.

[0080] like Figure 5 As shown, the present invention is a method for detecting facial emotion indicators based on vibration images. By collecting facial videos, analyzing and measuring the frequency and amplitude of pixels in facial muscles, various psychological and physiological indicators are calculated, and the halo of facial contour is displayed through video frames, so that the results based on the frequency and amplitude changes of pixels in facial muscles can be intuitively seen. The present invention can play an important role in practical applications, such as security monitoring, health monitoring and other fields, and provides professionals with a powerful analysis tool.

[0081] Therefore, the present invention adopts the above-mentioned physical and mental health evaluation method based on facial micro-vibration images, which can calculate the psychological indicators and some physiological indicators related to emotions through the vibration of facial muscles, and improve the accuracy of facial emotion recognition by integrating multiple indicators.

[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for evaluating physical and mental health based on facial micro-vibration images, characterized in that: The following steps are involved: S1, collect facial video; S2, analyzing and measuring the frequency and amplitude of the pixels in the facial muscles; S3, the halo of the facial contour is displayed through the video frame to intuitively see the results of the frequency and amplitude changes based on the pixels in the facial muscles; S4. Calculate different parameter models based on the frequency and amplitude of the pixels in the facial muscles, and then calculate the psychological index and the physiological index.

2. The method for evaluating physical and mental health based on facial micro-vibration images according to claim 1, characterized in that: The parameter models of the psychological indicators are respectively an aggression model algorithm, a stress model algorithm, a suspicion model algorithm, a balance model algorithm, a charm model algorithm, a vitality model algorithm, a self-regulation model algorithm, an inhibition model algorithm, a neuroticism model algorithm, a depression model algorithm, a happiness model algorithm, and an extroversion model algorithm.

3. The method for evaluating physical and mental health based on facial micro-vibration images according to claim 1, characterized in that: The algorithm models of the physiological indicators are a heart rate algorithm model and a respiratory rate algorithm model.

4. The method for evaluating physical and mental health based on facial micro-vibration images according to claim 1, characterized in that: In S3, the vibration halo of the facial contour is obtained by an algorithm based on the frequency and amplitude of the pixels in the facial muscles. The amplitude formula and the frequency formula are as follows: Amplitude formula: Among them, x, y are the coordinates of the pixel point, U x,y,i is the amplitude of the signal at (x, y) in the i-th frame; N is the number of frames in which the amplitude component of the vibration image is accumulated; Frequency formula: Among them, Δ i is the inter-frame difference of the i-th point in the image, N is the number of frames accumulated by the amplitude component of the vibration image, and F m is the frequency of the maximum value in the density of the frequency distribution histogram.

5. The method for evaluating physical and mental health based on facial micro-vibration images according to claim 2, characterized in that: The model formula of the psychological index parameter includes a negative emotion model, a positive emotion model and a psychological and physiological emotion model.

6. The method for evaluating physical and mental health based on facial micro-vibration images according to claim 5, characterized in that: The formula of the negative emotion model is as follows: The aggressive model is as follows: Among them, the T1 parameter is determined by the frequency histogram; F m is the frequency of the maximum value in the frequency distribution histogram density; is the average frequency in the frequency density histogram, F i is the frequency obtained in N frames for the reference number of the i-th frequency in the distribution density histogram; F in is the vibration image processing frequency; n is the reference number of N frames with inter-frame differences greater than the threshold; The pressure model is as follows: Among them, the T2 parameter is determined by the degree of asymmetry of the external vibration image, that is, the degree of facial asymmetry affected by micro-movements on the left and right sides of the head; the huge difference in the amplitude and frequency of left and right head movements is a characteristic of the increased level of the T2 parameter; among them, is the total amplitude of the vibration image of the i-th row on the left side of the object, is the total amplitude of the vibration image of the i-th row on the right side of the object, yes and The maximum value between is the total frequency of the vibration image of the i-th row on the left side of the object, is the total frequency of the vibration image of the i-th row on the right side of the object, yes and The maximum value between ; n is the number of rows occupied by an image; The anxiety model is as follows: Among them, the T3 parameter is determined based on the relationship between the high-frequency part of the vibration spectrum and the total power of the human head micro-motion frequency spectrum; P i (f) is the frequency distribution spectral power of vibration imaging; f max is the maximum frequency of the vibration imaging frequency distribution spectrum; The suspicion model is as follows: T4=T1*30%+T2*30%+T3*40%; Among them, the T4 parameter is defined as the average of the sum of the first three conditional negative emotions T1, T2, and T3, which describes the overall level of conditional negative emotions in a person's state; The depression model is as follows: Among them, the T11 parameter is determined by the frequency histogram, σ is the standard deviation of the vibration frequency in the frequency histogram, and Mean is the average value of the vibration frequency in the frequency histogram.

7. The method for evaluating physical and mental health based on facial micro-vibration images according to claim 5, characterized in that: The formula of the positive emotion model is as follows: The equilibrium model is as follows: Among them, the T5 parameter is defined by the frequency histogram, which indicates the similarity between the current frequency histogram and the normal distribution law; K is the allocation coefficient of the obtained frequency histogram, and y' is the normal distribution density; The charm model is as follows: Among them, the T6 parameter is determined by the symmetry of facial micro-movements; W li is the average amplitude value on the left side of each row of external vibration images, W ri is the average amplitude value on the right side of each row of external vibration images, |W li -W ri | is the difference between the average amplitude values ​​on the left and right sides of each row of external vibration images; C li is the maximum frequency value on the left side of each row of external vibration images, C ri is the maximum frequency value on the right side of each row of external vibration images, |C li -C ri | is the difference between the maximum frequency values ​​on the left and right sides of each line of the external vibration image, and N is the number of video frames; The vitality model is as follows: Among them, the T7 parameter is determined by the frequency histogram; M is the maximum value on the frequency histogram, σ is the mean square deviation determined by the frequency histogram; Fps is the maximum vibration input frequency; The self-balancing model is as follows: T8=T5*60%+T6*40%; Among them, the T8 parameter is defined as the average of the sum of conditional positive emotions T5 and T6, which describes the overall level of conditional positive emotions of a person at a given time; The happy model is as follows: Among them, I is the information validity of the psychological and physiological state, E is the energy reduction characteristic of the psychological and physiological state; dI is the change in the information validity of the psychological and physiological state, and dE is the change in the energy reduction characteristic of the psychological and physiological state.

8. The method for evaluating physical and mental health based on facial micro-vibration images according to claim 5, characterized in that: The formula of the psychophysiological emotion model is as follows: The inhibition model is as follows: Among them, the T9 parameter is in seconds and has real physical dimensions; it describes the minimum reaction time of a person to a given event or stimulus; Mean_freq is the average value of the main frequency component of the frequency histogram; The neuroticism model is as follows: T10 = σ(F1); Among them, the T10 parameter represents the variance of the measured horizontal value, and F1 is the average value of the sum of pixel differences between frames.

9. The method for evaluating physical and mental health based on facial micro-vibration images according to claim 5, characterized in that: The negative emotion ratio Neg is calculated based on the psychological parameters T1, T2, T3, T4, T5, T6, T7, T8, T9, T10, T11 and T12 to intuitively reflect the emotional results: Among them, N is the average value of negative emotions, P is the average value of positive emotions, and Phy is the average value of psychological and physiological parameter indicators.

10. The method for evaluating physical and mental health based on facial micro-vibration images according to claim 3, characterized in that: The calculation process of the algorithm model of the physiological index is as follows: based on the periodic nature of heart rate and respiratory signals, significant frequency components will appear in the frequency domain, and heart rate and respiratory rate have significantly different frequency ranges; Through the physiological parameter algorithm, the frequency histogram of facial muscle vibration is obtained based on the vibration image, and then the heart rate and breathing parameters are obtained by corresponding different physiological signals to different frequency ranges.

Images