Multi-dimensional fusion psychological index early warning method and device and electronic equipment

Abstract

The invention provides a multi-dimensional fusion psychological index early warning method and device and electronic equipment. The method comprises the steps that a physiological index numerical value, a behavior performance numerical value, a face recognition numerical value and a report questionnaire numerical value of a target object are acquired; performing weighted calculation on the physiological index value, the behavior performance value, the face recognition value and the report questionnaire value based on a preset weight coefficient to obtain a comprehensive mood index of the target object; and performing early warning on psychological indexes of the target object based on the comprehensive mood index and a preset mood index threshold. The accuracy of psychological index early warning is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of psychological analysis, and in particular to a multi-dimensional fusion psychological indicator early warning method, device and electronic equipment. Background Art

[0002] At present, the most common way to investigate psychology is to ask the subjects to manually report their current mood (such as anger, calmness, happiness), or to use functional magnetic resonance imaging or positron emission tomography to determine the relevant physical signs of the subjects. However, a single measurement method and technical means cannot guarantee the accuracy of the results. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a multi-dimensional fusion psychological indicator early warning method, device and electronic equipment to improve the accuracy of psychological indicator early warning.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0005] In a first aspect, the present invention provides a multi-dimensional fusion psychological indicator warning method, comprising: obtaining physiological indicator values, behavioral performance values, facial recognition values ​​and report questionnaire values ​​of a target object; performing weighted calculation on the physiological indicator values, behavioral performance values, facial recognition values ​​and report questionnaire values ​​based on a pre-set weight coefficient to obtain a comprehensive mood index of the target object; and issuing a warning on the psychological indicators of the target object based on the comprehensive mood index and a pre-set mood index threshold.

[0006] Optionally, obtaining the target object's physiological indicator values, behavioral performance values, facial recognition values ​​and report questionnaire values ​​includes: obtaining the target object's physiological indicators, and calculating the physiological indicator values ​​corresponding to the physiological indicators; wherein the physiological indicators include at least: the target object's heart rate, blood pressure, pupil diameter and pupil area; obtaining the target object's behavioral indicators, and calculating the behavioral performance values ​​corresponding to the behavioral indicators; wherein the behavioral indicators include at least: the target object's breathing frequency, breathing depth, audio data and behavior category; obtaining the target object's facial image, and performing expression recognition on the facial image to obtain a facial recognition value; obtaining the target object's report questionnaire results, and performing weighted calculation on the report questionnaire results to obtain the target object's report questionnaire values; wherein the report questionnaire results include: survey report results and self-report results.

[0007] Optionally, calculating the physiological indicator values ​​corresponding to the physiological indicators includes: determining the heart rate analysis value of the target object based on a predetermined correspondence between the heart rate and the heart rate analysis value; determining the blood pressure analysis value of the target object based on a predetermined correspondence between the blood pressure and the blood pressure analysis value; determining the mood state of the target object based on a predetermined correspondence between the pupil diameter and the pupil area and the mood state, and determining the pupil analysis value of the target object based on the predetermined correspondence between the mood state and the analysis value; standardizing the heart rate analysis value, the blood pressure analysis value and the pupil analysis value, and performing weighted calculation on the standardized heart rate analysis value, the blood pressure analysis value and the pupil analysis value to obtain the physiological indicator values ​​corresponding to the physiological indicators.

[0008] Optionally, behavioral indicators of the target object are obtained, and behavioral performance values ​​corresponding to the behavioral indicators are calculated, including: performing respiration detection on the target object, obtaining the respiration frequency and respiration depth of the target object, calculating the PAD value of the target object based on the respiration frequency and respiration depth, and determining the respiration analysis value of the target object based on the PAD value; performing human voice detection on the target object, obtaining an audio waveform of the target object, and determining the mood state of the target object based on predetermined audio waveforms of different moods, and determining the audio analysis value of the target object based on the correspondence between the predetermined mood state and the analysis value; obtaining video data of the target object, and identifying the behavior category of the target object based on the video data to obtain the behavior analysis value of the target object; standardizing the respiration analysis value, audio analysis value, and behavior analysis value, and performing weighted calculation on the standardized respiration analysis value, audio analysis value, and behavior analysis value to obtain the behavior performance value corresponding to the behavioral indicator.

[0009] Optionally, determining the breathing analysis value of the target object based on the PAD value includes: calculating a first distance between the PAD value and a predetermined PAD value of a pleasant state, and calculating a second distance between the PAD value and a predetermined PAD value of a depressed state; determining the mood state of the target object based on the first distance and the second distance, and determining the breathing analysis value of the target object based on the mood state of the target object.

[0010] Optionally, video data of the target object is obtained, and the behavior category of the target object is identified based on the video data to obtain a behavior analysis value of the target object, including: obtaining video data of the target object, and extracting skeleton data of the target object in the video data through human posture estimation technology; based on the skeleton data, using human bone point posture evaluation analysis to identify the preliminary action type of the target object; identifying objects in the video data through target detection technology, and obtaining the category information and spatial position information of the objects; based on the spatial position information of the target object and the object, identifying the interaction behavior between the target object and the object; determining the behavior category of the target object based on the preliminary action type, the type information of the object and the interaction behavior between the target object and the object; determining the behavior analysis value of the target object based on the correspondence between the predetermined behavior category and the behavior analysis value.

[0011] Optionally, facial expression recognition is performed on a facial image to obtain a facial recognition value, including: preprocessing the facial image and extracting expression features from the preprocessed facial image using an independent component analysis method; inputting the expression features into a K-nearest neighbor classifier to obtain a predicted mood state, and determining the facial recognition value of the target object based on a correspondence between a predetermined mood state and the analysis value.

[0012] In a second aspect, the present invention provides a multi-dimensional fusion psychological indicator warning device, including: a value acquisition module, used to obtain the physiological indicator values, behavioral performance values, facial recognition values ​​and report questionnaire values ​​of the target object; a comprehensive mood index calculation module, used to perform weighted calculation on the physiological indicator values, behavioral performance values, facial recognition values ​​and report questionnaire values ​​based on a preset weight coefficient to obtain the comprehensive mood index of the target object; an early warning module, used to warn the psychological indicators of the target object based on the comprehensive mood index and a preset mood index threshold.

[0013] In a third aspect, the present invention provides an electronic device comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement the steps of any one of the methods provided in the first aspect above.

[0014] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any one of the methods provided in the first aspect are executed.

[0015] The present invention brings the following beneficial effects:

[0016] The above-mentioned multi-dimensional fusion psychological index early warning method, device and electronic device provided by the present invention first obtain the target object's physiological index value, behavioral performance value, facial recognition value and report questionnaire value; then weighted calculation is performed on the physiological index value, behavioral performance value, facial recognition value and report questionnaire value based on a pre-set weight coefficient to obtain the target object's comprehensive mood index; finally, the target object's psychological index is warned based on the comprehensive mood index and the pre-set mood index threshold. The above method combines the values ​​of the four dimensions of physiological indicators, behavioral indicators, facial indicators and report questionnaire indicators, obtains the target object's comprehensive mood index according to different weight coefficients, and combines multiple comprehensive mood index measurement results and mood index thresholds for early warning, thereby improving the accuracy of psychological indicator early warning.

[0017] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0018] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the preferred embodiments are specifically mentioned below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A flowchart of a multi-dimensional fusion psychological indicator early warning method provided by an embodiment of the present invention;

[0021] Figure 2 A mood change curve provided by an embodiment of the present invention;

[0022] Figure 3 An audio waveform diagram of different emotions provided by an embodiment of the present invention;

[0023] Figure 4 A flow chart of behavior recognition provided by an embodiment of the present invention;

[0024] Figure 5 A schematic diagram of the spatial position relationship between a human body and an object provided by an embodiment of the present invention;

[0025] Figure 6A schematic diagram of a hierarchical analysis method provided by an embodiment of the present invention;

[0026] Figure 7 A schematic diagram of the structure of a multi-dimensional fusion psychological indicator early warning device provided by an embodiment of the present invention;

[0027] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] At present, the existing methods of psychological examination have the following shortcomings and problems: (1) The manual registration method cannot be guaranteed in terms of authenticity and effectiveness. Usually, due to the influence of the understanding of mood and the honesty of mentality, it is impossible to judge whether the results are true and reliable, which greatly reduces the effectiveness of the obtained data. (2) Existing academic theories have not yet formed a universal and highly accurate single way to identify mood. (3) Through the method of neuroimaging measurement, it is difficult to achieve frequent and regular measurement and collection in special environments, and it is impossible to guarantee that the measurement results are 100% accurate and reliable. (4) The single data analysis model cannot make complete and accurate judgments in actual complex scenarios.

[0030] Based on this, the embodiments of the present invention provide a multi-dimensional fusion psychological indicator early warning method, device and electronic equipment, which can improve the accuracy of psychological indicator early warning.

[0031] To facilitate understanding of this embodiment, a multi-dimensional fusion psychological indicator early warning method disclosed in an embodiment of the present invention is first introduced in detail. The method can be executed by an electronic device, such as a smart phone, a computer, a tablet computer, etc. Figure 1 The flowchart of a multi-dimensional fusion psychological indicator early warning method shown in FIG. 1 illustrates that the method mainly includes the following steps S101 to S103:

[0032] Step S101: Obtain physiological index values, behavioral performance values, facial recognition values, and report questionnaire values ​​of the target object.

[0033] In one embodiment, a common and simple method can be used to measure the physiological indicator dimension, that is, the skin conductance level (SCL) or short-term skin conductance response (SCRs) is quantified by common methods such as heart rate (HR) and blood pressure (BP), and the face of the target object is photographed by a high-precision camera for pupil detection, and then the physiological indicator value is obtained according to the heart rate, blood pressure and pupil detection results. For the behavioral indicator dimension, the behavioral performance value can be obtained through breathing detection, human voice audio detection and morphological recognition comparison. For the facial indicator dimension, the expression features of the target object can be extracted through facial recognition, and the mood state of the target object can be determined according to the table features, and then the facial recognition value can be obtained. For the report questionnaire indicator dimension, the mood data can be obtained by allowing the target object to independently select the current state and answer the comprehensive questionnaire, and then the report questionnaire value can be obtained. In an embodiment of the present invention, the target object can be a tested person from various industries such as students and teachers.

[0034] Step S102: Perform weighted calculation on physiological index values, behavioral performance values, facial recognition values ​​and report questionnaire values ​​based on pre-set weight coefficients to obtain a comprehensive mood index of the target object.

[0035] In one embodiment, corresponding weight coefficients can be set in advance according to the recognition accuracy of different dimensions. After obtaining the values ​​of each dimension, the physiological index values, behavioral performance values, facial recognition values ​​and report questionnaire values ​​are weighted according to the set weight coefficients to obtain the comprehensive mood index of the target object. The weight coefficients of each dimension are shown in Table 1.

[0036] Table 1 Comparison table of weight coefficients of each dimension

[0037] Weight Code Weight Name Weight coefficient <![CDATA[C1]]> Physiological indicators 0.1 <![CDATA[C2]]> Behavioral manifestations 0.3 <![CDATA[C3]]> Facial Recognition 0.5 <![CDATA[C4]]> Report Questionnaire 0.1

[0038] In the embodiment of the present invention, the scene characteristics and the factors that may affect the mood are classified, and different dimensions are obtained by different measurement methods (i.e., physiological index values, behavioral performance values, facial recognition values, and report questionnaire values). In the embodiment of the present invention, the target object's comprehensive mood index Y is composed of measurement and analysis scores of four different dimensions, namely: G1 (physiological index values), G2 (behavioral performance values), G3 (facial recognition values), G4 (report questionnaire values), and the values ​​of multiple components of each dimension are not uniform, and they need to be standardized and weighted, and then the Y value is finally calculated through the weight coefficients of different dimensions.

[0039] Y=Q1C1+Q2C2+Q3C3+Q4C4

[0040] Among them, Q1, Q2, Q3, and Q4 are physiological indicator values, behavioral performance values, facial recognition values, and report questionnaire values, respectively.

[0041] Step S103: issuing an early warning of the psychological index of the target object based on the comprehensive mood index and a preset mood index threshold.

[0042] In one embodiment, through step S101 and step S102, the comprehensive mood index of the target object can be measured once, and the comprehensive mood index can be saved. In this embodiment, the historical data in the target object information collection file can be retrieved, multiple comprehensive mood indexes can be screened and arranged according to the time axis to obtain the mood change curve of the target object within a specific time. Figure 2 Then, the curve data trend is analyzed through the set warning threshold (i.e., mood index threshold). If the mood index threshold is exceeded, it indicates that the target object has psychological problems and timely intervention measures are needed.

[0043] The above-mentioned multi-dimensional fusion psychological indicator early warning method provided by the present invention combines the values ​​of four dimensions of physiological indicators, behavioral indicators, facial indicators and report questionnaire indicators, obtains the comprehensive mood index of the target object according to different weight coefficients, and combines multiple comprehensive mood index measurement results and mood index thresholds for early warning, thereby improving the accuracy of psychological indicator early warning.

[0044] In one embodiment, for the aforementioned step S101, i.e., obtaining the physiological index values, behavioral performance values, facial recognition values, and report questionnaire values ​​of the target object, the following methods may be used, including but not limited to:

[0045] (1) Acquire physiological indicators of the target object and calculate physiological indicator values ​​corresponding to the physiological indicators; wherein the physiological indicators at least include: heart rate, blood pressure, pupil diameter and pupil area of ​​the target object.

[0046] In a specific implementation, calculating the physiological indicator value corresponding to the physiological indicator includes the following steps (1.1) to (1.4):

[0047] Step (1.1): Determine the heart rate analysis value of the target object based on a predetermined correspondence between the heart rate and the heart rate analysis value.

[0048] Step (1.2): Determine the blood pressure analysis value of the target object based on the predetermined correspondence between the blood pressure and the blood pressure analysis value.

[0049] Specifically, studies have shown that people's heart rate is fastest when they are angry or afraid, followed by when they are happy. Sadness and prosperity make the heart rate slow down, and the heart rate reaches its lowest point when they are disgusted. Heart rate variability is also an indicator closely related to one's mood state. It refers to the change in the speed of the heart rate rhythm over time, reflecting the time difference between the primary and secondary cardiac cycles. A lower HRV indicates a relaxed state, while an enhanced HRV may indicate a state of mental tension and frustration.

[0050] Based on this, in the embodiment of the present invention, the heart rate and blood pressure of the target object can be measured by a common and simple method, and the heart rate analysis value of the target object can be obtained according to the predetermined correspondence between the heart rate and the heart rate analysis value, and the blood pressure analysis value of the target object can be obtained according to the predetermined correspondence between the blood pressure and the blood pressure analysis value. Among them, the correspondence between the heart rate and the heart rate analysis value can be determined according to the heart rate of the object in different mood states, and a corresponding heart rate analysis value is set in advance for each mood state, so that the correspondence between the heart rate and the heart rate analysis value can be obtained according to the heart rate in different mood states. Similarly, the correspondence between the blood pressure and the blood pressure analysis value can be determined according to the blood pressure of the object in different mood states, and a corresponding blood pressure analysis value is set in advance for each mood state, so that the correspondence between the blood pressure and the blood pressure analysis value can be obtained according to the blood pressure in different mood states.

[0051] Step (1.3): Determine the target object's mood state based on the predetermined correspondence between pupil diameter and pupil area and mood state, and determine the target object's pupil analysis value based on the predetermined correspondence between mood state and analysis value.

[0052] Specifically, as shown in Table 2, different mood states are associated with pupil diameter and area. Therefore, in the embodiment of the present invention, a high-precision camera can be used to shoot the face of the target object to perform pupil detection, obtain the pupil diameter and pupil area, and determine the mood state of the target object according to Table 2, and then obtain the pupil analysis value of the target object according to the correspondence between the predetermined mood state and the analysis value.

[0053] Table 2 Correspondence between pupil size and mood state

[0054] Group n Size Alteration Happy 23 5.34±1.41 3.24±1.99 calm 8 3.50±1.25 1.62±1.25 anxiety 18 317±0.86 1.01±0.89 impatient 11 4.91±1.81 2.86±1.93

[0055] Step (1.4): Standardize the heart rate analysis values, blood pressure analysis values, and pupil analysis values, and perform weighted calculation on the standardized heart rate analysis values, blood pressure analysis values, and pupil analysis values ​​to obtain physiological indicator values ​​corresponding to the physiological indicators.

[0056] Specifically, since the values ​​of heart rate analysis, blood pressure analysis and pupil analysis are not uniform under the physiological indicator dimension, it is necessary to standardize the heart rate analysis values, blood pressure analysis values ​​and pupil analysis values ​​to ensure that they are comparable at the same level; then, weighted calculation is performed on the standardized heart rate analysis values, blood pressure analysis values ​​and pupil analysis values ​​to obtain the physiological indicator values ​​corresponding to the physiological indicators.

[0057] (2) Obtaining behavioral indicators of the target object and calculating behavioral performance values ​​corresponding to the behavioral indicators; wherein the behavioral indicators include at least: breathing frequency, breathing depth, audio data and behavior category of the target object.

[0058] In the specific implementation, the behavior index of the target object is obtained, and the behavior performance value corresponding to the behavior index is calculated, including the following steps (2.1) to (2.4):

[0059] Step (2.1): Perform breathing detection on the target object, obtain the breathing frequency and breathing depth of the target object, calculate the PAD value of the target object based on the breathing frequency and breathing depth, and determine the breathing analysis value of the target object based on the PAD value.

[0060] In one embodiment, the breathing frequency can reflect changes in mood. When the human body is in a state of tension or fear, the number of breaths will significantly increase and the breathing depth will become shallower. In the embodiment of the present invention, the face of the target object can be photographed by a thermal imaging camera, and the frequency can be identified by the thermal air flow exhaled under the nose, and the breathing frequency and breathing depth can be calculated. Then, the PAD value of the target object is calculated based on the breathing frequency and breathing depth, and finally the breathing analysis value is obtained based on the PAD value.

[0061] Furthermore, when determining the breathing analysis value of the target object based on the PAD value, the following methods may be used, including but not limited to: first calculating the distance between the PAD value and the PAD value of a predetermined reference state; then determining the mood state of the target object based on the distance, and determining the breathing analysis value of the target object based on the mood state of the target object.

[0062] In specific implementation, according to the PAD emotional model, the emotional state can be quantified using three dimensions: pleasure, arousal, and dominance. First, the PAD value of the target object is calculated based on the breathing frequency and breathing depth, and the distance between the PAD value of the target object and the PAD value of the reference state is calculated based on the set PAD value of the reference state. Then, the mood state of the target object is determined based on the distance, that is, the smaller the distance value, the closer the target object is to its corresponding mood state. Finally, the breathing analysis value of the target object is determined based on the mood state of the target object, that is, the distance corresponding to the mood state of the target object is determined as the breathing analysis value.

[0063] The specific steps of breathing detection are as follows:

[0064] Step 1: Data collection.

[0065] First, use a thermal imaging camera to capture the target object's face to obtain thermal image data, focusing on the temperature changes in the area under the nose. When breathing, the periodic changes in thermal airflow will be displayed as temperature fluctuations on the thermal map. Then, the continuous thermal image data is converted into a signal curve of temperature changes over time, and the breathing depth is estimated based on the signal curve. The breathing depth is related to the amplitude of temperature changes during exhalation. Deeper breathing is usually accompanied by obvious temperature changes. By analyzing the amplitude of temperature fluctuations, the breathing depth can be estimated.

[0066] Step 2: Calculate breathing rate and identify breathing depth.

[0067] The formula for calculating the breathing rate (BRR) is:

[0068]

[0069] Where B is the total number of breaths during the measurement period and T is the total time (in minutes) for counting breaths.

[0070] The method of identifying the depth of breathing is to analyze the intensity and range of thermals through thermal imaging. Deep breathing produces stronger thermals, while shallow breathing produces weaker thermals. Specifically, the depth of breathing can be identified by setting the threshold of thermal intensity and range.

[0071] Step 3: Quantify personal psychological and emotional indicators.

[0072] According to the PAD emotional model, emotional state can be quantified using three dimensions: pleasure, arousal, and dominance.

[0073] Pleasure P value: reflects the positive and negative aspects of the user's emotional state.

[0074] Activation A value: reflects the user's neurophysiological activation level and excitement level.

[0075] Dominance D value: reflects the strength of the mutual dominance between users and the external environment.

[0076] Step 4: PAD value calculation.

[0077] Pleasure (P) is usually associated with slower, regular breathing, so in the embodiment of the present invention, a linear mapping relationship inversely proportional to the standardized breathing frequency can be used for calculation. The calculation formula is:

[0078] P = a × (1-f norm )+b

[0079] Among them, P represents the degree of pleasure, a and b are constants or coefficients, and f norm It is a standardized respiratory rate index, which can be obtained by standardizing the respiratory rate of the target object.

[0080] Arousal (A) is usually associated with a higher respiratory rate and is proportional to the normalized respiratory rate. It is calculated as:

[0081] A=c×f norm +d

[0082] Among them, A represents the activation, c and d are constants or coefficients, and f norm It is a standardized respiratory rate indicator.

[0083] Dominance (D) is related to breathing depth. Assume there is an indicator representing breathing depth, and the value range of this indicator is 0 to 1. Its calculation formula is:

[0084] D=e×depth index+f

[0085] Wherein, D represents dominance, e and f are constants or coefficients, and the "depth index" is a variable that quantifies the depth of breathing. Specifically, the breathing intensity interval can be divided according to the intensity and range of the thermal flow, and each interval is corresponded to the range of 0-1. The larger the depth index, the deeper the breathing. In the embodiment of the present invention, the corresponding depth index can be determined according to the intensity and range of the thermal flow generated when the target object breathes.

[0086] Step 5: PAD value and target object mood state derivation.

[0087] First, based on experience or evaluation data, define two reference standards, such as a happy state and a depressed state, which can be represented by the values ​​of P, A, and D. For example, a happy state: (P, A, D) = (0.8, 0.5, 0.7), a depressed state: (P, A, D) = (0.2, 0.5, 0.7).

[0088] Then, the Euclidean distance comparison method is used for the next step of derivation. Among them, the Euclidean distance calculation formula is:

[0089] d = (P1 - P2) 2 + (A1 - A2) 2 + (D1 - D2) 2

[0090] Among them, (P1, A1, D1) is the PAD value of the target object, and (P2, A2, D2) is the PAD value of the reference state.

[0091] Suppose the current mood state of the target object is obtained through the above steps, and the specific PAD value is: (P, A, D) = (0.33, 0.33, 0.7).

[0092] Calculate the distance between the target object and the pleasant state:

[0093] d1 = (0.33 - 0.8) 2 + (0.33 - 0.5) 2 + (0.7 - 0.7) 2 ≈0.4998

[0094] Calculate the distance between the target object and the depressed state:

[0095] d2 = (0.33 - 0.2) 2 + (0.33 - 0.5) 2 + (0.7 - 0.7) 2 ≈0.214

[0096] Furthermore, compare the two distances d1 and d2 to determine whether the current mood state of the target object is closer to the pleasant state or the depressed state. Since d2 < d1, the current mood state of the target object is closer to the depressed state, and d2 is determined as the respiration analysis value.

[0097] Step (2.2): Perform voice detection on the target object, obtain the audio waveform diagram of the target object, and determine the mood state of the target object based on the audio waveform diagrams of different moods determined in advance, and determine the audio analysis value of the target object based on the corresponding relationship between the mood state determined in advance and the analysis value.

[0098] In specific implementation, refer to Figure 3As shown, when a person expresses different emotions, there are differences in audio intonation data. Based on this, in an embodiment of the present invention, the target object can be asked to read a text to obtain an audio waveform of the target object, and then the audio waveform of the target object can be compared with the audio waveforms of different moods to determine the waveform closest to the audio waveform of the target object, thereby obtaining the mood state of the target object, and finally determining the audio analysis value of the target object according to the corresponding relationship between the mood state and the analysis value.

[0099] Step (2.3): Obtain video data of the target object, and identify the behavior category of the target object based on the video data to obtain a behavior analysis value of the target object.

[0100] In specific implementation, by analyzing the behavior and psychological state of the target object, its potential emotional state can be identified. Based on this theory, in the embodiment of the present invention, computer vision technology can be used to capture the behavior of the target object and identify its behavior and duration. Specifically, behavior recognition includes: first, obtaining the video data of the target object, and extracting the skeleton data of the target object in the video data through human posture estimation technology; secondly, based on the skeleton data, using human skeleton point posture evaluation analysis to identify the preliminary action type of the target object; then identifying the object in the video data through target detection technology, and obtaining the category information and spatial position information of the object; then based on the spatial position information of the target object and the object, identifying the interaction between the target object and the object; then based on the preliminary action type, the type information of the object and the interaction between the target object and the object, determining the behavior category of the target object; finally, determining the behavior analysis value of the target object based on the correspondence between the predetermined behavior category and the behavior analysis value.

[0101] See also Figure 4 As shown, firstly, the original video data is obtained from the camera, and the skeleton data of the human body is extracted through the human posture estimation technology. Then, the method of human skeleton point posture evaluation and analysis is applied to identify the individual's preliminary action type; at the same time, the key objects in the video scene are identified through the target detection technology, and the category and spatial position information of the objects are obtained; then, the spatial position relationship between the human body and the object is analyzed to identify the interaction behavior between the person and the object; finally, by comprehensively considering the preliminary action classification, the specific information of the object and the interaction relationship between the person and the object, the decision-making level is integrated to obtain an accurate behavior recognition conclusion.

[0102] The human skeleton point posture assessment includes: the human skeleton point positioning technology aims to determine the accurate position of the key parts of the human body in the image. In the embodiment of the present invention, this process can be implemented by a real-time posture detection model based on deep learning, using the VGG-19 convolutional neural network architecture to extract key features in the image, and assigning these features to two different processing paths, each undergoing an iterative learning process of T stages. In the first path, the system generates a confidence set S = (S1, S2, ..., Sj, ..., SJ), where S contains each element from S1 to SJ, each element Sj corresponds to the confidence of the jth human skeleton point, and J represents the total number of skeleton points; and in the second path, the system outputs an affinity set L = (L1, L2, ..., Lc, ..., LC), containing each element from L1 to LC, each element Lc represents the affinity of the cth connection between human skeleton points, and C is the total number of connections. After identifying the skeleton points, the affinity set L is used for high-precision clustering analysis to achieve accurate recognition of human posture.

[0103] The loss function of the confidence branch (denoted by S) at training stage t Defined as:

[0104]

[0105] Where W is a binary mask. When the skeleton point annotation information of point p in the image is lost, W(p) is 0. Represents the true value of the confidence S.

[0106] The loss function of the affinity branch (denoted by L) at training stage t Defined as:

[0107]

[0108] in, Represents the true value of affinity L.

[0109] can be and The global loss function f is calculated and expressed as:

[0110]

[0111] The spatial position relationship recognition between the human body and the object includes: Figure 5 As shown in the figure, by identifying the skeleton points, we can get two people sitting on a chair, with one hand close to the head, which is similar to thinking. However, this information alone is not enough to analyze the accurate behavior. Therefore, the embodiment of the present invention proposes a method of human skeleton point recognition and spatial position calculation between objects to further enhance the accuracy of the analysis results. Figure 5In the scenario, the distance between the two people and the table (d), the distance between the elbow joint and the table (e), and the distance between the palm and the head (c) are comprehensively analyzed and calculated, so as to judge that the two people are playing chess based on the distance values.

[0112] Decision fusion includes: according to the pre-set rules, the decision fusion module is used to fuse and judge the initial behavior category of the personnel, the object category, and the human-object interaction relationship to obtain the final human behavior category. Figure 5 Taking the target object playing chess as an example, Table 3 is a comparison table of the set behavior categories and target values. The decision fusion strategy can be expressed as follows:

[0113] Table 3 Comparison table of behavior categories and target values

[0114]

[0115]

[0116] 1) When the category number of the initial behavior is 0, it indicates that the initial behavior category does not need to be modified.

[0117] 2) When the initial behavior category number is 1 or 2, the spatial position between the human body and the object will affect the final behavior category.

[0118] 3) When the table is not detected, the behavior category is sitting and thinking.

[0119] 4) When the table is detected and the distance relationship among d, e, and c satisfies the pre-made model value, the behavior category is modified to playing chess while sitting.

[0120] In addition, in the embodiment of the present invention, a visual big model can be used to assist decision-making. The video stream screen is synchronously transmitted to the visual big model, and the visual big model infers the content of the screen and gives the picture understanding content and the corresponding confidence level; the content and confidence level are synchronously transmitted to the decision fusion link, and the final result is output after setting a certain threshold.

[0121] Step (2.4): Standardize the respiratory analysis values, audio analysis values, and behavioral analysis values, and perform weighted calculation on the standardized respiratory analysis values, audio analysis values, and behavioral analysis values ​​to obtain behavioral performance values ​​corresponding to the behavioral indicators.

[0122] Specifically, since the values ​​of respiratory analysis, audio analysis, and behavioral analysis are not uniform under this behavioral performance dimension, it is necessary to standardize the respiratory analysis values, audio analysis values, and behavioral analysis values ​​to ensure that they are comparable at the same level; then, weighted calculation is performed on the standardized respiratory analysis values, audio analysis values, and behavioral analysis values ​​to obtain the behavioral performance values ​​corresponding to the behavioral indicators.

[0123] (3) Obtain a facial image of the target object, perform expression recognition on the facial image, and obtain a facial recognition value.

[0124] In specific implementation, when performing expression recognition on a face image and obtaining a facial recognition value, the following methods may be used, including but not limited to:

[0125] First, the face image is preprocessed, and the independent component analysis method is used to extract expression features from the preprocessed face image. Specifically, after the initial detection and capture of the target object's face using a camera, the collected face image is subjected to geometric preprocessing such as correction, scaling, cutting, and grayscale histogram. Then, the JAFFE standard expression library that conforms to Asian characteristics is used to extract expression features through independent component analysis (ICA).

[0126] Then, the expression features are input into the K-nearest neighbor classifier to obtain the predicted mood state, and the facial recognition value of the target object is determined based on the correspondence between the predetermined mood state and the analysis value. Specifically, after the result is calculated by the FastICA algorithm, it is combined with the K-nearest neighbor classifier to obtain the predicted mood state corresponding to the expression features of the target object, and finally the facial recognition value of the target object is determined based on the correspondence between the determined mood state and the analysis value.

[0127] Specifically, facial recognition includes the following steps:

[0128] Step 1: Data preparation.

[0129] Specifically, data preparation includes: data collection, preprocessing, and data formatting to ensure that the data is suitable for subsequent feature extraction and classification processing.

[0130] Evaluation and selection of datasets: Based on the characteristics of Asians and the maturity of the dataset, the JAFFE (Japanese Female Facial Expression) dataset is used, which contains facial expressions under 7 basic emotions (happiness, surprise, anger, disgust, sadness, fear, and neutral).

[0131] Image preprocessing: Before performing expression recognition, images need to be preprocessed to ensure that the size, lighting conditions, and facial position of each image are consistent. Common preprocessing steps include:

[0132] Grayscale: Convert a color image to a grayscale image to simplify calculations.

[0133] Normalization: Scale pixel values ​​to the range of 0 to 1.

[0134] Alignment and cropping: Detect facial feature points through facial detection algorithms (such as Haar Cascades), crop the face area, and align the face to ensure that the facial expression area is consistent.

[0135] Data formatting: Convert all preprocessed images into vectorized format. Each image is converted into a one-dimensional vector, and the dimension of the vector is equal to the number of pixels in the image (for example, a 64x64 image is converted into a 4096-dimensional vector).

[0136] The image dataset will form a matrix of size (m×n), where m is the number of images and n is the number of pixels in each image. If there are 100 facial images in the dataset and each image is 64x64 pixels in size, then the dataset is a (100×4096) matrix.

[0137] Step 2: Feature extraction.

[0138] Feature extraction is one of the core tasks in the expression recognition process. By extracting key features from facial expressions, the data dimension can be reduced while retaining key information related to emotions. ICA (Independent Component Analysis) is an effective feature extraction method that aims to extract independent components from mixed data, which can usually capture the main structure in the data. For facial expression data, ICA can extract independent facial features (for example, changes in the mouth, opening and closing of the eyes, etc.) from image data. Feature extraction mainly includes:

[0139] A: Center the image data matrix (i.e., subtract the mean of each feature), the formula is:

[0140] X centered =XE[X]

[0141] Where X represents the original data matrix X centered represents the centered data matrix, and E[X] represents the expected value (or mean) of X.

[0142] B: Whitening. Whitening means performing a linear transformation on the data so that the variance of each dimension is 1 and the covariance is 0. In the embodiment of the present invention, principal component analysis (PCA) is used for whitening, and the formula is:

[0143] X whitened =WX centered

[0144] Among them, X whitened represents the data matrix after whitening, W represents the whitening matrix, X centered Represents the centered data matrix.

[0145] C: Apply the ICA algorithm to extract independent components. By using algorithms such as FastICA, the independent component matrix {S} is calculated, which is the key expression features extracted from the facial expression image. Each independent component usually corresponds to a certain expression feature of the image. For example, if the ICA algorithm is used to extract 10 independent components, each independent component may correspond to an expression feature such as a raised mouth or raised eyebrows. The calculation formula of FastICA is:

[0146] W←E[X g (w T X)]-E[g t (w T X)]w

[0147] Among them, w represents the weight vector, X represents the data matrix, and g t represents the activation function, g t is the derivative of the activation function, and E represents the expected value.

[0148] Step 3: Classification modeling.

[0149] The classification modeling step inputs the extracted expression features into the K-nearest neighbor (K-NN) classifier to predict the mood corresponding to the current facial expression. K-NN is an instance-based learning method that performs classification by measuring the distance between the input data and the training data set. The key to K-NN is to choose the appropriate K value and distance measurement method.

[0150] The K-NN calculation steps include:

[0151] A. Choose a suitable K value. A common approach is to select the optimal K through cross-validation.

[0152] B. For each sample to be classified, calculate its distance from each sample in the training set. The commonly used distance metric is Euclidean distance, and the formula is:

[0153]

[0154] Among them, d(x i , x j ) represents the sample x i and x j The Euclidean distance between ik and x jk They are samples x i and x j The value at the kth dimension, where n is the total number of dimensions.

[0155] C. Select K training samples with the smallest distance, vote according to the categories of these neighbors, and finally determine the classification result.

[0156] For example, for the test sample x t , assuming K = 3, and the three nearest neighbors calculated belong to the categories of "happy", "angry" and "happy" respectively, then the final classification result is "happy".

[0157] D. Perform model training and prediction classification, using facial expression features extracted from the JAFFE dataset (independent components extracted by ICA) as input features, and use K-NN classifier for training. Specifically, use labeled data (i.e., the emotion label of each image) as the training set to build a K-NN model. When a new facial expression image is obtained, it is first subjected to the same preprocessing and feature extraction (ICA) steps to obtain the expression feature vector, and then input into the trained K-NN classifier to obtain the predicted emotion label.

[0158] Step 4: Result analysis.

[0159] By comparing with the real emotion labels in the JAFFE dataset, the accuracy of the model is calculated:

[0160]

[0161] Through the above steps, combined with the JAFFE dataset, ICA feature extraction and K-NN classifier, the mood state corresponding to facial expressions can be effectively identified and predicted. Experimental verification shows that this method can achieve high classification accuracy on multiple emotion categories, indicating its effectiveness in practical applications.

[0162] The following will combine the above steps to perform the calculation process from facial feature extraction to target object mood prediction:

[0163] Assume that the 5-dimensional feature vector data of the target object is: [0.9, 0.8, 0.2, 0.1, 0.5], and the known training set feature data vector is:

[0164] x1=[0.8,0.9,0.1,0.2,0.6],tab:Pleasure 1

[0165] x2=[0.7,0.6,0.2,0.1,0.4],tab:Pleasure 2

[0166] x3=[0.2,0.1,0.8,0.9,0.3],tab:Frustrated

[0167] In the K-NN classifier, K=3 is selected, and the distance sequence between the test sample and each training sample is calculated using the Euclidean distance: {0.141, 0.223, 1.204}. Since there are two values ​​in the sequence that belong to the category of pleasure, it is judged that the current target object's mood state is close to pleasure.

[0168] (4) Obtaining the report questionnaire results of the target object, and performing weighted calculation on the report questionnaire results to obtain the report questionnaire value of the target object; wherein the report questionnaire results include: survey report results and self-report results.

[0169] In one embodiment, there are two items under the report questionnaire dimension: self-report and questionnaire. Mood data is obtained by allowing the target subject to independently select the current state and answer the questionnaire; wherein, in the questionnaire method, the evaluation can be performed through online questions (mainly focusing on the current status) and offline questions (mainly focusing on estimating the current mood state by recalling long-term memory events). Then, the report questionnaire results of the two items are weighted and calculated to obtain the report questionnaire value of the target subject.

[0170] In the embodiment of the present invention, for the four dimensions G1, G2, G3, and G4, it is assumed that the relationship between the sub-items constituting G1, G2, G3, and G4 is:

[0171] G1 consists of G11 (pupil analysis value), G12 (heart rate analysis value), and G13 (blood pressure analysis value); G2 consists of G21 (respiration analysis value), G22 (audio analysis value), and G23 (behavior analysis value); G3 has no sub-items; G4 consists of G41 (questionnaire analysis value) and G42 (self-report analysis value). The formula for standardization is:

[0172]

[0173] Among them, x is the original value, μ is the mean, σ is the standard deviation, and z is the standardized value. The weighted summation formula is:

[0174] G i =w i1 ·z i1 +w i2 ·z i2 +…+w in ·z in

[0175] Among them, G i is the indicator value of the i-th dimension, w i1 is the weight of the first sub-item in the i-th dimension, z i1 It is the normalized value of the first item in the i-th dimension.

[0176] See also Figure 6As shown, the method provided by the embodiment of the present invention uses the analytic hierarchy process (AHP) to model the data, samples the indicator data of different dimensions, and sets the weight coefficient according to the reliability of the dimension, so that the comprehensive data is more realistic, which solves the problem that the authenticity of the data cannot be judged simply by manual reporting; at the same time, the method is suitable for measuring data in a special environment in a way that is less harmful to the human body and easier to operate, and ensures the health and safety of the subject under the premise of multi-frequency measurement; finally, the above-mentioned methods apply target detection, spatial relationship between the human body and the object, and skeletal posture evaluation, thereby improving the accuracy of the detection algorithm.

[0177] For the multi-dimensional fusion psychological indicator early warning method provided in the above embodiment, the embodiment of the present invention also provides a multi-dimensional fusion psychological indicator early warning device, see Figure 7 The schematic diagram of the structure of a multi-dimensional fusion psychological indicator early warning device is shown, indicating that the device mainly includes the following parts:

[0178] The value acquisition module 701 is used to acquire the physiological index values, behavioral performance values, facial recognition values ​​and report questionnaire values ​​of the target object;

[0179] A comprehensive mood index calculation module 702 is used to perform weighted calculation on physiological index values, behavioral performance values, facial recognition values ​​and report questionnaire values ​​based on a preset weight coefficient to obtain a comprehensive mood index of the target object;

[0180] The early warning module 703 is used to issue an early warning to the psychological index of the target object based on the comprehensive mood index and the preset mood index threshold.

[0181] The above-mentioned multi-dimensional fusion psychological indicator early warning device provided by the present invention combines the values ​​of four dimensions of physiological indicators, behavioral indicators, facial indicators and report questionnaire indicators, obtains the comprehensive mood index of the target object according to different weight coefficients, and combines multiple comprehensive mood index measurement results and mood index thresholds for early warning, thereby improving the accuracy of psychological indicator early warning.

[0182] It should be noted that the implementation principle and technical effects of the device provided in the embodiment of the present invention are the same as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference can be made to the corresponding contents in the aforementioned method embodiment. The specific numerical values ​​provided in the implementation of the present invention are only exemplary and are not limited here.

[0183] An embodiment of the present invention further provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above implementation methods.

[0184] Figure 8 A structural diagram of an electronic device provided in an embodiment of the present invention, the electronic device 100 includes: a processor 80, a memory 81, a bus 82 and a communication interface 83, wherein the processor 80, the communication interface 83 and the memory 81 are connected via the bus 82; the processor 80 is used to execute an executable module stored in the memory 81, such as a computer program.

[0185] The memory 81 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 83 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0186] The bus 82 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0187] Among them, the memory 81 is used to store programs, and the processor 80 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any embodiment of the above-mentioned embodiments of the present invention can be applied to the processor 80 or implemented by the processor 80.

[0188] The processor 80 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 80. The above processor 80 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 81, and the processor 80 reads the information in the memory 81 and completes the steps of the above method in combination with its hardware.

[0189] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be referred to the previous method embodiments, which will not be repeated here.

[0190] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0191] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A multi-dimensional fusion psychological indicator early warning method, characterized in that: include: Obtain the target object's physiological index values, behavioral performance values, facial recognition values, and report questionnaire values; Performing weighted calculation on the physiological index value, the behavioral performance value, the facial recognition value and the report questionnaire value based on a preset weight coefficient to obtain a comprehensive mood index of the target object; An early warning is given to the psychological index of the target object based on the comprehensive mood index and a preset mood index threshold.

2. The method according to claim 1, characterized in that: Obtain the target object's physiological index values, behavioral performance values, facial recognition values, and report questionnaire values, including: Acquire physiological indicators of the target object, and calculate physiological indicator values ​​corresponding to the physiological indicators; wherein the physiological indicators at least include: heart rate, blood pressure, pupil diameter and pupil area of ​​the target object; Obtaining the behavior index of the target object, and calculating the behavior performance value corresponding to the behavior index; wherein the behavior index at least includes: the breathing frequency, breathing depth, audio data and behavior category of the target object; Acquire a facial image of the target object, and perform expression recognition on the facial image to obtain a facial recognition value; The report questionnaire results of the target object are obtained, and the report questionnaire results are weighted to obtain the report questionnaire value of the target object; wherein the report questionnaire results include: survey report results and self-report results.

3. The method according to claim 2, characterized in that Calculating the physiological indicator value corresponding to the physiological indicator, including: Determining the heart rate analysis value of the target object based on a predetermined correspondence between the heart rate and the heart rate analysis value; Determining the blood pressure analysis value of the target object based on a predetermined correspondence between the blood pressure and the blood pressure analysis value; Determining the target object's mood state based on a predetermined correspondence between the pupil diameter and the pupil area and the mood state, and determining the target object's pupil analysis value based on a predetermined correspondence between the mood state and the analysis value; The heart rate analysis value, the blood pressure analysis value and the pupil analysis value are standardized, and the standardized heart rate analysis value, blood pressure analysis value and pupil analysis value are weightedly calculated to obtain physiological indicator values ​​corresponding to the physiological indicators.

4. The method according to claim 2, characterized in that: Obtaining a behavior indicator of the target object and calculating a behavior performance value corresponding to the behavior indicator includes: Performing respiration detection on the target object, obtaining the respiration frequency and respiration depth of the target object, calculating the PAD value of the target object based on the respiration frequency and the respiration depth, and determining the respiration analysis value of the target object based on the PAD value; Performing human voice detection on the target object, obtaining an audio waveform of the target object, and determining the mood state of the target object based on the predetermined audio waveforms of different moods, and determining the audio analysis value of the target object based on the correspondence between the predetermined mood state and the analysis value; Acquire video data of the target object, and identify the behavior category of the target object based on the video data to obtain a behavior analysis value of the target object; The breathing analysis value, the audio analysis value and the behavior analysis value are standardized, and the standardized breathing analysis value, audio analysis value and behavior analysis value are weighted to obtain a behavior performance value corresponding to the behavior indicator.

5. The method according to claim 4, characterized in that Determining a respiratory analysis value of the target object based on the PAD value includes: Calculating a distance between the PAD value and a PAD value of a predetermined reference state; The mood state of the target object is determined based on the distance, and a breathing analysis value of the target object is determined based on the mood state of the target object.

6. The method according to claim 4, characterized in that Acquiring video data of the target object, and identifying the behavior category of the target object based on the video data to obtain a behavior analysis value of the target object, including: Acquire video data of the target object, and extract skeleton data of the target object in the video data by using human posture estimation technology; Based on the skeleton data, a human skeleton point posture evaluation analysis is used to identify the preliminary action type of the target object; Identify objects in the video data by using target detection technology, and obtain category information and spatial location information of the objects; Based on the spatial position information of the target object and the object, identifying the interaction behavior between the target object and the object; Determining a behavior category of the target object based on the preliminary action type, the type information of the object, and the interaction behavior between the target object and the object; The behavior analysis value of the target object is determined based on a predetermined correspondence between the behavior category and the behavior analysis value.

7. The method according to claim 2, characterized in that: Performing facial expression recognition on the face image to obtain a facial recognition value includes: Preprocessing the facial image, and extracting expression features from the preprocessed facial image using an independent component analysis method; The expression feature is input into a K-nearest neighbor classifier to obtain a predicted mood state, and the facial recognition value of the target object is determined based on a correspondence between a predetermined mood state and an analysis value.

8. A multi-dimensional fusion psychological indicator early warning device, characterized in that: include: A value acquisition module is used to obtain the physiological index values, behavioral performance values, facial recognition values ​​and report questionnaire values ​​of the target object; A comprehensive mood index calculation module, used to perform weighted calculation on the physiological index value, the behavioral performance value, the facial recognition value and the report questionnaire value based on a preset weight coefficient to obtain a comprehensive mood index of the target object; The early warning module is used to warn the psychological index of the target object based on the comprehensive mood index and a preset mood index threshold.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are performed.

Images