Psychological state recognition method and device based on face micro-expression recognition

Abstract

The embodiment of the invention provides a psychological state recognition method and device based on face micro-expression recognition, and the method comprises the steps: determining an initial sample group, and obtaining initial face micro-expression data based on the initial sample group; performing data annotation on the initial face micro-expression data, and determining annotated data; performing data preprocessing based on the annotation data, and determining target face micro-expression data; performing feature extraction on the target face micro-expression data to determine micro-expression features; training the initial classifier model based on the micro-expression features, and determining a target classifier model; and obtaining test features, inputting the test features into the target classifier model, and determining the psychological state. By means of computer vision and machine learning algorithms, micro-expression features are extracted from the face image, correlation analysis is carried out on the micro-expression features and the psychological state, and rapid and accurate psychological state recognition is achieved.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of mental health technology, and in particular to a method for identifying a mental state based on facial micro-expression recognition. Background Art

[0002] Mental state recognition is one of the important areas of psychology research. Traditional mental state recognition methods usually rely on the experience and observation of psychological counselors or psychiatrists, but this method is subjective and has errors. In recent years, with the development of computer vision and deep learning technology, mental state recognition technology based on facial micro-expressions has gradually become a research hotspot.

[0003] Mental state is the external manifestation of people's internal mental state, which is of great significance for the assessment and intervention of mental health. Traditional mental state identification methods often rely on questionnaires and interviews, which are subjective and time-consuming. Therefore, a better solution is urgently needed. Summary of the invention

[0004] In view of this, the embodiments of this specification provide a method for identifying a psychological state based on facial micro-expression recognition. One or more embodiments of this specification also relate to a psychological state identification device based on facial micro-expression recognition, a computing device, a computer-readable storage medium, and a computer program to solve the technical defects existing in the prior art.

[0005] According to a first aspect of an embodiment of this specification, a method for identifying a psychological state based on facial micro-expression recognition is provided, comprising: Determine an initial sample population, and obtain initial facial micro-expression data based on the initial sample population; Annotate the initial facial micro-expression data and determine the annotated data; Perform data preprocessing based on the labeled data to determine the target facial micro-expression data; Extract features from the target person’s facial micro-expression data and determine the micro-expression features; Train the initial classifier model based on the micro-expression features and determine the target classifier model; Obtain test features, input the test features into the target classifier model, and determine the psychological state.

[0006] In a possible implementation, initial facial micro-expression data is obtained based on an initial sample population, including: Determine multiple acquisition channels, multiple angle parameters and multiple orientation parameters; Based on multiple angle parameters and multiple orientation parameters, the initial facial micro-expression data is acquired through multiple acquisition channels; wherein the initial facial micro-expression data includes facial muscle movement data, facial expression data and eye data.

[0007] In a possible implementation, data preprocessing is performed based on the labeled data to determine the target facial micro-expression data, including: Perform image enhancement on the labeled data to determine enhanced image data; Performing rotation, scaling and flipping operations based on the enhanced image data to determine the diversity image data; Perform data cleaning on diverse image data to determine the target facial micro-expression data.

[0008] In a possible implementation, feature extraction is performed on the target facial micro-expression data to determine the micro-expression features, including: The convolutional neural network is used to extract the features of the target facial micro-expression data and determine the sequence features; The sequence data is modeled through a recurrent neural network to determine the micro-expression features.

[0009] In a possible implementation, feature extraction is performed on the target facial micro-expression data to determine the micro-expression features, including: Perform key point detection on the target facial micro-expression data to determine the initial key points; Position and refine the target key points based on the initial key points; Determine the position information of the target key point, and associate the position information with the key point to determine the position key point; Local features are generated based on orientation key points to determine micro-expression features.

[0010] In a possible implementation, the initial classifier model is trained based on the micro-expression features to determine the target classifier model, including: Build a model based on the neural network model and labeled data to determine the initial classifier model; Determine the loss function, iteratively adjust the parameters of the initial classifier model through the gradient descent algorithm and the loss function, and determine the target classifier model.

[0011] In a possible implementation, after determining the target classifier model, the method further includes: Evaluate the target classifier model through a cross-validation method to determine the performance data of the target classifier model; When the performance data is less than the performance threshold, the target classifier model is optimized.

[0012] According to a second aspect of the embodiments of this specification, a psychological state recognition device based on facial micro-expression recognition is provided, comprising: An initial data determination module is configured to determine an initial sample population and obtain initial facial micro-expression data based on the initial sample population; A labeling data determination module is configured to label the initial facial micro-expression data and determine the labeling data; A data preprocessing module is configured to perform data preprocessing based on the labeled data to determine the target facial micro-expression data; A feature extraction module is configured to extract features from the target facial micro-expression data and determine micro-expression features; A classifier training module is configured to train an initial classifier model based on micro-expression features to determine a target classifier model; The psychological state determination module is configured to obtain test features, input the test features into the target classifier model, and determine the psychological state.

[0013] According to a third aspect of an embodiment of this specification, a computing device is provided, including: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the above-mentioned psychological state recognition method based on facial micro-expression recognition are implemented.

[0014] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer-executable instructions, and when the instructions are executed by a processor, the steps of the above-mentioned psychological state recognition method based on facial micro-expression recognition are implemented.

[0015] According to a fifth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned method for psychological state recognition based on facial micro-expression recognition.

[0016] The embodiments of this specification provide a method and device for identifying a psychological state based on facial micro-expression recognition, wherein the method for identifying a psychological state based on facial micro-expression recognition includes: determining an initial sample population, obtaining initial facial micro-expression data based on the initial sample population; annotating the initial facial micro-expression data to determine the annotated data; performing data preprocessing based on the annotated data to determine the target facial micro-expression data; extracting features from the target facial micro-expression data to determine micro-expression features; training an initial classifier model based on micro-expression features to determine a target classifier model; obtaining test features, inputting the test features into the target classifier model, and determining the psychological state. Using computer vision and machine learning algorithms, extract micro-expression features from facial images, and perform correlation analysis with psychological states to achieve fast and accurate psychological state recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a flow chart of a method for identifying a psychological state based on facial micro-expression recognition provided by an embodiment of this specification; Figure 2 It is a schematic diagram of a psychological state recognition method based on facial micro-expression recognition provided by an embodiment of this specification; Figure 3 It is a structural schematic diagram of a psychological state recognition device based on facial micro-expression recognition provided by an embodiment of this specification; Figure 4 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0018] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0019] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0020] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0021] In this specification, a method for identifying a psychological state based on facial micro-expression recognition is provided. This specification also relates to a psychological state recognition device based on facial micro-expression recognition, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0022] See also Figure 1 , Figure 1A flowchart of a method for identifying a psychological state based on facial micro-expression recognition provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0023] Step 101: Determine an initial sample population, and obtain initial facial micro-expression data based on the initial sample population.

[0024] In a possible implementation, initial facial micro-expression data is obtained based on an initial sample population, including: determining multiple acquisition channels, multiple angle parameters, and multiple orientation parameters; based on the multiple angle parameters and the multiple orientation parameters, acquiring initial facial micro-expression data through multiple acquisition channels; wherein the initial facial micro-expression data includes facial muscle movement data, facial expression data, and eye data.

[0025] In practical applications, a large amount of facial micro-expression data containing different psychological states is first collected. During the data collection process, it is necessary to ensure the stability and consistency of the collection to ensure the accuracy and reliability of subsequent processing. At the same time, a multi-channel collection method is used to collect facial micro-expression data from multiple angles and directions to improve the diversity and reliability of the data. The data includes micro-expression features such as facial muscle movements, facial expressions, and eye contact.

[0026] Step 102: annotate the initial facial micro-expression data to determine the annotated data.

[0027] In practical applications, see Figure 2 , the collected data needs to be labeled, for example, labeled as happy and anxious.

[0028] Step 103: Perform data preprocessing based on the labeled data to determine the target facial micro-expression data.

[0029] In one possible implementation, data preprocessing is performed based on the annotated data to determine the target facial micro-expression data, including: performing image enhancement on the annotated data to determine the enhanced image data; performing rotation, scaling and flipping operations based on the enhanced image data to determine the diversity image data; and performing data cleaning on the diversity image data to determine the target facial micro-expression data.

[0030] In practical applications, the collected facial micro-expression data is normalized and preprocessed. The preprocessed data will be used for subsequent feature extraction, model training, and classifier design. In the preprocessing process, image enhancement technology is used to rotate, scale, flip, and other operations on the image to increase the diversity and complexity of the data and improve the generalization ability of the model. At the same time, data cleaning technology is also used to remove invalid and abnormal data and improve the purity and accuracy of the data.

[0031] Step 104: extracting features from the target facial micro-expression data to determine micro-expression features.

[0032] In one possible implementation, feature extraction is performed on the target facial micro-expression data to determine the micro-expression features, including: extracting features from the target facial micro-expression data through a convolutional neural network to determine sequence features; and modeling the sequence data through a recurrent neural network to determine the micro-expression features.

[0033] In practical applications, deep learning algorithms are used to extract features from preprocessed facial micro-expression data, including facial muscle movements, subtle eye movements, and micro-movements of the corners of the mouth.

[0034] These features can be automatically extracted by training a deep neural network model. In the feature extraction process, a convolutional neural network (CNN) model is used to extract the feature representation of micro-expressions from facial images. At the same time, a recurrent neural network (RNN) model is also used to model the sequence data to capture the temporal dependency of facial micro-expressions. By combining the CNN and RNN models, the feature information of facial micro-expressions can be extracted more comprehensively.

[0035] In one possible implementation, feature extraction is performed on the target facial micro-expression data to determine the micro-expression features, including: performing key point detection on the target facial micro-expression data to determine the initial key points; positioning and refining based on the initial key points to determine the target key points; determining the orientation information of the target key points, and associating the orientation information with the key points to determine the orientation key points; generating local features based on the orientation key points to determine the micro-expression features.

[0036] In practical applications, local feature technology is used to extract features around facial key points to characterize the subtle differences in expression changes. The specific algorithm is implemented as follows: Key point detection: Find "key points" in the image. These points are usually very stable and repeatable, and can remain unchanged under different scales, rotations and lighting conditions.

[0037] Key point localization and refinement: After finding the candidate key points, their positions and scales are further precisely located through an accurate fitting model.

[0038] Key point orientation: To enhance the invariance to image rotation, the image gradient or main direction around each key point is calculated and the direction information is associated with the key point.

[0039] Local feature generation: A neighborhood window is selected around each key point, the grayscale value attributes of the pixels in the neighborhood are calculated, and then these attributes are converted into a compact and comparable vector representation.

[0040] The extracted local features can be compared between different images, and the distance or similarity between the features can be calculated to determine whether the two features correspond to the same physical feature point, thereby realizing functions such as image matching.

[0041] Step 105: Train the initial classifier model based on the micro-expression features to determine the target classifier model.

[0042] In one possible implementation, an initial classifier model is trained based on micro-expression features to determine a target classifier model, including: constructing a model based on a neural network model and labeled data to determine the initial classifier model; determining a loss function, iteratively adjusting parameters of the initial classifier model through a gradient descent algorithm and the loss function to determine the target classifier model.

[0043] In practical applications, based on the extracted features, a classifier is designed to identify and classify mental states. The classifier uses a neural network machine learning algorithm. By training the classifier model, high-precision identification and classification of mental states can be achieved. In the classifier design, a multi-class classification strategy is used to classify and identify different mental states; at the same time, an ensemble learning strategy is used to improve the generalization ability and performance of the model; in addition, a transfer learning strategy is used to transfer a large number of existing labeled data sets to improve the adaptability and generalization ability of the model.

[0044] Specifically, a large amount of labeled data is used for model training. The labeled data includes facial micro-expression data and corresponding psychological state labels. By training the model, the accuracy and generalization ability of the model can be improved. The specific implementation steps are as follows: Build the model: Use a large amount of labeled data to build the model, use a neural network model as the model architecture, and initialize the model parameters.

[0045] Loss function definition: The mean square error (MSE) loss function is used to measure the difference between the model's output and the true label. The goal of the training process is to minimize the loss function.

[0046] Algorithm optimization: The gradient descent algorithm is used to adjust the parameters of the model so that the loss function reaches the minimum value. The gradient of the loss function with respect to the parameters is calculated, and the parameters are updated in the opposite direction of the gradient.

[0047] Training process: In each training batch or the entire training set, the model calculates the output based on the current parameters and evaluates the quality of the prediction through the loss function; an adaptive learning rate adjustment strategy is used to automatically adjust the learning rate according to the performance during training, thereby accelerating the convergence of the model and improving the performance of the model; the gradient of the loss function with respect to the model parameters is calculated, that is, the partial derivative of the loss function value with respect to each parameter; the back-propagation algorithm is used to calculate and accumulate the gradient layer by layer; the model parameters are updated according to the direction and size of the gradient so that the loss function moves toward the global minimum.

[0048] Iterative training: Repeat the above steps until the predetermined number of iterations is reached or the performance on the validation set no longer improves.

[0049] In a possible implementation, after determining the target classifier model, the method further includes: evaluating the target classifier model through a cross-validation method to determine performance data of the target classifier model; and optimizing the target classifier model when the performance data is less than a performance threshold.

[0050] In practical applications, the model is evaluated using test data. Evaluation indicators include accuracy, recall, F1 value, etc. By evaluating the model performance, the pros and cons of the model can be determined and optimized. In model evaluation, the cross-validation method is used to divide the data set into multiple parts for training and testing to improve the accuracy and reliability of the evaluation. At the same time, a multi-task learning strategy is used to learn and optimize different mental state recognition tasks as a whole to improve the synergy and generalization ability of the model.

[0051] Step 106: Obtain test features, input the test features into the target classifier model, and determine the psychological state.

[0052] In practical applications, this technology is used in the fields of psychological counseling, diagnosis of mental illness, mental health assessment, etc. to assist psychological counselors or psychiatrists in more accurate diagnosis and treatment of mental states; it can also be used for mental health assessments to help individuals understand their own mental health status; it can also be used on social media to understand the mental health status of the public; in the field of security monitoring, it can detect abnormal behaviors and analyze whether people's mental states in public places are stable so that corresponding measures can be taken in time to intervene and avoid possible crises; in the field of human-computer interaction, facial micro-expression recognition technology can be used to identify and analyze user emotions to improve the experience and effect of human-computer interaction.

[0053] It should be noted that the test features are features obtained based on the test population, and the processing method is the same as the above-mentioned micro-expression features, which will not be repeated here.

[0054] The embodiments of this specification provide a method and device for identifying a psychological state based on facial micro-expression recognition, wherein the method for identifying a psychological state based on facial micro-expression recognition includes: determining an initial sample population, obtaining initial facial micro-expression data based on the initial sample population; annotating the initial facial micro-expression data to determine the annotated data; performing data preprocessing based on the annotated data to determine the target facial micro-expression data; extracting features from the target facial micro-expression data to determine micro-expression features; training an initial classifier model based on micro-expression features to determine a target classifier model; obtaining test features, inputting the test features into the target classifier model, and determining the psychological state. Using computer vision and machine learning algorithms, extract micro-expression features from facial images, and perform correlation analysis with psychological states to achieve fast and accurate psychological state recognition.

[0055] Furthermore, the technology is characterized by using deep learning algorithms and neural network models to extract facial micro-expression features and design classifiers, achieving high-precision and high-quality psychological state recognition and classification. At the same time, the use of multi-task learning strategies and transfer learning strategies improves the synergy and generalization ability of the model; specifically, the following beneficial effects are achieved.

[0056] (1) A facial micro-expression feature extraction method based on deep learning algorithm is proposed, which can automatically extract micro-expression features such as facial muscle movement, facial expression, and eye contact; (2) A neural network-based classifier was designed to achieve high-accuracy automatic recognition and classification of psychological states; (3) A model training method based on a large amount of labeled data is proposed to improve the accuracy and generalization ability of the model; (4) A model evaluation method based on test data is proposed, which can determine the quality of the model and optimize it; (5) The technology is applied to the fields of psychological counseling, diagnosis of mental illness, and mental health assessment, achieving automated and standardized identification and analysis of mental states; (6) Through the research and application of this application, the work efficiency and diagnostic accuracy of psychological counselors or psychiatrists can be improved, and new ideas and methods can be provided for the development of the field of mental state identification.

[0057] Corresponding to the above method embodiment, this specification also provides an embodiment of a psychological state recognition device based on facial micro-expression recognition, Figure 3 FIG. 1 is a schematic diagram showing a structure of a psychological state recognition device based on facial micro-expression recognition provided by an embodiment of the present specification. Figure 3 As shown, the device comprises: The initial data determination module 301 is configured to determine an initial sample population and obtain initial facial micro-expression data based on the initial sample population; The annotated data determination module 302 is configured to annotate the initial facial micro-expression data and determine the annotated data; The data preprocessing module 303 is configured to perform data preprocessing based on the labeled data to determine the target facial micro-expression data; The feature extraction module 304 is configured to extract features from the target facial micro-expression data and determine micro-expression features; The classifier training module 305 is configured to train the initial classifier model based on the micro-expression features and determine the target classifier model; The mental state determination module 306 is configured to obtain test features, input the test features into the target classifier model, and determine the mental state.

[0058] In a possible implementation, initial facial micro-expression data is obtained based on an initial sample population, including: Determine multiple acquisition channels, multiple angle parameters and multiple orientation parameters; Based on multiple angle parameters and multiple orientation parameters, the initial facial micro-expression data is acquired through multiple acquisition channels; wherein the initial facial micro-expression data includes facial muscle movement data, facial expression data and eye data.

[0059] In a possible implementation, data preprocessing is performed based on the labeled data to determine the target facial micro-expression data, including: Perform image enhancement on the labeled data to determine enhanced image data; Performing rotation, scaling and flipping operations based on the enhanced image data to determine the diversity image data; Perform data cleaning on diverse image data to determine the target facial micro-expression data.

[0060] In a possible implementation, feature extraction is performed on the target facial micro-expression data to determine the micro-expression features, including: The convolutional neural network is used to extract the features of the target facial micro-expression data and determine the sequence features; The sequence data is modeled through a recurrent neural network to determine the micro-expression features.

[0061] In a possible implementation, feature extraction is performed on the target facial micro-expression data to determine the micro-expression features, including: Perform key point detection on the target facial micro-expression data to determine the initial key points; Position and refine the target key points based on the initial key points; Determine the position information of the target key point, and associate the position information with the key point to determine the position key point; Local features are generated based on orientation key points to determine micro-expression features.

[0062] In a possible implementation, the initial classifier model is trained based on the micro-expression features to determine the target classifier model, including: Build a model based on the neural network model and labeled data to determine the initial classifier model; Determine the loss function, iteratively adjust the parameters of the initial classifier model through the gradient descent algorithm and the loss function, and determine the target classifier model.

[0063] In a possible implementation, after determining the target classifier model, the method further includes: Evaluate the target classifier model through a cross-validation method to determine the performance data of the target classifier model; When the performance data is less than the performance threshold, the target classifier model is optimized.

[0064] The above is a schematic scheme of a psychological state recognition device based on facial micro-expression recognition of this embodiment. It should be noted that the technical scheme of the psychological state recognition device based on facial micro-expression recognition and the technical scheme of the psychological state recognition method based on facial micro-expression recognition belong to the same concept, and the details of the technical scheme of the psychological state recognition device based on facial micro-expression recognition that are not described in detail can all be referred to the description of the technical scheme of the psychological state recognition method based on facial micro-expression recognition.

[0065] Figure 4 The block diagram of a computing device 400 according to one embodiment of the present specification is shown. The components of the computing device 400 include but are not limited to a memory 410 and a processor 420. The processor 420 is connected to the memory 410 via a bus 430, and the database 450 is used to store data.

[0066] The computing device 400 also includes an access device 440 that enables the computing device 400 to communicate via one or more networks 460. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 440 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a world-wide interoperability for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, and a near field communication (NFC).

[0067] In one embodiment of the present specification, the above components of the computing device 400 and Figure 4 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 4 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0068] The computing device 400 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 400 may also be a mobile or stationary server.

[0069] Among them, the processor 420 is used to execute the following computer executable instructions, which, when executed by the processor, implement the steps of the above-mentioned method for identifying a psychological state based on facial micro-expression recognition. The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the above-mentioned method for identifying a psychological state based on facial micro-expression recognition belong to the same concept. The details not described in detail in the technical scheme of the computing device can be found in the description of the technical scheme of the above-mentioned method for identifying a psychological state based on facial micro-expression recognition.

[0070] An embodiment of the present specification also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the above-mentioned psychological state recognition method based on facial micro-expression recognition.

[0071] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the psychological state recognition method based on facial micro-expression recognition belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the psychological state recognition method based on facial micro-expression recognition.

[0072] An embodiment of the present specification also provides a computer program, wherein when the computer program is executed in a computer, the computer is caused to execute the steps of the above-mentioned method for psychological state recognition based on facial micro-expression recognition.

[0073] The above is a schematic scheme of a computer program of this embodiment. It should be noted that the technical scheme of the computer program and the technical scheme of the psychological state recognition method based on facial micro-expression recognition are of the same concept, and the details not described in detail in the technical scheme of the computer program can be found in the description of the technical scheme of the psychological state recognition method based on facial micro-expression recognition.

[0074] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0075] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0076] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0077] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0078] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A psychological state recognition method based on facial micro-expression recognition, characterized in that: include: Determining an initial sample population, and obtaining initial facial micro-expression data based on the initial sample population; Annotating the initial facial micro-expression data to determine the annotated data; Performing data preprocessing based on the labeled data to determine the target person's facial micro-expression data; Extracting features from the target person's facial micro-expression data to determine micro-expression features; Training an initial classifier model based on the micro-expression features to determine a target classifier model; A test feature is obtained, and the test feature is input into the target classifier model to determine the psychological state.

2. The method according to claim 1, characterized in that The obtaining of initial facial micro-expression data based on the initial sample population includes: Determine multiple acquisition channels, multiple angle parameters and multiple orientation parameters; Based on the multiple angle parameters and the multiple orientation parameters, the multiple acquisition channels are used to acquire initial facial micro-expression data; wherein the initial facial micro-expression data includes facial muscle movement data, facial expression data and eye data.

3. The method according to claim 1, characterized in that The performing data preprocessing based on the labeled data to determine the target facial micro-expression data includes: Performing image enhancement on the labeled data to determine enhanced image data; Performing rotation, scaling and flipping operations based on the enhanced image data to determine diverse image data; The diverse image data is cleaned to determine target facial micro-expression data.

4. The method according to claim 1, characterized in that: The step of extracting features from the target facial micro-expression data to determine micro-expression features includes: Extracting features of the target facial micro-expression data through a convolutional neural network to determine sequence features; The sequence data is modeled by a recurrent neural network to determine micro-expression features.

5. The method according to claim 1, characterized in that The step of extracting features from the target facial micro-expression data to determine micro-expression features includes: Performing key point detection on the target facial micro-expression data to determine initial key points; Positioning and refining the initial key points to determine the target key points; Determine the position information of the target key point, and associate the position information with the key point to determine the position key point; Local features are generated based on the orientation key points to determine micro-expression features.

6. The method according to claim 1, characterized in that The step of training the initial classifier model based on the micro-expression features to determine the target classifier model includes: Building a model based on the neural network model and the labeled data to determine an initial classifier model; A loss function is determined, and the parameters of the initial classifier model are iteratively adjusted through a gradient descent algorithm and the loss function to determine a target classifier model.

7. The method according to claim 1, characterized in that After determining the target classifier model, it also includes: Evaluate the target classifier model by a cross-validation method to determine performance data of the target classifier model; When the performance data is less than a performance threshold, the target classifier model is optimized.

8. A psychological state recognition device based on facial micro-expression recognition, characterized in that: include: An initial data determination module is configured to determine an initial sample population, and obtain initial facial micro-expression data based on the initial sample population; A labeling data determination module is configured to label the initial facial micro-expression data and determine the labeling data; A data preprocessing module is configured to perform data preprocessing based on the labeled data to determine the target facial micro-expression data; A feature extraction module is configured to extract features from the target facial micro-expression data to determine micro-expression features; A classifier training module is configured to train an initial classifier model based on the micro-expression features to determine a target classifier model; The psychological state determination module is configured to obtain test features, input the test features into the target classifier model, and determine the psychological state.

9. A computing device, characterized in that include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the computer-executable instructions are executed by the processor, the steps of the psychological state recognition method based on facial micro-expression recognition as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the method for psychological state recognition based on facial micro-expression recognition as described in any one of claims 1 to 7.

Images