Micro-expression recognition method based on improved cuckoo algorithm

Abstract

The present invention discloses a micro-expression recognition method based on an improved cuckoo algorithm, which uses a convolutional neural network face recognition technology to identify the face area of ​​an image collected by a camera, performs face image preprocessing, extracts image features, and feeds a facial feature vector set into a deep learning classifier based on the improved cuckoo algorithm. The classifier generates an initial population, and sets the target optimization function as the accuracy of image classification. When the maximum number of iterations or the termination condition is reached, the optimal feature subset and model parameters are output and applied. When the maximum number of iterations or the termination condition is not reached, the feedback optimization method based on the improved cuckoo algorithm is used for feedback optimization. The present invention uses the improved cuckoo algorithm to optimize the deep learning model, obtains a model with better performance for real-time monitoring, and finally uses the nearest neighbor algorithm for emotional abnormal point detection and conducts risk level research and judgment of psychological problems according to emotional components, so as to conduct scientific and reasonable mental health assessment.

Description

Technical Field

[0001] The present invention relates to micro-expression classification and recognition technology, and in particular to a micro-expression recognition method based on an improved cuckoo algorithm. Background Art

[0002] Micro-expressions are a type of facial expression that flashes in an instant. They are extremely subtle facial movements in both time and space, and can reveal a person's true emotions. Micro-expressions can also reflect psychological states. Micro-expression recognition can be used to determine the risk level of psychological problems and conduct mental health assessments.

[0003] Traditional micro-expression recognition methods are generally based on manual features, such as local binary patterns, optical flow histograms, gradient histograms, etc., to achieve micro-expression analysis. However, these methods rely too much on prior knowledge, and the extracted information is mostly superficial and lacks abstract features to characterize micro-expressions.

[0004] In the field of micro-expression recognition, the more mainstream classifiers cannot achieve a good balance between the number of model parameters and recognition accuracy. Models with good recognition accuracy have too many parameters to be deployed on small devices, while lightweight models do not have good recognition effects. The field of micro-expression recognition has a short development time and insufficient technical accumulation, which makes the introduction of more advanced algorithms in the field of micro-expression recognition a very urgent topic. Summary of the invention

[0005] In order to solve the deficiencies in the prior art, the object of the present invention is to provide a micro-expression recognition method based on an improved cuckoo algorithm.

[0006] To achieve the purpose of the present invention, the technical solution adopted by the present invention is:

[0007] A micro-expression recognition method based on an improved cuckoo algorithm comprises the following steps:

[0008] (1) Using camera equipment to collect images in real time;

[0009] (2) Use convolutional neural network face recognition technology to identify the face area of ​​the image collected by the camera to obtain a face image;

[0010] (3) Perform face image preprocessing;

[0011] (4) Extract image features from the preprocessed face image;

[0012] (5) After feature extraction, the facial feature vector set is sent to a deep learning classifier based on the improved cuckoo algorithm for micro-expression classification and recognition.

[0013] Furthermore, step (2) also includes performing facial similarity matching on the facial image to determine the identity.

[0014] Furthermore, in step (5), the deep learning classifier based on the improved cuckoo algorithm is specifically set as follows:

[0015] Generate an initial population for the input feature vector to be optimized, set the target optimization function to the accuracy of image classification, and then set the maximum number of iterations or early termination conditions. When the maximum number of iterations or termination conditions are reached, output the optimal feature subset and model parameters and apply them. If they are not reached, feedback optimization is performed using a feedback optimization method based on the improved cuckoo algorithm.

[0016] Furthermore, the deep learning classifier based on the improved cuckoo algorithm also includes encoding and setting facial feature vectors, and setting parameter boundaries for parameter encoding of the deep learning classification model.

[0017] Furthermore, a feedback optimization method based on an improved cuckoo algorithm includes selecting a subset of facial feature vectors and performing special conversion processing using a binary method; integer values ​​are processed by setting the boundaries of the integer type and rounding off continuous values ​​to obtain the nearest value.

[0018] Furthermore, the method also includes classifying facial images to obtain 7-dimensional emotion vectors, including anger, disgust, fear, excitement, calmness, frustration and surprise, and using a nearest neighbor algorithm to classify facial emotion classification predictions by time to detect emotional abnormalities.

[0019] Furthermore, a nearest neighbor algorithm is used to detect emotional anomalies, including the steps of: classifying the collected facial images to form a 7-dimensional emotion vector, and using the output layer of the deep learning classifier to normalize to calculate the probability value of each emotion vector; aggregating the facial images collected for each person every hour, and calculating the average value of the emotion vector probability value from 7 dimensions to represent the emotion vector of the individual within the hour; performing vector distance calculation on the aggregating data of each person for the previous n days using the aggregating data of the current hour; sorting in ascending order of distance, and selecting the k emotion vectors closest to the emotion vector of the individual within the hour, and determining the classification frequency of the k points as abnormal points; returning the classification with the highest frequency of the first k points closest to the distance as the abnormal prediction classification of the current point.

[0020] Furthermore, the method also includes, after identifying the abnormal points of emotion, defining the risk level and identifying individuals with greater emotional fluctuations.

[0021] Furthermore, the seven emotion vectors are divided into positive emotions and negative emotions; a 3sigma test is performed on each emotion vector and the emotion vector of the past week; if a negative emotion vector is greater than the average value of the emotion vector of the week plus the standard deviation of the three one-week emotion vector sequences, the risk level is increased by 1; if a positive emotion vector is less than the average value of the emotion vector of the week minus the standard deviation of the three one-week emotion vector sequences, the risk level is reduced by 1.

[0022] The beneficial effect of the present invention lies in that, compared with the prior art, the present invention utilizes a camera to collect facial images and extract facial features without contact, adopts an improved cuckoo algorithm to optimize the parameters of the deep learning model, obtains a model with better performance for real-time monitoring, and finally adopts a nearest neighbor algorithm to detect emotional anomalies and judge the risk level of psychological problems according to the emotional components, and uses artificial intelligence technology to conduct scientific and reasonable mental health assessments.

[0023] This paper adopts a joint optimization strategy combining facial feature sets and deep learning model hyperparameters, aiming to comprehensively improve the synergistic effect of model hyperparameters and facial feature sets. Through a carefully designed optimization framework, the best feature subset and model hyperparameter configuration are selected, thus effectively breaking through the performance limitation of micro-expression classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flow chart of the micro-expression recognition method based on the improved cuckoo algorithm;

[0025] Figure 2 is a deep learning classifier model;

[0026] Figure 3 It is an encode-optimize-decode instance;

[0027] Figure 4 It is a flow chart of detecting abnormal emotions using the nearest neighbor algorithm. DETAILED DESCRIPTION

[0028] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the protection scope of this application.

[0029] like Figure 1 As shown, the micro-expression recognition method based on the improved cuckoo algorithm of the present invention is used for fine analysis of facial expressions so as to infer emotional states. Specifically, it includes the following steps:

[0030] (1) Using cameras and other equipment to collect images in real time;

[0031] (2) Use convolutional neural network face recognition technology to identify the face area of ​​the image collected by the camera to obtain a face image;

[0032] (3) Perform facial similarity matching on facial images to determine identity;

[0033] The face area is parsed into a 128-dimensional facial feature vector and made into a template and stored in the cache. The face area recognized in real time is parsed into a 128-dimensional facial feature vector and matched with the template using cosine similarity. The identity is determined by the largest match with a similarity greater than 95%.

[0034] (4) Perform face image preprocessing;

[0035] A series of preprocessing operations are performed on the input face image data, which are designed to optimize the image data for feature extraction in the subsequent steps. Face image preprocessing includes image standardization, normalization, resizing, and noise removal, which is to ensure that the image data meets the needs of algorithm processing while maintaining its essential characteristics.

[0036] (5) Extract image features from the preprocessed face image;

[0037] The preprocessed facial images then enter the feature extraction stage, the goal of which is to extract decisive visual information from the image. The extracted features must be sensitive enough to capture subtle changes in facial expressions, which are crucial for identifying micro-expressions.

[0038] Since facial images may be deformed due to changes in posture, expression and lighting conditions, the detected faces need to be aligned. After alignment, the facial structure information is captured by detecting feature points including the corners of the eyes, eyebrows, nose tip, corners of the mouth, etc., and then the histogram directional gradient algorithm is used to extract facial features and generate feature vectors.

[0039] (6) After feature extraction, the facial feature vector set is sent to the micro-expression classifier for micro-expression classification and recognition;

[0040] The classifier accurately classifies micro-expressions based on the input features. This step is the decisive output of the entire process, and its accuracy directly affects the effectiveness of micro-expression recognition.

[0041] like Figure 2 As shown in the figure, the classifier adopts a deep learning classifier model based on the improved cuckoo algorithm, in which the selection of facial feature vectors and the parameter setting of the deep learning classification model have a decisive influence on the performance of the micro-expression classifier model. The purpose of feature selection is to optimize the performance of the model and ensure that only the features that are most helpful in distinguishing different categories of micro-expressions are used to train the classifier, thereby improving computational efficiency and reducing model complexity.

[0042] The facial feature vector is encoded and set, such as the feature value 0 represents selection, and 1 represents non-selection. For the parameters of the deep learning classification model, such as learning rate, training batch, optimization function, activation function, number of network layers and number of neurons, the data type and value range of each parameter variable to be optimized are different, and the parameter boundary needs to be encoded and set. The learning rate is 1e -3 To 1-e -6 The training batches are integers, the optimization functions are adam and sgd, and the activation functions are sigmod and tanh.

[0043] Generate the initial population of the input feature vector to be optimized, the population size is a ( a It can be set to 30), and the target optimization function is set to the accuracy of image classification, that is, the number of correctly classified images divided by the total number of images. Then set the maximum number of iterations m Or early termination conditions, for example, set the maximum number of iterations to 200 or the early termination condition is that if the accuracy of 10 consecutive optimizations is improved by less than 0.1%, the algorithm is considered to have converged and terminates early. When the stopping condition is met, the optimal feature subset and model parameters are output and applied. If it is not met, feedback optimization is performed.

[0044] The present invention adopts a feedback optimization method based on the cuckoo algorithm, which includes two types: Levy flight and random disturbance, and can prevent the algorithm from falling into a local optimum.

[0045] The original cuckoo algorithm is a heuristic optimization algorithm with the following main features and steps:

[0046] 1. At the beginning of the algorithm, a set of solutions is randomly generated as the initial population of candidate solutions;

[0047] 2. Select a candidate solution and generate a new solution through Levy flight;

[0048] Levy flight is a random walk whose step size conforms to the Levy distribution. This random step size enables the algorithm to effectively balance between global search and local search.

[0049] 3. The newly generated solution will replace a randomly selected solution with a certain probability; if the quality of the new solution (usually the objective function or fitness value) is better than the replaced solution, the new solution is accepted; otherwise, the original solution is retained;

[0050] 4. Discover and eliminate the worst solution with a certain probability. After elimination, randomly generate new solutions to replace the eliminated solutions.

[0051] 5. Iterative update: Repeat steps 2 to 4 until the stopping condition is met, such as reaching the maximum number of iterations or the quality of the solution reaches a predetermined standard.

[0052] The main advantages of the original cuckoo algorithm are its simplicity and effectiveness, especially when solving global optimization problems. Its long-distance exploration capability introduced by Levy flights can effectively avoid local optimal solutions and improve the algorithm's global search capability. In addition, the algorithm has relatively few parameters and is easy to implement and adjust.

[0053] However, the original algorithm is not suitable for optimizing the algorithm of the present invention, and is not suitable for the case where the population code is an integer. Therefore, the present invention makes corresponding improvements to the original cuckoo algorithm to obtain a feedback optimization method based on the improved cuckoo algorithm.

[0054] The improved cuckoo algorithm includes two points: first, the selection of facial feature vector subsets uses a binary method for special transformation processing;

[0055] The feature selection is 0 or 1, so a binary method is used for special transformation processing, as shown in the formula:

[0056]

[0057] x i+1 represents the eigenvalue of the i+1th iteration, x i It represents the eigenvalue of the i-th iteration. This processing is only performed during decoding, and the updated continuous value is still used for the next round of Levy flight and random perturbation.

[0058] Second, the processing of integer values ​​is to set the boundaries of the integer type and round off consecutive values ​​to get the nearest value;

[0059] For example, there are 4 optimization functions, and the value boundary of the optimization function is set to 4; for example, the value is 3.4562, then the cuckoo algorithm optimization function value is 3, and the decoding corresponds to adam, and other integer solutions are deduced in the same way. Figure 3 As shown, the encoding-optimization-decoding process is demonstrated using an example.

[0060] (7) After obtaining the optimal classifier model, the collected facial images are classified to form a 7-dimensional emotion vector, such as anger, disgust, fear, excitement, calmness, frustration and surprise. The nearest neighbor algorithm is used to classify the facial emotion classification predictions by time to detect emotional abnormalities;

[0061] like Figure 4 As shown, the nearest neighbor algorithm is used to detect emotional anomalies, including the following steps:

[0062] Classify the collected face images to form a 7-dimensional emotion vector, and use the output layer of the deep learning classifier to normalize it to calculate the probability value of each emotion vector;

[0063] The facial images collected from each person every hour are aggregated, and the average of the probability values ​​of the emotion vector is calculated from 7 dimensions to represent the emotion vector of the individual in that hour;

[0064] The vector distance calculation is performed on the collected data of each person in the previous n days using the collected data of the current hour; the calculation formula for the distance between vectors x and y uses the Euclidean distance formula, as follows:

[0065]

[0066] n is 7, indicating 7 emotion vectors, x n and n represents the sentiment vector value of the nth dimension, and d represents the Euclidean distance between vectors x and y.

[0067] Sort in ascending order of distance, select the k emotion vectors closest to the emotion vector of the individual within the hour, and determine the classification frequency of the k points as abnormal points; return the classification with the highest frequency of the first k points closest to the distance as the abnormal prediction classification of the current point.

[0068] For example, the nearest neighbor parameter k is selected as 4, that is, the nearest emotion vectors of the current points of 4 individuals are calculated each time, and the value of n is 7, that is, a list of the individual's emotion vectors for one week is obtained to calculate the nearest neighbor algorithm.

[0069] (8) After using the nearest neighbor algorithm to identify outliers, it is necessary to set the risk level to identify individuals with greater mood swings within a week;

[0070] The seven emotion vectors are divided into two categories, positive emotions and negative emotions; positive emotions include excitement and calmness, and negative emotions include anger, disgust, fear, frustration, and surprise. A 3sigma test is performed on each emotion vector and the emotion vector of the past week, that is, if a negative emotion vector is greater than the average value of the emotion vector of the week plus the standard deviation of the three weekly emotion vector sequences, the risk level is increased by 1, and if a positive emotion vector is less than the average value of the emotion vector of the week minus the standard deviation of the three weekly emotion vector sequences, the risk level is reduced by 1.

[0071] The above method can identify individuals with greater mood swings within a week and define their risk levels, providing decision support for scientific intervention.

[0072] The application of the method of the present invention provides accurate data support for mental health assessment decisions, demonstrating the potential of deep learning in the field of refined emotion recognition and its application value in mental health monitoring. Through in-depth analysis of facial expressions and model optimization, the present invention not only promotes the development of micro-expression recognition technology, but also provides new perspectives and tools for research and practice in the field of mental health.

[0073] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0074] The applicant of the present invention has made a detailed explanation and description of the implementation examples of the present invention in conjunction with the drawings in the specification. However, those skilled in the art should understand that the above implementation examples are only preferred implementation schemes of the present invention, and the detailed description is only to help readers better understand the spirit of the present invention, but not to limit the scope of protection of the present invention. On the contrary, any improvements or modifications based on the inventive spirit of the present invention should fall within the scope of protection of the present invention.

Claims

1. A micro-expression recognition method based on an improved cuckoo algorithm, characterized in that: Includes steps: (1) Using camera equipment to collect images in real time; (2) Use convolutional neural network face recognition technology to identify the face area of ​​the image collected by the camera to obtain a face image; (3) Perform face image preprocessing; (4) Extract image features from the preprocessed face image; (5) After feature extraction, the facial feature vector set is sent to a deep learning classifier based on the improved cuckoo algorithm for micro-expression classification and recognition; A deep learning classifier based on the improved cuckoo algorithm, specifically, A deep learning classifier is used for micro-expression classification and recognition. First, the facial feature vector is encoded and set, and the parameter boundary is set for the parameter encoding of the deep learning classification model; Then, the initial population of the input feature vector to be optimized is generated, and the target optimization function is set to the accuracy of image classification. Then, the maximum number of iterations or early termination conditions are set. When the maximum number of iterations or the termination conditions are reached, the optimal feature subset and model parameters are output and applied. If they are not reached, feedback optimization is performed using a feedback optimization method based on the improved cuckoo algorithm. Among them, based on the feedback optimization method of the improved cuckoo algorithm, the selection of facial feature vector subsets is specially transformed using a binary method, and the feature values ​​are encoded as values ​​of 0 or 1; the floating-point data of the training batch size, activation function, and optimization function are rounded to the nearest integer value; (6) Classify the facial images to obtain 7-dimensional emotion vectors, including anger, disgust, fear, excitement, calmness, frustration and surprise, and use the nearest neighbor algorithm to classify the facial emotion classification predictions by time to detect emotional anomalies; Use the nearest neighbor algorithm to detect emotional anomalies, including classifying the collected facial images to form 7-dimensional emotion vectors, and use the output layer of the deep learning classifier to normalize to calculate the probability value of each emotion vector; classify the facial images collected for each person every hour, and calculate the average value of the emotion vector probability value from the 7 dimensions to represent the individual's emotion vector within the hour; use the current hour's collected data to calculate the vector distance of each person's collected data for the previous n days; sort in ascending order of distance, and select the k emotion vectors closest to the individual's emotion vector within the hour, and determine the classification frequency of the k points as abnormal points; return the classification with the highest frequency of the first k points closest to the distance as the abnormal prediction classification of the current point; (7) After identifying emotional anomalies, define the risk level and identify individuals with greater emotional fluctuations.

2. The micro-expression recognition method based on the improved cuckoo algorithm according to claim 1, characterized in that: Step (2) also includes performing facial similarity matching on the facial image to determine the identity.

3. The micro-expression recognition method based on the improved cuckoo algorithm according to claim 1, characterized in that: In step (7), the seven emotion vectors are divided into positive emotions and negative emotions; a 3sigma test is performed on each emotion vector and the emotion vector of the past week; if a negative emotion vector is greater than the average value of the emotion vector of the week plus the standard deviation of the three weekly emotion vector sequences, the risk level is increased by 1; if a positive emotion vector is less than the average value of the emotion vector of the week minus the standard deviation of the three weekly emotion vector sequences, the risk level is reduced by 1.

Images