Micro-expression recognition method based on network intensity perception potential release and program product

Abstract

The invention discloses a micro-expression recognition method based on network intensity perception potential release and a program product. The method comprises the following steps: acquiring a micro-expression database and a macro-expression database; for each micro-expression video sample, extracting a starting frame and a middle frame from the micro-expression video sample, and extracting a middle frame optical flow feature; for each macro expression video sample, extracting a starting frame and a peak value frame from the macro expression video sample, and extracting a peak value frame optical flow feature; constructing an intensity perception neural network sharing weights, wherein the intensity perception neural network predicts sentiment classification confidence scores of micro-expressions or macro-expressions; training the intensity perception neural network, wherein a loss function during training is the weighted sum of micro-expression cross entropy loss, macro-expression cross entropy loss, feature mean square error loss and sparse constraint loss; and extracting middle frame optical flow features from a to-be-identified micro-expression video, inputting the middle frame optical flow features into the trained intensity perception neural network, and identifying a micro-expression category. The method is higher in recognition accuracy.

Description

Technical Field

[0001] This invention relates to micro-expression recognition technology, and more particularly to a micro-expression recognition method and program product based on network strength perception potential release. Background Technology

[0002] Micro-expressions are involuntary, rapid, and localized facial expressions characterized by subtle changes in facial muscles and texture, and typically short durations. While people can intentionally mask or suppress their true emotions by faking macro-expressions, micro-expressions are uncontrollable and can therefore reveal genuine emotions that humans attempt to conceal. Due to this ability to reveal true feelings, micro-expression recognition has significant applications in lie detection, psychological counseling, and other scenarios; therefore, building an accurate micro-expression recognition system is of great importance.

[0003] However, compared to common macro-expressions, micro-expressions exhibit much lower motion intensity. This means that the key emotional movements in micro-expressions are too subtle for most existing methods to learn discriminative features, making micro-expression recognition a challenging task. To address this issue, researchers have introduced video motion magnification techniques to enhance facial movements in micro-expressions at the image level, making discriminative features more prominent and reducing recognition difficulty, thus improving recognition accuracy. Following this idea, classical physical magnification methods, such as Eulerian Motion Magnification (EMM) and Global Lagrangian Motion Magnification (GLMM), as well as deep learning-based motion magnification (LMM), have been applied to micro-expression recognition tasks and have been proven to be easier to recognize to some extent compared to the original micro-expression samples. However, motion magnification methods require manual adjustment of the magnification factor, which can introduce additional noise or result in insignificant magnification, thus limiting their effectiveness. Researchers have introduced macro-expression guidance to micro-expression learning high-intensity knowledge at the feature level. Their main idea is to use transfer learning techniques, such as pre-training a knowledge network on macro-expression samples for fine-tuning on micro-expression samples, aligning the macro-expression features and micro-expression features extracted by the macro-expression network and micro-expression network, thereby forcing the network to extract discriminative features of micro-expressions. This has been proven to achieve more stable and better recognition performance compared to motion amplification methods. However, good transferability requires both the source and target domains to extract highly discriminative features, but micro-expression networks often fail to extract highly discriminative features, leading to room for improvement in accuracy. Summary of the Invention

[0004] To address the problems existing in the prior art, the purpose of this invention is to provide a micro-expression recognition method and program product based on network strength perception potential release that does not require transfer learning and has higher recognition accuracy.

[0005] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0006] A micro-expression recognition method based on network strength-aware potential release includes the following steps:

[0007] (1) Obtain a micro-expression database and a macro-expression database. The micro-expression database contains several micro-expression video samples and corresponding micro-expression category tags, and the macro-expression database contains several macro-expression video samples and corresponding macro-expression category tags.

[0008] (2) For each micro-expression video sample in the micro-expression database, extract the starting frame and the intermediate frame and extract the optical flow features of the intermediate frame of the micro-expression through preprocessing.

[0009] (3) For each macro expression video sample in the macro expression database, extract the start frame and peak frame and extract the peak frame optical flow features of the macro expression through preprocessing;

[0010] (4) Construct a shared-weight intensity-aware neural network, which is used to predict the sentiment classification confidence score of micro-expressions or macro-expressions by taking the intermediate frame optical flow features of micro-expressions or the peak frame optical flow features of macro-expressions as input.

[0011] (5) The optical flow features of the intermediate frames of micro-expressions and the optical flow features of the peak frames of macro-expressions are input into the intensity-sensing neural network at intervals for training. The loss function during training is the weighted sum of the cross-entropy loss of micro-expressions, the cross-entropy loss of macro-expressions, the feature mean square error loss, and the sparsity constraint loss. The feature mean square error loss is the difference loss between the pseudo-macro-expression features learned by the perception enhancement module and the macro-expression features of all macro-expression video samples. The sparsity constraint loss is the sparsity constraint loss of the perception enhancement module. The perception enhancement module is used to learn pseudo-macro-expression features based on all micro-expression video samples.

[0012] (6) Extract the optical flow features of the intermediate frames from the video of the micro-expression to be identified in step (2), input them into the trained intensity-sensing neural network, and identify the micro-expression category.

[0013] Furthermore, the preprocessing in step (2) includes face detection, face region alignment in each frame, face region cropping in each frame, and feature extraction. Specifically, feature extraction involves extracting the horizontal optical flow field, vertical optical flow field, and optical strain information of the intermediate frame relative to the starting frame, and superimposing them into a 3-channel optical flow feature map, which serves as the optical flow feature of the intermediate frame for micro-expressions.

[0014] Furthermore, the preprocessing in step (3) includes face detection, face region alignment in each frame, face region cropping in each frame, and feature extraction. Specifically, feature extraction involves extracting the horizontal optical flow field, vertical optical flow field, and optical strain information of the peak frame relative to the starting frame, and superimposing them into a 3-channel optical flow feature map as the peak frame optical flow feature of the macro expression.

[0015] Furthermore, step (4) specifically includes:

[0016] (4-1) Using ResNet18 as the backbone, construct an intensity-aware neural network with shared weights. The ResNet18 includes a convolutional part, a fully connected layer, and a softAMX operation.

[0017] (4-2) Input the optical flow features of the intermediate frames into the intensity-aware neural network, and output the sentiment classification confidence score for each micro-expression video sample. Input the optical flow features of the peak frames into the intensity-aware neural network, and output the sentiment classification confidence score for each macro-expression video sample.

[0018]

[0019] In the formula, The confidence score for the sentiment classification of the i-th micro-expression video sample represents the confidence score for the i-th micro-expression video sample. f represents the sentiment classification confidence score of the j-th macro-expression video sample. I i The feature map representing the i-th micro-expression video sample. represents the feature map of the j-th macro-expression video sample, FE represents the convolutional function in the intensity-sensing neural network, and FC represents the fully connected layer and softAMX operation function in the intensity-sensing neural network.

[0020] Furthermore, the micro-expression cross-entropy loss mentioned in step (5) is:

[0021]

[0022] In the formula, For micro-expression cross-entropy loss, y i ∈{1,2,…,L I} represents the micro-expression type label of micro-expression video sample i, L I This indicates the total number of micro-expression type tags. It is an indicator function, if y i If the result of =l is true, return 1; otherwise, return 0. represents the sentiment classification confidence score of the i-th micro-expression video sample, and N represents the number of micro-expression video samples.

[0023] Furthermore, the macro-expression cross-entropy loss mentioned in step (5) is:

[0024]

[0025] In the formula, For macro-expression cross-entropy loss, y j ∈{1,2,…,L A} represents the macro expression type label for macro expression video sample j, L A This indicates the total number of macro emoticon type tags. It is an indicator function, if y j If the result of =l is true, return 1; otherwise, return 0. Let M represent the sentiment classification confidence score of the j-th macro-expression video sample, and M represent the number of macro-expression video samples.

[0026] Furthermore, the feature mean square error loss described in step (5) is as follows:

[0027]

[0028] f AEI =W·f I

[0029] f I =[f I i |i=1,2,…,N]

[0030] In the formula, f represents the feature mean square error loss. AEI The linearly weighted fusion feature map of micro-expressions enhanced by the perception enhancement module is used as a pseudo-macro expression feature, f. A f represents the feature map resulting from merging the feature maps of all macro-expression video samples. I This represents the feature map resulting from merging the feature maps of all micro-expression video samples. Let f represent the feature map of the j-th macro-expression video sample. I i represents the feature map of the i-th micro-expression video sample, W∈R M×N Let M represent the perception enhancement weight matrix of the perception enhancement module, M represent the number of macro-expression video samples, N represent the number of micro-expression video samples, and |||2 represent the L2 norm.

[0031] Furthermore, the sparse constraint loss mentioned in step (5) specifically refers to:

[0032]

[0033] In the formula, Let represent the sparse constraint loss, W represent the perceptual enhancement weight matrix learned by the perceptual enhancement module, and |||1 represent the L1 norm.

[0034] Furthermore, the loss function during training in step (5) is:

[0035]

[0036] In the formula, Let be the total loss function, and α, β, γ be the equilibrium loss function. and The weight of the proportion, For micro-expression cross-entropy loss, This indicates the total number of macro emoticon type tags. This represents the characteristic mean square error loss. This represents the sparse constraint loss.

[0037] A computer program product includes a computer program / instructions that, when executed by a processor, implement the above-described method.

[0038] Compared with existing technologies, the beneficial effects of this invention are as follows: The invention adopts the idea of ​​intensity perception potential release, and constructs a model with shared macro-expression and micro-expression weights through four constraints, thereby introducing highly discriminative parameters into the network for micro-expression learning. The network can passively enhance the weak discriminative features of micro-expressions during the optimization process. On this basis, a perception enhancement module is designed to linearly reconstruct the target macro-expression features from the micro-expression features, forcing the network to learn the importance of micro-expression features relative to the target macro-expression features, thereby stimulating the network's ability to perceive low-intensity micro-expressions and achieving higher accuracy in micro-expression recognition without using transfer learning. Attached Figure Description

[0039] Figure 1 This is a flowchart illustrating the micro-expression recognition method based on network strength perception potential release provided in an embodiment of the present invention.

[0040] Figure 2 This is a schematic diagram of the intensity-sensing neural network of the present invention. Detailed Implementation

[0041] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0042] Example 1

[0043] This invention provides a micro-expression recognition method based on network strength-aware potential release, such as... Figure 1 As shown, it includes the following steps:

[0044] (1) Obtain the micro-expression database and macro-expression database.

[0045] The micro-expression database contains several micro-expression video samples and corresponding micro-expression category tags, while the macro-expression database contains several macro-expression video samples and corresponding macro-expression category tags.

[0046] (2) For each micro-expression video sample in the micro-expression database, extract the starting frame and the intermediate frame and extract the optical flow features of the intermediate frame of the micro-expression through preprocessing.

[0047] The preprocessing includes face detection, face region alignment in each frame, facial region cropping in each frame, and feature extraction. Specifically, feature extraction involves extracting the horizontal optical flow field, vertical optical flow field, and optical strain information of the intermediate frame relative to the starting frame, and superimposing them into a 3-channel optical flow feature map, which serves as the optical flow feature of the intermediate frame for micro-expressions.

[0048] (3) For each macro expression video sample in the macro expression database, extract the start frame and peak frame and extract the peak frame optical flow features of the macro expression through preprocessing.

[0049] The preprocessing includes face detection, face region alignment in each frame, face region cropping in each frame, and feature extraction. Specifically, feature extraction involves extracting the horizontal optical flow field, vertical optical flow field, and optical strain information of the peak frame relative to the starting frame, and superimposing them into a 3-channel optical flow feature map, which serves as the peak frame optical flow feature of the macro expression.

[0050] (4) Construct a strength-aware neural network with shared weights.

[0051] The intensity-aware neural network (IANN) takes the optical flow features of intermediate frames of micro-expressions or the optical flow features of peak frames of macro-expressions as input, thereby introducing highly discriminative macro-expression features for micro-expression feature learning and predicting the sentiment classification confidence score of micro-expressions or macro-expressions. The INN uses ResNet18 as its backbone, which includes convolutional parts, fully connected layers, and softAMX operations. The convolutional part includes several residual blocks, each containing a 3×3 convolutional layer and residual connections that help directly add the input to the output of deeper layers. Inputting intermediate frame optical flow features into the INN outputs the sentiment classification confidence score for each micro-expression video sample. Similarly, inputting peak frame optical flow features into the INN outputs the sentiment classification confidence score for each macro-expression video sample.

[0052]

[0053] In the formula, The confidence score for the sentiment classification of the i-th micro-expression video sample represents the probability that the model classifies this micro-expression descriptor as belonging to each sentiment category. f represents the sentiment classification confidence score of the j-th macro-expression video sample. I i The feature map representing the i-th micro-expression video sample. represents the feature map of the j-th macro-expression video sample, FE represents the convolutional function in the intensity-sensing neural network, and FC represents the fully connected layer and softAMX operation function in the intensity-sensing neural network.

[0054] (5) Input the optical flow features of the intermediate frames of micro-expressions and the optical flow features of the peak frames of macro-expressions into the intensity-sensing neural network at intervals for training.

[0055] The training loss function is a weighted sum of micro-expression cross-entropy loss, macro-expression cross-entropy loss, feature mean square error loss, and sparsity constraint loss. The feature mean square error loss is the difference loss between the macro-expression features learned by the perception enhancement module and the macro-expression features of all macro-expression video samples. The sparsity constraint loss is the sparsity constraint loss of the perception enhancement module, which is used to learn the corresponding macro-expression features based on the features of all micro-expression video samples.

[0056] The micro-expression cross-entropy loss is:

[0057]

[0058] In the formula, For micro-expression cross-entropy loss, y i ∈{1,2,…,L I} represents the micro-expression type label of micro-expression video sample i, L I This indicates the total number of micro-expression type tags. It is an indicator function, if y i If the result of =l is true, return 1; otherwise, return 0. represents the sentiment classification confidence score of the i-th micro-expression video sample, and N represents the number of micro-expression video samples.

[0059] The macro-expression cross-entropy loss is:

[0060]

[0061] In the formula, For macro-expression cross-entropy loss, y j ∈{1,2,…,L A} represents the macro expression type label for macro expression video sample j, L A This indicates the total number of macro emoticon type tags. It is an indicator function, if y j If the result of =l is true, return 1; otherwise, return 0. Let M represent the sentiment classification confidence score of the j-th macro-expression video sample, and M represent the number of macro-expression video samples.

[0062] Two cross-entropy loss functions can establish the relationship between the network's prediction results for micro-expression samples and macro-expression samples and the sample sentiment labels, respectively. Since this intensity-aware neural network shares weights, during gradient optimization, the macro-expression cross-entropy loss can guide the network to learn highly discriminative feature extraction parameters from the input macro-expression knowledge. Then, after successfully converging on macro-expression samples, the micro-expression cross-entropy loss forces micro-expressions, which can only extract weakly discriminative features, to move towards strong discriminativeness under the encouragement of highly discriminative knowledge. Ultimately, this inspires the network to passively and subtly enhance the discriminativeness of the extracted micro-expression features.

[0063] Building upon this foundation, to enhance the network's ability to perceive low-intensity micro-expressions, a perception enhancement module W is designed. This module learns pseudo-macro-expression features based on the features of all micro-expression video samples, forcing the network to learn the relative importance of micro-expression features to macro-expression features. The feature mean square error constrains the network to actively enhance the weak discriminative features of micro-expressions. For learnable perception enhancement modules, sparsity constraints are used to eliminate unimportant weights in the module.

[0064] For the perception enhancement module W, the feature mean squared error loss function is constructed as follows:

[0065]

[0066] f AEI =W·f I

[0067] f I =[f I i |i=1,2,…,N]

[0068] In the formula, f represents the feature mean square error loss. AEI f represents the linearly weighted fusion feature map of micro-expressions enhanced by the perception enhancement module. A The feature map representing the merged feature maps of all macro-expression video samples is used as the pseudo-macro-expression feature. I This represents the feature map resulting from merging the feature maps of all micro-expression video samples. Let f represent the feature map of the j-th macro-expression video sample, where D represents the dimension of the feature map. I i The feature map representing the i-th micro-expression video sample, W∈R M×NThe perceptual enhancement weight matrix represents the perceptual enhancement module, and ||2 represents the L2 norm. The feature mean square error loss function constrains the perceptually enhanced pseudo-macro-expression features to approximate the target macro-expression feature map, so as to enable the network to perceive the importance of each micro-expression feature map relative to each target macro-expression feature map, thereby actively enhancing the weak discriminative features of micro-expressions.

[0069] Based on the aforementioned perceptual enhancement module W, this module learns the perceptual enhancement weights for each micro-expression feature map corresponding to each target macro-expression feature map. Since many micro-expression feature maps are only minimally affected by the perceptual enhancement of the target macro-expression feature map, without adding constraints to limit its effective learning, the network will learn too many unimportant weight coefficients from the perceptual enhancement module, leading to difficulties in fitting and effective optimization. To address these issues, a sparse constraint loss function is constructed to eliminate unimportant weight coefficients in the module. The sparse constraint loss function is specifically set as follows:

[0070]

[0071] In the formula, Let represent the sparse constraint loss, and ‖‖1 represent the L1 norm. By further imposing L1 norm constraints on network parameters and other losses, less important weight coefficients in the perceptual enhancement weights are eliminated during network gradient optimization, resulting in sparser modules and a better network learning trend.

[0072] By using the aforementioned feature mean squared error loss function, the importance of each micro-expression feature map relative to each target macro-expression feature map can be effectively learned. This allows the network to be forced to sensitively perceive low-intensity micro-expression samples again, leveraging knowledge of macro-expressions, ultimately achieving enhanced intensity perception through an active strategy. Since not all weights in the perception enhancement module storing learnable weight parameters contain important information—that is, many micro-expression feature maps do not participate in linear reconstruction or have very small weights—the aforementioned feature mean squared error loss function prevents the network from learning this redundant information, instead focusing on learning the important weight sparsity.

[0073] Finally, the four loss functions are weighted together to form the total loss function, which is then optimized using gradient descent to train the intensity-aware neural network. The total loss function is:

[0074]

[0075] In the formula, Let be the total loss function, and α, β, γ be the equilibrium loss function. and The weight of the proportion.

[0076] (6) Extract the optical flow features of the intermediate frames from the video of the micro-expression to be identified in step (2), input them into the trained intensity-sensing neural network, and identify the micro-expression category.

[0077] To verify the effectiveness of the present invention, micro-expression recognition tasks were performed on the CAME2 micro-expression database, the HS sub-database of the SMIC database, and the SAMM micro-expression database.

[0078] The facial expressions in the CASME2 database were processed as follows: categories with insufficient sample sizes were omitted, and only the five main emotion categories from the original seven categories were selected to complete the five-category recognition task: others, disgust, happiness, surprise, and repression. The SMIC database itself has categories of negative, positive, and surprise, so a three-category recognition task was completed. The facial expressions in the SAMM database were processed as follows: categories with insufficient sample sizes were omitted, and only the five main emotion categories from the original eight categories were selected to complete the five-category recognition task: anger, contempt, happiness, surprise, and other. The required categories and sample numbers for the micro-expression recognition tasks in the three databases are shown in Table 1.

[0079] Table 1

[0080]

[0081]

[0082] The validation accuracy results and comparisons with other state-of-the-art methods are shown in Table 2:

[0083] Table 2

[0084] method years CASME2 SMIC-HS SAMM DIKD 2020 72.61 76.06 N / A MTMNet 2020 75.60 76.80 74.10 DSTAN 2021 75.20 77.40 N / A SLSTT-LSTM 2022 75.81 75.00 72.39 AMAN 2022 75.40 79.87 68.85 NSPT 2022 N / A 79.47 N / A Graph-TCN 2020 73.98 N / A 75.00 TS-AUCNN 2022 72.61 76.06 N / A MicroNet 2020 75.60 76.80 74.10 MiNet 2021 79.90 78.60 76.70 PLMaM-Net 2023 82.52 82.32 76.47 This method 2024 82.93 82.93 80.15

[0085] In Table 2, N / A indicates that there are no relevant records.

[0086] Experimental results show that the micro-expression recognition method proposed in this invention achieves a high micro-expression recognition rate.

[0087] Example 2

[0088] This invention also provides a computer product, such as an app on a mobile phone or tablet, or an installer on a computer. This product includes a computer program / instructions that, when executed by a processor, implement the method described in Embodiment 1. The code for a computer-executable program that performs the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0089] It should be understood that the embodiments and descriptions above are only the principles, main features and advantages of the present invention. Various changes and modifications can be made to the present invention without departing from the spirit and scope of the invention, and all such changes and modifications fall within the protection scope of the present invention.

Claims

1. A micro-expression recognition method based on network strength perception potential release, characterized in that, Includes the following steps: (1) Obtain a micro-expression database and a macro-expression database. The micro-expression database contains several micro-expression video samples and corresponding micro-expression category tags, and the macro-expression database contains several macro-expression video samples and corresponding macro-expression category tags. (2) For each micro-expression video sample in the micro-expression database, extract the starting frame and the intermediate frame and extract the optical flow features of the intermediate frame of the micro-expression through preprocessing. (3) For each macro expression video sample in the macro expression database, extract the start frame and peak frame and extract the peak frame optical flow features of the macro expression through preprocessing; (4) Construct a shared-weight intensity-aware neural network, which is used to predict the sentiment classification confidence score of micro-expressions or macro-expressions by taking the intermediate frame optical flow features of micro-expressions or the peak frame optical flow features of macro-expressions as input. (5) The optical flow features of the intermediate frames of micro-expressions and the optical flow features of the peak frames of macro-expressions are input into the intensity-sensing neural network at intervals for training. The loss function during training is the weighted sum of the cross-entropy loss of micro-expressions, the cross-entropy loss of macro-expressions, the feature mean square error loss, and the sparsity constraint loss. The feature mean square error loss is the difference loss between the pseudo-macro-expression features learned by the perception enhancement module and the macro-expression features of all macro-expression video samples. The sparsity constraint loss is the sparsity constraint loss of the perception enhancement module. The perception enhancement module is used to learn pseudo-macro-expression features based on all micro-expression video samples. (6) Extract the optical flow features of the intermediate frames from the video of the micro-expression to be identified in step (2), input them into the trained intensity-sensing neural network, and identify the micro-expression category.

2. The micro-expression recognition method based on network strength perception potential release according to claim 1, characterized in that: The preprocessing in step (2) includes face detection, face region alignment in each frame, face region cropping in each frame, and feature extraction. Specifically, feature extraction involves extracting the horizontal optical flow field, vertical optical flow field, and optical strain information of the intermediate frame relative to the starting frame, and superimposing them into a 3-channel optical flow feature map, which serves as the optical flow feature of the intermediate frame for micro-expressions.

3. The micro-expression recognition method based on network strength perception potential release according to claim 1, characterized in that: The preprocessing in step (3) includes face detection, face region alignment in each frame, face region cropping in each frame, and feature extraction. Specifically, feature extraction involves extracting the horizontal optical flow field, vertical optical flow field, and optical strain information of the peak frame relative to the starting frame, and superimposing them into a 3-channel optical flow feature map, which serves as the peak frame optical flow feature of the macro expression.

4. The micro-expression recognition method based on network strength perception potential release according to claim 1, characterized in that: Step (4) specifically includes: (4-1) Using ResNet18 as the backbone, construct an intensity-aware neural network with shared weights. The ResNet18 includes a convolutional part, a fully connected layer, and a softAMX operation. (4-2) Input the optical flow features of the intermediate frames into the intensity-aware neural network, and output the sentiment classification confidence score for each micro-expression video sample. Input the optical flow features of the peak frames into the intensity-aware neural network, and output the sentiment classification confidence score for each macro-expression video sample. In the formula, The confidence score for the sentiment classification of the i-th micro-expression video sample represents the confidence score for the i-th micro-expression video sample. The sentiment classification confidence score of the j-th macro-expression video sample. The feature map representing the i-th micro-expression video sample. represents the feature map of the j-th macro-expression video sample, FE represents the convolutional function in the intensity-sensing neural network, and FC represents the fully connected layer and softAMX operation function in the intensity-sensing neural network.

5. The micro-expression recognition method based on network strength perception potential release according to claim 1, characterized in that: The micro-expression cross-entropy loss mentioned in step (5) is: In the formula, For micro-expression cross-entropy loss, y i ∈{1,2,…,L I } represents the micro-expression type label of micro-expression video sample i, L I This indicates the total number of micro-expression type tags. It is an indicator function, if y i If the result of =l is true, return 1; otherwise, return 0. represents the sentiment classification confidence score of the i-th micro-expression video sample, and N represents the number of micro-expression video samples.

6. The micro-expression recognition method based on network strength perception potential release according to claim 1, characterized in that: The macro-expression cross-entropy loss mentioned in step (5) is: In the formula, For macro-expression cross-entropy loss, y j ∈{1,2,…,L A } represents the macro expression type label for macro expression video sample j, L A This indicates the total number of macro emoticon type tags. It is an indicator function, if y j If the result of =l is true, return 1; otherwise, return 0. Let M represent the sentiment classification confidence score of the j-th macro-expression video sample, and M represent the number of macro-expression video samples.

7. The micro-expression recognition method based on network strength perception potential release according to claim 1, characterized in that: The specific feature mean square error loss mentioned in step (5) is as follows: f AEI =W·f I In the formula, f represents the feature mean square error loss. AEI The linearly weighted fusion feature map of micro-expressions enhanced by the perception enhancement module is used as a pseudo-macro expression feature, f. A f represents the feature map resulting from merging the feature maps of all macro-expression video samples. I This represents the feature map resulting from merging the feature maps of all micro-expression video samples. This represents the feature map of the j-th macro-expression video sample. The feature map representing the i-th micro-expression video sample, W∈R M×N Let M represent the perception enhancement weight matrix of the perception enhancement module, M represent the number of macro-expression video samples, N represent the number of micro-expression video samples, and |||2 represent the L2 norm.

8. The micro-expression recognition method based on network strength perception potential release according to claim 1, characterized in that: The sparse constraint loss mentioned in step (5) is specifically as follows: In the formula, Let represent the sparse constraint loss, W represent the perceptual enhancement weight matrix learned by the perceptual enhancement module, and |||1 represent the L1 norm.

9. The micro-expression recognition method based on network strength perception potential release according to claim 1, characterized in that: The loss function during training in step (5) is: In the formula, Let be the total loss function, and α, β, γ be the equilibrium loss function. and The weight of the proportion, For micro-expression cross-entropy loss, This indicates the total number of macro emoticon type tags. This represents the characteristic mean square error loss. This represents the sparse constraint loss.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method of any one of claims 1-9.

Images