Micro-expression recognition method based on space-time appearance movement attention network

Abstract

The invention relates to a micro-expression recognition method based on a space-time appearance movement attention network, and the method comprises the following steps: carrying out the preprocessingof a micro-expression sample, and obtaining an original image sequence and an optical flow sequence with a fixed number of frames; constructing a space-time appearance motion network which comprisesa space-time appearance network STAN and a space-time motion network STMN, designing the STAN and the STMN by adopting a CNN-LSTM structure, learning spatial features of micro-expressions by using a CNN model, and learning time features of the micro-expressions by using an LSTM model; introducing hierarchical convolution attention mechanisms into CNN models of an STAN and an STMN, applying a multi-scale kernel space attention mechanism to a low-level network, applying a global double-pooling channel attention mechanism to a high-level network, and respectively obtaining an STAN network added with the attention mechanism and an STMN network added with the attention mechanism; inputting the original image sequence into the STAN network added with the attention mechanism to be trained, inputting the optical flow sequence into the STMN network added with the attention mechanism to be trained, integrating output results of the original image sequence and the optical flow sequence through the feature cascade SVM to achieve a micro-expression recognition task, and improving the accuracy of micro-expression recognition.

Description

technical field [0001] The technical solution of the present invention relates to image data processing for micro-expression recognition, in particular to a micro-expression recognition method based on a spatio-temporal appearance motion attention network. Background technique [0002] Micro-expressions are imperceptible facial expressions that a person tries to hide his true inner feelings but involuntarily reveal, which are fast, spontaneous, and unconscious. The duration of micro-expressions is short and the intensity is low, usually lasting 1 / 25s-1 / 5s, and the muscle movement caused by micro-expressions only appears in a small area of ​​the face, so it is difficult to correctly understand and recognize micro-expressions. To some extent, it limits the performance of micro-expression recognition. In recent years, a large number of algorithms using computer vision technology have emerged for automatic recognition of micro-expressions, which has greatly improved the applica...

AI Technical Summary

Problems solved by technology

[0004] The document "OFF-ApexNet on micro-expression recognition system" combines manual features with a fully data-driven CNN architecture, and proposes a vertex frame network based on optical flow characteristics, but only using the vertex frames of the micro-expression sequence as input data will lose micro-expression. A lot of useful information about emoticons

CN111353390A discloses a micro-expression recognition method based on deep learning. This method adopts the network structure of 3DCNN and ConvLSTM, which avoids the problem of low recognition accuracy caused by artificially concealing emotions or no obvious expression changes on the face, but this method will Each pixel of the micro-expression frame is treated equally, ignoring the contribution of different pixels or channels to micro-expression recognition

CN110348271A discloses a micro-expression recognition method based on long-short-term memory network, which adopts convolutional neural network and long-short-term memory network to extract the features of micro-expression sequence, but the extracted features are single, and different network layers are ignored complementarity of features

CN109034143A discloses a face micro-expression recognition method based on video amplification and deep learning. The method utilizes video amplification technology to amplify the range of movement of micro-expression video data, but inevitably introduces some noise, which affects the subtle movement changes of micro-expressions

CN108629314A discloses a micro-expression recognition method based on active transfer learning, which realizes the migration from expression data to micro-expression data, but the migration of similar tasks needs to find high-quality, large-scale source domain data similar to the target domain , and requires higher technical requirements

Images