Micro-expression classification method based on AU region and multi-level Transform fusion module

Abstract

The invention provides a micro-expression classification method based on an AU region and a multi-level Transform fusion module, and the method comprises the steps: building a micro-expression classification network, carrying out the learning and fusion of embedded vectors in a hierarchical manner, and carrying out the classification of finally obtained sample embedded vectors; the implementation process comprises AU region division, embedded vector generation and first-level fusion, and learning fusion is carried out on the embedded vector of each AU region to obtain a local embedded vector containing AU region features; second-level fusion: carrying out learning fusion on the local embedded vector of each frame to obtain a global embedded vector containing the expression image feature of the frame; third-level fusion: performing learning fusion on the global embedded vector of each sample to obtain a sample embedded vector containing expression spatial features and time sequence features of the sample; and reversely utilizing an Attention mechanism to calculate the attention degree of each embedded vector in the sequence, and taking the attention degree as a weight to carry out weighted average on the value vectors of different embedded vectors to obtain a fused embedded vector.

Description

technical field [0001] The invention belongs to the technical field of deep learning in machine learning, and in particular relates to a micro-expression classification method based on an AU region and a multi-level Transformer fusion module. Background technique [0002] Currently, mainstream deep learning networks for sequence frame classification fall into two categories: [0003] The first category is to use the combination of 2D convolution and time series network to sequentially perform spatial feature extraction and temporal feature extraction on each frame in the image sequence. The time series network mainly uses the RNN / LSTM series of networks, such as the ELRCN network proposed in recent years for micro-expression classification (Document 1). The experimental results show that temporal and spatial features play different roles in micro-expression recognition, and good The recognition effect depends on the effective combination of the two. [0004] The second lar...

AI Technical Summary

Problems solved by technology

[0005] However, the time series network needs to be recursive step by step to obtain global information, and the information at the next moment depends on the information at the previous moment, that is, there is a sequence dependency, so the parallel computing ability of this series of networks is very poor.

Although the 3D convolutional network is easy to parallelize, it can only obtain local information. The receptive field is increased by superimposing the number of convolutional layers, and the ability to learn long dependencies is weak.

Images