Method and apparatus for micro-expression recognition based on hybrid spatio-temporal convolution model

A convolution model and recognition method technology, applied in character and pattern recognition, acquisition/recognition of facial features, instruments, etc., can solve problems such as difficulty in landing, slow calculation efficiency, high requirements, etc., to reduce computational complexity and benefit Effect of productization, reduced requirements

CN109389045AActive Publication Date: 2019-02-26GCI SCI & TECH +1
10 Cites 13 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Publication Date
2019-02-26

Smart Images

  • Figure 1
    Figure 1
  • Figure 2
    Figure 2
  • Figure 3
    Figure 3
Patent Text Reader

Abstract

The invention provides a micro-expression recognition method and a device based on a mixed spatio-temporal convolution model. The method comprises the following steps: training a mixed spatio-temporalconvolution network established in advance according to a pre-acquired image training sample to obtain a mixed spatio-temporal convolution model; Wherein the hybrid spatio-temporal convolution network comprises a plurality of cyclically alternately connected 3D residual modules, each 3D residual module comprising a 1*3*3 convolution layer and a 3*1*1 convolution layer; An image to be recognized is input to the mixed spatio-temporal convolution model to obtain a microexpression classification result. The mixed 1*3*3 convolution (2D) +3*1*1 convolution (1D) is used for convolution calculation,on the one hand, the invention guarantees the precision requirement of 3D CNN in micro expression recognition; on the other hand, the invention adopts the mixed 1*3*3 convolution (2D) +3*1*1 convolution (1D) to perform convolution calculation. On the other hand, it greatly reduces the computational complexity, and reduces the requirement of computer hardware, which is more conducive to the production.
Need to check novelty before this filing date? Find Prior Art

Description

technical field

[0001] The invention relates to the technical field of micro-expression recognition, in particular to a micro-expression recognition method and device based on a mixed spatiotemporal convolution model. Background technique

[0002] Microexpressions are very fleeting, involuntary facial expressions that humans reveal when they try to suppress or hide their true emotions. The difference between it and ordinary expressions is that the duration of micro-expressions is very short, only 1 / 25 second to 1 / 5 second. Therefore, most people are often unaware of its existence. This rapid and subtle facial expression is thought to be associated with an ego defense mechanism, expressing repressed emotions. However, the psychological and neural mechanisms of the generation and recognition of micro-expressions are still being studied, and the frequency of micro-expressions is relatively low, and ordinary people's ability to recognize micro-expressions is not high. A micro...

Examples

Embodiment 1

[0052] see figure 1 , which is a schematic flowchart of a micro-expression recognition method based on a hybrid spatio-temporal convolution model provided by an embodiment of the present invention. The methods include:

[0053] S100: Train the pre-established hybrid spatio-temporal convolution network according to the pre-acquired image training samples to obtain a hybrid spatio-temporal convolution model; wherein, the hybrid spatio-temporal convolution network includes a plurality of cyclic alternately connected 3D residual modules, each The 3D residual modules each include a 1*3*3 convolutional layer and a 3*1*1 convolutional layer;

[0054] S200: Input the image to be recognized into the hybrid spatiotemporal convolution model to obtain a micro-expression classification result.

[0055] In step S100, a hybrid spatio-temporal convolution model consisting of a plurality of alternately connected 3D residual modules including 1*3*3 convolutional layers and 3*1*1 convolutional...

Embodiment 2

[0133] see Figure 5, which is a schematic block diagram of a micro-expression recognition device based on a hybrid spatio-temporal convolution model provided by an embodiment of the present invention, the device includes:

[0134] The model construction module 1 is used to train the pre-established hybrid spatiotemporal convolutional network according to pre-acquired image training samples to obtain a hybrid spatiotemporal convolutional model; Difference module, each 3D residual module includes 1*3*3 convolutional layer and 3*1*1 convolutional layer;

[0135] The micro-expression recognition module 2 is configured to input the image to be recognized into the mixed spatio-temporal convolution model to obtain a micro-expression classification result.

[0136] In an optional embodiment, the model building module 1 includes:

[0137] The data classification unit is used to classify the pre-collected expression image data according to several predefined micro-expressions;

[01...