Unlock instant, AI-driven research and patent intelligence for your innovation.

A transfer learning method from macro-expression to micro-expression

A transfer learning and micro-expression technology, applied in the field of computer vision and pattern recognition, can solve the problems of limited number of micro-expression database samples, unsatisfactory recognition effect, limited binding force, etc.

Active Publication Date: 2021-05-14
SHANDONG UNIV
View PDF7 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

The duration of micro-expressions is only 0.04s-0.2s, and the intensity is extremely weak. Due to the influence of lighting, noise and other factors, the recognition effect of micro-expression feature representation methods is still not ideal.
The current linear subspace learning method has enabled micro-expression recognition to have a better classification ability through sparse constraints, but this constraint is limited and is still largely affected by human facial features, light, and noise. Difficulties in feature representation and extraction of micro-expressions
At the same time, there are still limitations in the limited number of samples in the micro-expression database and the difficulties in sample data collection and labeling.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • A transfer learning method from macro-expression to micro-expression
  • A transfer learning method from macro-expression to micro-expression
  • A transfer learning method from macro-expression to micro-expression

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0080] A transfer learning method from macro-expression to micro-expression, such as figure 1 shown, including the following steps:

[0081] (1) Extract macro-expression features and micro-expression features respectively: extract LBP features from macro-expression to form macro-expression feature representation matrix X a , micro-expression extraction LBP-TOP features to form a micro-expression feature representation matrix X b , and take the average of the three orthogonal planes of the LBP-TOP feature. The original LBP-TOP feature is the LBP feature extracted from the three orthogonal planes of XY, XT, and YT, forming a 3*59-dimensional feature. Here, for two The dimensions of expression are unified, and the LBP features on the three orthogonal planes are accumulated and averaged to form a 59-dimensional LBP-TOP feature, forming a micro-expression representation feature that is unified with the macro-expression dimension;

[0082] (2) Project the micro-expression feature ...

Embodiment 2

[0087] According to a method of transfer learning from macro-expressions to micro-expressions described in Embodiment 1, the difference is that,

[0088] In step (2), the objective function is as shown in formula (I):

[0089]

[0090] In formula (I), α, β, γ are balance coefficients, α>0, β>0, γ>0;

[0091] x a is the macro-expression feature matrix, n a Represents the number of macro expression samples, d represents the feature dimension;

[0092] x b is the micro-expression feature matrix, n b Represents the number of dimension expression samples, and d represents the feature dimension;

[0093] P∈R d×np , P is the projection matrix, np is the subspace dimension;

[0094] V is the reconstruction coefficient matrix (equivalent to the secondary projection matrix);

[0095] E is an error matrix, which stores the respective characteristic components in the macro-expression and micro-expression, and the characteristic components include their unique characteri...

Embodiment 3

[0114] According to a method of transfer learning from macro-expression to micro-expression described in embodiment 1 or 2, the difference is that

[0115] In step (3), the projection matrix P is initialized by NMF, and the target projection matrix is ​​iteratively optimized by solving the objective function, and the target projection matrix P is output when the convergence condition is met. 1 , including the following steps:

[0116] E. Convert the objective function into an approximate form for solution: such as figure 2 said;

[0117] The objective function is shown in formula (Ⅵ):

[0118]

[0119] Considering various constraints, the objective function is non-convex, and the objective function is transformed into an approximate form, as shown in formula (VII):

[0120]

[0121] In formula (VII), Y is by the eigenvector that WY=ΛDY problem solves as the matrix that row vector forms, Λ is to form diagonal matrix as diagonal element by corresponding eigenvalue;

...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention relates to a transfer learning method from macro-expression to micro-expression. In the training process, the model adopts subspace projection, projects micro-expression feature matrix and macro-expression feature matrix into common subspace, and uses macro-expression feature matrix Reconstruct the micro-expression feature matrix, minimize the feature and structure difference between macro-expression and micro-expression in the common subspace by imposing constraints, maximize the correlation between macro-expression and micro-expression in the subspace, iteratively optimize the target projection matrix, and finally The test set data is projected into the common subspace using the obtained target projection matrix and classified by KNN. The useful information of the existing macro-expression domain samples is transferred to the micro-expression domain, which is equivalent to expanding the number of marked micro-expression samples, and overcomes the disadvantage of the difficulty of labeling with few micro-expression samples. This method not only reduces the human waste of marking, but also greatly improves the recognition performance, providing another strategy for micro-expression recognition.

Description

technical field [0001] The invention belongs to the field of computer vision and pattern recognition, and relates to a transfer learning method from macro-expression to micro-expression, in particular to a non-negative matrix factorization (NMF) for initialization, which can maintain the common latent characteristics of the two domains before and after projection and promote A method for transfer learning from macro-expression to micro-expression based on subspace projection of two-domain distance. [0002] technical background [0003] As early as 1966, Haggard and others discovered the existence of micro-expressions, see Haggard E A, Isaacs KS. Micromomentary facial expressions as indicators of ego mechanisms in psychotherapy[M] / / Methods of research in psychotherapy. Springer, Boston, MA, 1966 :154-165. They think that micro-expression is one of the manifestations of the human body's self-protection mechanism, which can express the thoughts or emotions that people suppress ...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Patents(China)
IPC IPC(8): G06K9/00G06K9/62G06N20/00
CPCG06V40/174G06F18/211
Inventor 贲晛烨肖瑞雪王德强朱雪娜巩力铜赵耕达
Owner SHANDONG UNIV