Facial expression recognition method and system based on cross-scale local difference deep subspace features

Abstract

The application discloses an expression recognition method and system based on cross-scale local difference deep subspace features, which obtains images in different scale spaces through a plurality of Gaussian filters, then blocks the images in different scale spaces, extracts local differences of the images, thereby obtains a cross-scale local difference matrix of a training set, trains a plurality of convolution kernels in a first stage, and extracts again a cross-scale local difference matrix of the images after the first stage of convolution, trains a plurality of convolution kernels in a second stage, thereby learns the convolution kernels in the two stages. After the image to be recognized is convolved in the two stages, the image features are obtained through nonlinear processing and histogram statistics, so that classification and recognition are carried out. The method fuses local differences of different scales of images to extract features, and the network structure is simple, a large number of training samples are not needed, and the hardware requirement is low.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of machine vision, and particularly relates to an expression recognition method and system based on a cross-scale local difference deep subspace feature. BACKGROUND

[0002] Face expression recognition is a key research direction in many fields such as machine vision, pattern recognition and artificial intelligence. At present, the commonly used method for expression recognition is to extract expression features in an image, and then use a classifier to classify and recognize, so as to obtain the category of the face expression in the image. Due to the light changes in the image, individual differences, head poses, face occlusions and the like, the difficulty of recognition is intensified, so it is still a key and challenging task to further find suitable and discriminative expression features in an uncontrolled scene. At present, there are two types of expression feature extraction methods: a hand-designed method and a deep learning-based method. The hand-designed method mainly focuses on extracting features according to specific standards using the structure and directional information of the image. This method not only requires knowledge in the relevant field, but also performs poorly on spontaneous expressions in uncontrolled scenes. The deep learning-based method directly extracts high-level features from data through a neural network. This method usually has a complex network structure, high hardware requirements, and needs a large number of samples to train and adjust the network parameters. SUMMARY

[0003] The application provides an expression recognition method based on a cross-scale local difference deep subspace feature. The method fuses local differences of different scales of an image to extract features, and has a simple network structure, does not require a large number of training samples, and has low hardware requirements.

[0004] Technical scheme: The application discloses an expression recognition method based on a cross-scale local difference deep subspace feature, which comprises the following steps:

[0005] S1, using different Gaussian filters to filter training sample images respectively, and constructing an image of each scale space with the filtered images and :

[0006]

[0007] S2, dividing the image of each scale space into a plurality of sub-blocks with a size of ,​​​​​​​​​​ , ;

[0008] For the image of the training sample image , scale space, the mean value of the th sub-block is calculated:

[0009]

[0010] The local difference sub-matrix of each sub-block under each scale is calculated , ;

[0011] The local difference of the th sub-block under the th scale is spliced into a cross-scale local difference sub-matrix:

[0012]

[0013] Convert into a cross-scale local difference sub-vector ;

[0014] The th cross-scale local difference sub-vector is spliced into a cross-scale local difference matrix of the training sample image

[0015] ;

[0016] S3, the th sample image in the training set is subjected to steps S2 and S3 to obtain a cross-scale local difference matrix, and the training set corresponds to a cross-scale local difference matrix:

[0017]

[0018] wherein is the cross-scale local difference matrix of the th sample image , , is the number of sample images in the training set;

[0019] S4, solve the first maximization problem:

[0020]

[0021] Obtain vectors of dimension , , , is the number of first-stage convolution kernels, Describing the L1 norm, express The identity matrix;

[0022] Will Mapped to The matrix, as the first phase of the first stage The parameter matrix of each convolution kernel :

[0023] in This represents the mapping function, and the superscript number 1 indicates the convolution kernel in the first stage;

[0024] S5, each image in the training set respectively with Convolve the first-stage convolution kernels to obtain the first-stage... Output images ;

[0025] S6, Each image output in the first stage The cross-scale local difference matrix is ​​obtained by following steps S2 and S3, thus forming the cross-scale local difference matrix corresponding to the first stage. :

[0026]

[0027] in The first stage output One image, ;

[0028] S7. Solving the second maximization problem:

[0029]

[0030] get Each dimension is vector , , This refers to the number of convolutional kernels in the second stage. express The identity matrix;

[0031] Will Mapped to The matrix, as the second phase... The parameter matrix of each convolution kernel :

[0032] The superscript number 2 indicates the convolution kernel in the second stage;

[0033] S8, output from the first stage Each image is related to The second-stage convolution kernel is used for convolution to obtain the second-stage... Output images ;

[0034] S9. For each image in the training sample set ,get Groups, each group For each group of images, Each image is processed non-linearly using binary hash encoding to obtain an image. :

[0035]

[0036] in This is a Heaviside step function, whose input is 1 when it is positive and zero otherwise. Output image for the second stage exist Pixel value at; For image exist Pixel value at;

[0037] S10, Size used is The window, with an overlap ratio In the image Swipe up to get Sub-image , Calculate the histogram of each subgraph and concatenate them into a histogram vector. :

[0038]

[0039] This represents histogram statistics;

[0040] S11. For each image in the training sample set ,connect From the histogram vectors, we obtain the image. Corresponding image features : ;

[0041] S12. Use the image features of each image in the training sample set. and corresponding emoji tags Training the facial expression classifier;

[0042] S13, the image to be recognized and Convolve the first-stage convolution kernels to obtain the first-stage... Each output image, for Each output image is respectively compared with The second-stage convolution kernel is used for convolution to obtain the second-stage... Each output image, for Each output image is processed according to steps S10-S12 to obtain the image features of the image to be recognized; the image features of the image to be recognized are input into the trained expression classifier to obtain the expression recognition result.

[0043] Furthermore, in step S4, the PCA-L1 algorithm is used to solve the first maximization problem.

[0044] Furthermore, the facial expression classifier in step S12 is an SVM-based classifier.

[0045] Furthermore, the facial expression classifier in step S12 is a decision tree-based classifier.

[0046] On the other hand, the present invention also discloses a system for implementing the above-mentioned expression recognition method based on cross-scale local difference deep subspace features, comprising:

[0047] Gaussian filter module 1 is used to employ Different Gaussian filters are applied to the training sample images respectively. Perform filtering and... constitute Images of various scales , :

[0048]

[0049] Image cross-scale local difference matrix generation module 2 is used to generate training sample images. Cross-scale local difference matrix The specific steps are as follows:

[0050] Image The image at each scale space is divided into The size is The sub-block, the first Images of various scales The Middle The pixels in each sub-block are , , ;

[0051] For training sample images of The image at the i-th scale space is used to calculate the i-th... The average value of each sub-block:

[0052]

[0053] Calculate the local difference submatrix for each sub-block at each scale. ,in ;

[0054] The scale in the first The local differences of each sub-block are concatenated to form a cross-scale local difference submatrix:

[0055]

[0056] Will Transform into cross-scale local difference subvectors ;

[0057] The multi-scale local difference subvectors are concatenated to form the training sample image. Cross-scale local difference matrix:

[0058] ;

[0059] Training set cross-scale local difference matrix generation module 3 is used to generate the training set local difference matrix. The cross-scale local difference matrix is ​​obtained from each sample image, and the cross-scale local difference matrix corresponding to the training set is constructed as follows:

[0060]

[0061] in For the first Sample images The cross-scale local difference matrix, , The number of sample images in the training set;

[0062] The first-stage convolutional kernel generation module 4 is used to generate the first-stage convolutional kernels. Each convolutional kernel includes the following steps:

[0063] Solving the first maximization problem:

[0064]

[0065] get Each dimension is vector , , This represents the number of convolutional kernels in the first stage. Describing the L1 norm, express The identity matrix;

[0066] Will Mapped to The matrix, as the first phase of the first stage The parameter matrix of each convolution kernel :

[0067] in This represents the mapping function, and the superscript number 1 indicates the convolution kernel in the first stage;

[0068] The first-stage convolutional module 5 is used to respectively employ... The first-stage convolutional kernel convolves the image;

[0069] The first-stage cross-scale local difference matrix generation module 6 is used to obtain the cross-scale local difference matrix from the image output in the first stage.

[0070] The second-stage convolutional kernel generation module 7 is used to generate the second-stage convolutional kernels. Each convolutional kernel includes the following steps:

[0071] Solving the second maximization problem:

[0072]

[0073] get Each dimension is vector , , This refers to the number of convolutional kernels in the second stage. express The identity matrix;

[0074] Will Mapped to The matrix, as the second phase... The parameter matrix of each convolution kernel :

[0075] The superscript number 2 indicates the convolution kernel in the second stage;

[0076] The second-stage convolutional module 8 is used to respectively employ... A second-stage convolutional kernel convolves the image;

[0077] Nonlinear processing module 9 is used to perform nonlinear processing on the image output from the second stage using binary hash encoding to obtain the image. :

[0078]

[0079] in This is a Heaviside step function, whose input is 1 when it is positive and zero otherwise. Output image for the second stage exist Pixel value at; For image exist Pixel value at;

[0080] Histogram vector generation module 10 is used to generate histogram vectors of size 10. The window, with an overlap ratio In the image Swipe up to get Sub-image , Calculate the histogram of each subgraph and concatenate them into a histogram vector. :

[0081]

[0082] This represents histogram statistics;

[0083] Image feature generation module 11 is used to process the output of the first stage. By concatenating the histogram vectors corresponding to each image, the corresponding image features can be obtained;

[0084] The expression classifier training module 12 is used to employ image features from each image in the training sample set. and corresponding emoji tags Training the facial expression classifier;

[0085] The recognition module 13 is used to obtain the expression recognition result after the image features of the image to be recognized are input into the expression classifier.

[0086] The present invention also discloses a computer storage medium storing a computer program that, when executed by a processor, implements the above-described expression recognition method based on cross-scale local difference depth subspace features.

[0087] The present invention also discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-mentioned expression recognition method based on cross-scale local difference depth subspace features.

[0088] Beneficial Effects: This invention discloses an expression recognition method and system based on cross-scale local difference deep subspace features. It constructs a cross-scale local difference matrix by fusing local differences at different scales, thereby training a two-stage convolutional kernel. The image to be recognized undergoes two stages of convolution, followed by nonlinear processing and histogram statistics to construct image features for classification and recognition. The expression recognition method and system disclosed in this invention extract image features that fuse local differences at different scales, and only involves two stages of convolution. Compared with deep learning methods, it requires no large number of training samples, no GPU, and has low hardware requirements. Attached Figure Description

[0089] Figure 1 This is a flowchart of the facial expression recognition method based on cross-scale local difference deep subspace features disclosed in this invention;

[0090] Figure 2 A schematic diagram for calculating local difference subvectors across scales;

[0091] Figure 3 This is a schematic diagram of the composition of the facial expression recognition system based on cross-scale local difference deep subspace features disclosed in this invention. Detailed Implementation

[0092] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0093] This invention discloses an expression recognition method based on cross-scale local difference deep subspace features, such as... Figure 1 As shown, the steps include:

[0094] S1, Use Different Gaussian filters are applied to the training sample images respectively. Perform filtering and... constitute Images of various scales , :

[0095]

[0096] Where * denotes convolution operation, Indicates the first A Gaussian filter. A filter bank is formed by several Gaussian filters, and the variance of each filter is... The values ​​are all different; a filter bank is used to filter the images in the training sample set to obtain filtered images of the same image with different sharpness, that is, the images are transformed to different scale spaces.

[0097] S2. Based on the fact that multi-scale information is insensitive to noise and scale, in order to capture local texture features at multiple scales, the image in each scale space is divided into... The size is The sub-block, the first Images of various scales The Middle The pixels in each sub-block are , , That is, the center of each sub-block is ;

[0098] For training sample images of The image at the i-th scale space is used to calculate the i-th... The average value of each sub-block:

[0099]

[0100] Calculate the local difference submatrix for each sub-block at each scale. ,in That is, mean-reduction processing is performed on each sub-block to suppress light sensitivity.

[0101] The scale in the first The local differences of each sub-block are concatenated to form a cross-scale local difference submatrix:

[0102]

[0103] Will Transform into cross-scale local difference subvectors ;

[0104] like Figure 2 The image shows an example of calculating local difference subvectors across scales. , That is, there are 3 scales, and each sub-block is .

[0105] The multi-scale local difference subvectors are concatenated to form the training sample image. Cross-scale local difference matrix:

[0106] ;

[0107] S3, training set For each sample image, the cross-scale local difference matrix is ​​obtained according to steps S2 and S3, forming the cross-scale local difference matrix corresponding to the training set:

[0108]

[0109] in For the first Sample images The cross-scale local difference matrix, , The number of sample images in the training set;

[0110] S4. Since L1-norm principal component analysis (PCA) is more robust to outliers and noise than L2-norm PCA, this invention uses L1-norm PCA based on the cross-scale local difference matrix corresponding to the training set. To learn the parameters of the first-stage convolution kernel, this is actually a problem of maximizing the variance based on the L1 norm in the feature space, i.e., solving the following first maximization problem:

[0111]

[0112] This embodiment uses the PCA-L1 algorithm to solve the first maximization problem, obtaining... Each dimension is vector , , This represents the number of convolutional kernels in the first stage. Describing the L1 norm, express The identity matrix;

[0113] Will Mapped to The matrix, as the first phase of the first stage The parameter matrix of each convolution kernel :

[0114] in This represents the mapping function, and the superscript number 1 indicates the convolution kernel in the first stage;

[0115] S5, each image in the training set respectively with Convolve the first-stage convolution kernels to obtain the first-stage... Output images ;

[0116] S6, Each image output in the first stage The cross-scale local difference matrix is ​​obtained by following steps S2 and S3, thus forming the cross-scale local difference matrix corresponding to the first stage. :

[0117]

[0118] in The first stage output One image, ;

[0119] S7. Solve the following second maximization problem using the PCA-L1 algorithm:

[0120]

[0121] get Each dimension is vector , , This refers to the number of convolutional kernels in the second stage. express The identity matrix;

[0122] Will Mapped to The matrix, as the second phase... The parameter matrix of each convolution kernel :

[0123] The superscript number 2 indicates the convolution kernel in the second stage;

[0124] S8, output from the first stage Each image is related to The second-stage convolution kernel is used for convolution to obtain the second-stage... Output images ;

[0125] S9. For each image in the training sample set ,get Groups, each group For each group of images, Each image is processed non-linearly using binary hash encoding to obtain an image. :

[0126]

[0127] in This is a Heaviside step function, whose input is 1 when it is positive and zero otherwise. Output image for the second stage exist Pixel value at; For image exist Pixel value at;

[0128] S10, Size used is The window, with an overlap ratio In the image Swipe up to get Sub-image , Calculate the histogram of each subgraph and concatenate them into a histogram vector. :

[0129]

[0130] This represents histogram statistics;

[0131] S11. For each image in the training sample set ,connect From the histogram vectors, we obtain the image. Corresponding image features : ;

[0132] S12. Use the image features of each image in the training sample set. and corresponding emoji tags Training the facial expression classifier;

[0133] Facial expression classifiers can be based on SVM or decision trees.

[0134] Steps S1-S12 are the training phase of the facial expression classifier, and the recognition phase is step S13:

[0135] S13, the image to be recognized and Convolve the first-stage convolution kernels to obtain the first-stage... Each output image, for Each output image is respectively compared with The second-stage convolution kernel is used for convolution to obtain the second-stage... Each output image, for Each output image is processed according to steps S10-S12 to obtain the image features of the image to be recognized; the image features of the image to be recognized are input into the trained expression classifier to obtain the expression recognition result.

[0136] A system that implements the above-mentioned expression recognition method based on cross-scale local difference deep subspace features, such as Figure 3 ,include:

[0137] Gaussian filter module 1 is used to employ Different Gaussian filters are applied to the training sample images respectively. Perform filtering and... constitute Images of various scales , :

[0138]

[0139] Image cross-scale local difference matrix generation module 2 is used to generate training sample images. Cross-scale local difference matrix The specific steps are as follows:

[0140] Image The image at each scale space is divided into The size is The sub-block, the first Images of various scales The Middle The pixels in each sub-block are , , ;

[0141] For training sample images of The image at the i-th scale space is used to calculate the i-th... The average value of each sub-block:

[0142]

[0143] Calculate the local difference submatrix for each sub-block at each scale. ,in ;

[0144] The scale in the first The local differences of each sub-block are concatenated to form a cross-scale local difference submatrix:

[0145]

[0146] Will Transform into cross-scale local difference subvectors ;

[0147] The multi-scale local difference subvectors are concatenated to form the training sample image. Cross-scale local difference matrix:

[0148] ;

[0149] Training set cross-scale local difference matrix generation module 3 is used to generate the training set local difference matrix. The cross-scale local difference matrix is ​​obtained from each sample image, and the cross-scale local difference matrix corresponding to the training set is constructed as follows:

[0150]

[0151] in For the first Sample images The cross-scale local difference matrix, , The number of sample images in the training set;

[0152] The first-stage convolutional kernel generation module 4 is used to generate the first-stage convolutional kernels. Each convolutional kernel includes the following steps:

[0153] Solving the first maximization problem:

[0154]

[0155] get Each dimension is vector , , This represents the number of convolutional kernels in the first stage. Describing the L1 norm, express The identity matrix;

[0156] Will Mapped to The matrix, as the first phase of the first stage The parameter matrix of each convolution kernel :

[0157] in This represents the mapping function, and the superscript number 1 indicates the convolution kernel in the first stage;

[0158] The first-stage convolutional module 5 is used to respectively employ... The first-stage convolutional kernel convolves the image;

[0159] The first-stage cross-scale local difference matrix generation module 6 is used to obtain the cross-scale local difference matrix from the image output in the first stage.

[0160] The second-stage convolutional kernel generation module 7 is used to generate the second-stage convolutional kernels. Each convolutional kernel includes the following steps:

[0161] Solving the second maximization problem:

[0162]

[0163] get Each dimension is vector , , This refers to the number of convolutional kernels in the second stage. express The identity matrix;

[0164] Will Mapped to The matrix, as the second phase... The parameter matrix of each convolution kernel :

[0165] The superscript number 2 indicates the convolution kernel in the second stage;

[0166] The second-stage convolutional module 8 is used to respectively employ... A second-stage convolutional kernel convolves the image;

[0167] Nonlinear processing module 9 is used to perform nonlinear processing on the image output from the second stage using binary hash encoding to obtain the image. :

[0168]

[0169] in This is a Heaviside step function, whose input is 1 when it is positive and zero otherwise. Output image for the second stage exist Pixel value at; For image exist Pixel value at;

[0170] Histogram vector generation module 10 is used to generate histogram vectors of size 10. The window, with an overlap ratio In the image Swipe up to get Sub-image , Calculate the histogram of each subgraph and concatenate them into a histogram vector. :

[0171]

[0172] This represents histogram statistics;

[0173] Image feature generation module 11 is used to process the output of the first stage. By concatenating the histogram vectors corresponding to each image, the corresponding image features can be obtained;

[0174] The expression classifier training module 12 is used to employ image features from each image in the training sample set. and corresponding emoji tags Training the facial expression classifier;

[0175] The recognition module 13 is used to obtain the expression recognition result after the image features of the image to be recognized are input into the expression classifier.

[0176] The present invention also discloses a computer storage medium storing a computer program that, when executed by a processor, implements the above-described expression recognition method based on cross-scale local difference depth subspace features.

[0177] The present invention also discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-mentioned expression recognition method based on cross-scale local difference depth subspace features.

[0178] This embodiment compares the facial expression recognition method disclosed in this invention with commonly used recognition methods on four publicly available facial expression datasets: the CK+ dataset, the JAFFE dataset, the ISED dataset, and the BAUM-2i dataset. The comparison results are shown in Tables 1-4.

[0179] The CK+ dataset is a set of posed facial expression images in a controlled scene. The facial expression recognition accuracy of each method is shown in Table 1.

[0180] Table 1

[0181] Method Feature type Accuracy (%) PCANet Deep subspace feature 78.74 K-PCANet Deep subspace feature 81.89 E-PCANet Deep subspace feature 85.66 Inception Deep learning 93.20 AUDN (AURF) Deep learning 92.22 P-PCANet Deep subspace feature 92.67 ILCNN Deep learning 94.39 DAE Deep learning 95.79 CLGDNP Hand-crafted feature 94.27 LDSP Hand-crafted feature 94.49 LSDP Hand-crafted feature 94.68 3DCNN-DAP Deep learning 92.40 The method of the present invention Deep subspace feature 96.26

[0182] The JAFFE database is also a set of posed facial expression images in controlled scenes. The facial expression recognition accuracy of each method is shown in Table 2.

[0183] Table 2

[0184] Method Feature type Accuracy (%) PCANet Deep subspace feature 58.35 LAP Hand-crafted feature 68.54 K-PCANet Deep subspace feature 68.80 WLD Hand-crafted feature 69.01 E-PCANet Deep subspace feature 69.40 LPQ Hand-crafted feature 69.95 STL-CNN Deep learning 69.35 LSDP Hand-crafted feature 70.89 P-PCANet Deep subspace feature 71.38 The method of the present invention Deep subspace feature 73.15

[0185] The ISED database is a collection of spontaneous facial expression images in controlled scenarios. The facial expression recognition accuracy of each method is shown in Table 3.

[0186] Table 3

[0187] Method Feature type Accuracy (%) PCANet Deep subspace feature 73.78 NEDP ns ]]> Hand-crafted feature 77.39 NEDP s ]]> Hand-crafted feature 77.78 LSP+LB Hand-crafted feature 77.82 sLSP+LB Hand-crafted feature 78.03 The method of the present invention Deep subspace feature 80.71

[0188] The BAUM-2i database is a collection of spontaneous facial expression images in uncontrolled scenes. The facial expression recognition accuracy of each method is shown in Table 4.

[0189] Table 4

[0190] Method Feature type Accuracy (%) Baseline Hand-crafted feature 57.77 MBC_P Hand-crafted feature 58.10 LBP Hand-crafted feature 58.32 PCANet Deep subspace feature 60.83 DAM-CNN Deep learning 61.52 The method of the present invention Deep subspace feature 62.13

[0191] The results above show that the facial expression recognition method disclosed in this invention has the highest accuracy among all datasets.

Claims

1. A facial expression recognition method based on cross-scale local difference deep subspace features, characterized in that, Including the following steps: S1, Use Different Gaussian filters are applied to the training sample images respectively. Perform filtering and... constitute Images of various scales , : ； S2. Divide the image at each scale space into... The size is The sub-block, the first Images of various scales The Middle The pixels in each sub-block are , , ; For training sample images of The image at the i-th scale space is used to calculate the i-th... The average value of each sub-block: ； Calculate the local difference submatrix for each sub-block at each scale. ,in ; The scale in the first The local differences of each sub-block are concatenated to form a cross-scale local difference submatrix: ； Will Transform into cross-scale local difference subvectors ; The multi-scale local difference subvectors are concatenated to form the training sample image. Cross-scale local difference matrix: ； S3, training set For each sample image, a cross-scale local difference matrix is ​​obtained, which forms the cross-scale local difference matrix corresponding to the training set: ； in For the first Sample images The cross-scale local difference matrix, , The number of sample images in the training set; S4. Solve the first maximization problem: ； get Each dimension is vector , , This represents the number of convolutional kernels in the first stage. Describing the L1 norm, express The identity matrix; Will Mapped to The matrix, as the first stage The parameter matrix of each convolution kernel : ; in This represents the mapping function, and the superscript number 1 indicates the convolution kernel in the first stage; S5, each image in the training set respectively with Convolve the first-stage convolution kernels to obtain the first-stage... Output images ; S6, Each image output in the first stage All cross-scale local difference matrices were obtained, forming the cross-scale local difference matrix corresponding to the first stage. : ； in The first stage output One image, ; S7. Solving the second maximization problem: ； get Each dimension is vector , , This refers to the number of convolutional kernels in the second stage. express The identity matrix; Will Mapped to The matrix, as the second phase... The parameter matrix of each convolution kernel : ; The superscript number 2 indicates the convolution kernel in the second stage; S8, output from the first stage Each image is related to The second-stage convolution kernel is used for convolution to obtain the second-stage... Output images ; S9. For each image in the training sample set ,get Groups, each group Images, for each group Each image is processed non-linearly using binary hash encoding to obtain an image. : ； in This is a Heaviside step function, whose input is 1 when it is positive and zero otherwise. Output image for the second stage exist Pixel value at; For image exist Pixel value at; S10, Size used is The window, with an overlap ratio In the image Swipe up to get Sub-image , Calculate the histogram of each subgraph and concatenate them into a histogram vector. : ； This represents histogram statistics; S11. For each image in the training sample set ,connect From the histogram vectors, we obtain the image. Corresponding image features : ; S12. Use the image features of each image in the training sample set. and corresponding emoji tags Train the facial expression classifier; S13, the images to be recognized are respectively compared with... Convolve the first-stage convolution kernels to obtain the first-stage... One output image, for the first stage Each output image is respectively compared with The second-stage convolution kernel is used for convolution to obtain the second-stage... Each output image, for Each output image is processed according to steps S10-S12 to obtain the image features of the image to be recognized; the image features of the image to be recognized are input into the trained expression classifier to obtain the expression recognition result.

2. The facial expression recognition method based on cross-scale local difference deep subspace features according to claim 1, characterized in that, In step S4, the PCA-L1 algorithm is used to solve the first maximization problem.

3. The facial expression recognition method based on cross-scale local difference deep subspace features according to claim 1, characterized in that, The facial expression classifier in step S12 is an SVM-based classifier.

4. The facial expression recognition method based on cross-scale local difference deep subspace features according to claim 1, characterized in that, The facial expression classifier in step S12 is a decision tree-based classifier.

5. An expression recognition system based on cross-scale local difference deep subspace features, characterized in that, To implement the method according to any one of claims 1-4, comprising: Gaussian filter module (1), used to employ Different Gaussian filters are applied to the training sample images respectively. Perform filtering and... constitute Images of various scales , : ； Image cross-scale local difference matrix generation module (2) is used to generate training sample images. Cross-scale local difference matrix The specific steps are as follows: Image The image in each scale space is divided into The size is The sub-block, the first Images of various scales The Middle The pixels in each sub-block are , , ; For training sample images of The image at the i-th scale space is used to calculate the i-th... The average value of each sub-block: ； Calculate the local difference submatrix for each sub-block at each scale. ,in ; The scale in the first The local differences of each sub-block are concatenated to form a cross-scale local difference submatrix: ； Will Transform into cross-scale local difference subvectors ; The multi-scale local difference subvectors are concatenated to form the training sample image. Cross-scale local difference matrix: ； The training set cross-scale local difference matrix generation module (3) is used to generate the training set local difference matrix. The cross-scale local difference matrix is ​​obtained from each sample image, and the cross-scale local difference matrix corresponding to the training set is constructed as follows: ； in For the first Sample images The cross-scale local difference matrix, , The number of sample images in the training set; The first-stage convolutional kernel generation module (4) is used to generate the first-stage convolutional kernel. Each convolutional kernel includes the following steps: Solving the first maximization problem: ； get Each dimension is vector , , This represents the number of convolutional kernels in the first stage. Describing the L1 norm, express The identity matrix; Will Mapped to The matrix, as the first stage The parameter matrix of each convolution kernel : ; in This represents the mapping function, and the superscript number 1 indicates the convolution kernel in the first stage; The first-stage convolutional module (5) is used to respectively employ... The first-stage convolution kernel convolves the image; The first-stage cross-scale local difference matrix generation module (6) is used to obtain the cross-scale local difference matrix from the image output in the first stage. The second-stage convolutional kernel generation module (7) is used to generate the second-stage convolutional kernel. Each convolutional kernel includes the following steps: Solving the second maximization problem: ； get Each dimension is vector , , This refers to the number of convolutional kernels in the second stage. express The identity matrix; Will Mapped to The matrix, as the second phase... The parameter matrix of each convolution kernel : ; The superscript number 2 indicates the convolution kernel in the second stage; The second-stage convolution module (8) is used to respectively employ... A second-stage convolutional kernel convolves the image; The nonlinear processing module (9) performs nonlinear processing on the image output from the second stage using binary hash encoding to obtain the image. : ； in This is a Heaviside step function, whose input is 1 when it is positive and zero otherwise. Output image for the second stage exist Pixel value at; For image exist Pixel value at; Histogram vector generation module (10), used to generate histogram vectors of size 10 The window, with an overlap ratio In the image Swipe up to get Sub-image , Calculate the histogram of each subgraph and concatenate them into a histogram vector. : ； This represents histogram statistics; Image feature generation module (11) is used to process the output of the first stage. By concatenating the histogram vectors corresponding to each image, the corresponding image features can be obtained; The expression classifier training module (12) is used to employ image features from each image in the training sample set. and corresponding emoji tags Train the facial expression classifier; The recognition module (13) is used to obtain the expression recognition result obtained after the image features of the image to be recognized are input into the expression classifier.

6. The facial expression recognition system based on cross-scale local difference deep subspace features according to claim 5, characterized in that, The first stage convolution kernel generation module (4) uses the PCA-L1 algorithm to solve the first maximization problem.

7. The facial expression recognition system based on cross-scale local difference deep subspace features according to claim 5, characterized in that, The facial expression classifier in the training module (12) is an SVM-based classifier.

8. The facial expression recognition system based on cross-scale local difference deep subspace features according to claim 5, characterized in that, The facial expression classifier in the facial expression classifier training module (12) is a decision tree-based classifier.

9. A computer storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the facial expression recognition method based on cross-scale local difference depth subspace features as described in any one of claims 1-4.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the facial expression recognition method based on cross-scale local difference depth subspace features as described in any one of claims 1-4.

Images