An image patch-aware facial expression recognition method based on representation learning

Abstract

The present invention discloses an expression recognition method based on image block perception using representation learning. The method comprises the following steps: 1. constructing a data set based on preprocessing of original facial images; 2. locating expression-related key points in the image using a facial key point detection algorithm; 3. constructing an expression recognition network. The image is first divided into several small blocks and mapped into a feature space, thereby establishing relationships between key points and small blocks; two strategies are then used to process the small blocks containing key points, respectively, to obtain two mask images, and features are extracted through an encoder; finally, a decoder reconstructs the image using the features of the first mask image, and an expression classifier performs expression recognition based on the features of the two mask images. The present invention introduces facial key points into the model as prior knowledge for expression recognition and utilizes two complementary strategies to force the model to focus on subtle facial movements, thereby achieving accurate expression recognition.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning and computer vision, and specifically is a facial expression recognition method based on image block perception of representation learning. Background Art

[0002] As a branch of affective computing, facial expression recognition requires modeling fine-grained information in images to effectively extract facial expression-related features, which is very important for overcoming various influencing factors. Existing methods that consider both global and local features are mainly divided into two categories: key point-based methods and attention mechanism-based methods. Figure 2 The basic approaches of the above two methods are given. The key point-based method extracts facial features near key points related to expressions. This method uses prior knowledge of the face, that is, expressions are related to the movements of key areas of the face, and can therefore learn features related to expressions in a targeted manner. In contrast, the attention mechanism-based method is a data-driven method that automatically learns attention maps from massive amounts of data to identify the importance of different facial positions for expression recognition. Most existing methods use convolutional networks as their base model, but with the rise of visual transformers, some methods have also attempted to use them to better model long-range dependencies.

[0003] However, both keypoint-based and attention-based approaches have their own challenges. For the former, keypoint detection is crucial, but this cannot be guaranteed due to various factors. When keypoints cannot be located, the method becomes inapplicable. Furthermore, this method requires extracting local and global features from the vicinity of different keypoints and fusing them, which incurs significant computational overhead and requires consideration of appropriate fusion strategies. For the latter, data-driven approaches often lack effective supervision, resulting in generated attention maps that can only distinguish between different expressions to a certain extent, and are of limited use for expressions with minimal differences. Summary of the Invention

[0004] In order to address the shortcomings of the above-mentioned existing technologies, the present invention proposes an expression recognition method based on image block perception of representation learning, in order to learn the prior knowledge of facial key points related to expression, thereby extracting fine features related to expression to improve the accuracy and robustness of expression recognition.

[0005] In order to achieve the above-mentioned object, the present invention adopts the following technical solutions:

[0006] The feature of the facial expression recognition method based on image block perception of representation learning of the present invention is that it is performed according to the following steps:

[0007] Step 1: Construct facial expression image data set D;

[0008] Obtain a real face image dataset and preprocess it using cropping and normalization methods to obtain an expression image dataset Among them, I i , e i represents the i-th face image of size h×w and its corresponding expression category, e i ∈{1,2,…,C}, N is the number of images in the facial expression image dataset D, and C is the total number of facial expression categories;

[0009] Step 2: Use the facial key point detection algorithm to detect the i-th face image I i The positions of the J key points related to the expression are located to obtain the position coordinates of the J key points Among them, x i,j ,y i,j Represents the i-th face image I i The longitudinal and transverse coordinates of the j-th key point in ;

[0010] Step 3: Construct an expression recognition network, including: image block embedding layer, mask processing layer, visual encoder E, visual decoder D, and expression classifier R;

[0011] Step 3.1: The image block embedding layer is a convolution layer with a kernel size of a×a and a step size of a, which is used to embed the i-th face image I i Divide evenly into image blocks, and map each image block to the feature space to obtain the embedded features of each image block;

[0012] make The corresponding dimension of the image block is , then the position of the jth key point is In the matrix Row, No. Column, thereby establishing the relationship between key points and image blocks;

[0013] Step 3.2: Mask processing layer processing:

[0014] The mask processing layer sets the embedded features of the image block where each key point is located to a full zero vector with a certain probability p, thereby obtaining the i-th face image I i The first mask image I i1 ;

[0015] The mask processing layer sets the embedded features of the image blocks where all key points are located and the image blocks above, below, left, and right to all zero vectors, thereby obtaining the i-th face image I iThe second mask image I i2 ;

[0016] Step 3.3, the visual encoder E respectively encodes the first mask image I i1 and the second mask image I i2 Processing is performed to obtain the first mask image I i1 The encoded features and the second mask image I i2 The encoding feature f i2 ;

[0017] Step 3.4, the visual decoder D encodes the feature f i1 Process and obtain the reconstructed image I' i ;

[0018] Use formula (4) to calculate the reconstruction loss L of the visual decoder D rec :

[0019]

[0020] In formula (4), ||·||2 represents the second normal form of the matrix;

[0021] Step 3.5, construct the expression classifier R which is a fully connected layer;

[0022] Step 3.5.1, encode the feature f i1 and f i2 Input them into the classifier R respectively, and get two expression probability distributions logit accordingly i1 and logit i2 ;

[0023] Step 3.5.2: Use equations (5) and (6) to construct the expression classification loss L of the expression classifier R. cls and KL divergence loss L kl :

[0024]

[0025]

[0026] In formula (5), CEL(·,·) represents the cross entropy loss; and Represents the i-th face image I i Expression category e i and expression probability distribution logit i1 The probability value of the kth category in;

[0027] In formula (6), KLL(·,·) represents KL divergence loss, and ud represents uniform distribution; and ud kRepresents the probability distribution logit of expression i2 and the probability value of the kth category in the uniform distribution ud;

[0028] Step 3.6: Use formula (7) to construct the total classification loss L all :

[0029] L all =α×L rec +β×L cls +γ×L kl (7)

[0030] In formula (7), α, β and γ represent three weight factors;

[0031] Step 4: Use the gradient descent algorithm to train the expression recognition network and calculate the total classification loss L all To update the network parameters until the total classification loss L all Until convergence is reached, a trained expression recognition and classification model is obtained, which is used to classify images of different facial expressions.

[0032] The feature of the expression recognition method based on image block perception of representation learning of the present invention is that the visual encoder E is composed of L layers of encoding layers, wherein each encoding layer is composed of a multi-head self-attention sublayer and a feedforward sublayer connected in a residual manner;

[0033] The visual encoder E uses formula (1) to encode the first mask image I i1 Processing is performed to obtain the hidden layer feature h of the l+1th encoding layer l+1 , and thus use formula (2) to obtain the encoding feature w output by the l+1th encoding layer l+1 , and finally the L-th encoding layer outputs the encoding feature w L , and as the visual encoder E for the first mask image I i1 The encoding feature f i1 :

[0034] h l+1 =LN(w l +MSA(w l )) (1)

[0035] w l+1 =LN(h l+1 +FF(h l+1 )) (2

[0036] In formula (1), MSA(·) represents the multi-head self-attention sub-layer, LN(·) represents the layer regularization function, and w l Represents the encoding features output by the lth encoding layer. When l = 1, let w l =Ii1 or I i2 ;

[0037] In formula (2), FF(·) represents the feedforward sublayer, which contains two fully connected layers and a ReLU activation function, and has:

[0038] FF(h l+1 )=max(0,h l+1 W1+b1)W2+b2 (3)

[0039] In formula (3), W1 and W2 represent the two weight parameters to be learned, b1 and b2 represent the two bias parameters to be learned;

[0040] The visual encoder E follows the same process to encode the second mask image I i2 Processing is performed to finally obtain the second mask image I i2 The encoding feature f i2 .

[0041] The visual decoder D consists of a spatial transformation layer, a mask permutation layer, M decoding layers, and an image reconstruction layer. Each decoding layer consists of a multi-head self-attention sublayer and a feedforward sublayer connected in a residual manner.

[0042] Step 3.4.1, the spatial transformation layer is a linear layer used to transform the encoded feature f i1 Mapping to a low-dimensional space to obtain the reduced-dimensional features

[0043] Step 3.4.2, the mask permutation layer will reduce the dimension of the features In the image, the part corresponding to the image block where the selected key point is located is replaced with the mask symbol, thereby obtaining the feature after mask replacement

[0044] Step 3.4.3. Features after mask replacement After processing through the M layers of decoding layers, the decoding features are obtained

[0045] Step 3.4.4, the image reconstruction layer is a linear layer used to transform the decoded features Mapped to the image space, the reconstructed image I' is obtained i .

[0046] The electronic device of the present invention includes a memory and a processor, and is characterized in that the memory is used to store a program that supports the processor to execute an expression recognition method, and the processor is configured to execute the program stored in the memory.

[0047] The present invention provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, which is characterized in that the computer program executes the steps of the expression recognition method when the computer program is run by a processor.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] 1. The present invention uses expression-related facial key points as a priori knowledge. These key points can be used to divide the divided image blocks into two categories: expression-related and expression-irrelevant, and two strategies are used to process them respectively, so that the model can learn fine expression features.

[0050] 2. This invention reconstructs some expression-related regions, enabling the model to better learn expression-related features. It also explicitly considers expression-irrelevant regions, preventing the model from distinguishing different expressions when only input is small, irrelevant blocks, thereby reducing the influence of background, hair, and face shape on expression recognition. These two strategies enable the model to automatically perceive the contribution of different blocks to expression recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A structural diagram of the method proposed in the present invention;

[0052] Figure 2 A comparison chart of the present invention with two methods based on key points and attention mechanism;

[0053] Figure 3 This is the masking strategy of the present invention and the visualization effect after reconstruction. DETAILED DESCRIPTION

[0054] In this embodiment, see Figure 1 , an expression recognition method based on image block perception based on representation learning is carried out in the following steps:

[0055] Step 1: Construct facial expression image data set D;

[0056] Obtain a real face image dataset and preprocess it using cropping and normalization methods to obtain an expression image dataset Among them, I i , e i represents the i-th face image of size h×w and its corresponding expression category, e i ∈{1,2,…,C}, N is the number of images in the facial expression image dataset D, and C is the total number of facial expression categories;

[0057] In this embodiment, the facial expression dataset BU-3DFE is a 3D model dataset containing 100 subjects, including 44 males and 56 females. Facial images are generated from the 3D model and annotated into seven expressions: anger, disgust, fear, happiness, sadness, surprise, and neutral. The six basic expressions, except for neutral, all have four levels of intensity. In this experiment, only the six basic expressions are considered and three experimental settings are followed. The three settings respectively contain 12,000 images from five angles, 16,000 images from seven angles, and 21,000 images from 35 angles. The third setting only uses images with four intensity levels. The subjects were randomly divided into a training set of 80 individuals and a test set of 20 individuals. The facial expression dataset RAF-DB is a real-world facial expression dataset. It contains 29,672 face images collected through the Flickr image search API and independently labeled by 40 trained humans. In the experiment, images were labeled into seven expressions, with the training and test sets containing 12,271 and 3,068 images, respectively. Facial regions were detected using OpenCV. For the RAF-DB dataset, faces were located and aligned using five annotated keypoints. Finally, all images were cropped and resized to 224×224.

[0058] Step 2: Use the facial key point detection algorithm to detect the i-th face image I i The positions of the J key points related to the expression are located to obtain the position coordinates of the J key points Among them, x i,j ,y i,j Represents the i-th face image I i The vertical and horizontal coordinates of the jth keypoint in the image are as follows; in this example, the Dlib tool was used to extract 68 facial keypoints. Keypoints 1 to 17 are located in the lower half of the facial contour, while keypoints 18 to 68 are located in the areas surrounding the eyebrows, eyes, nose, and mouth. Because the behavior of these areas is directly related to facial expression, the keypoints contained in them are considered relevant to facial expression, resulting in a total of 51 keypoints.

[0059] Step 3: Construct an expression recognition network, including: image block embedding layer, mask processing layer, visual encoder E, visual decoder D, and expression classifier R;

[0060] Step 3.1: The image block embedding layer is a convolution layer with a kernel size of a×a and a step size of a, which is used to embed the i-th face image I i Divide evenly into Image blocks are mapped into the feature space to obtain the embedded features of each image block. In this embodiment, the convolution kernel size is 16×16 and the stride is 16. Therefore, the image block embedding layer divides the image of size 224×224 into 196 (14×14) small blocks of size 16×16. These small blocks are embedded into the feature space of dimension 1024, thereby obtaining the embedded features of each image block.

[0061] make The corresponding dimension of the image block is , then the position of the jth key point is In the matrix Row, No. Columns are created to establish a relationship between key points and image blocks. In this embodiment, the image blocks are organized into a 14×14 matrix, so the blocks where the key points are located can be found based on a simple mapping relationship. The behavior within these blocks is directly related to the expression.

[0062] Step 3.2: Mask processing layer processing:

[0063] The mask processing layer sets the embedded features of the image block where each key point is located to a full zero vector with a certain probability p, thereby obtaining the i-th face image I i The first mask image I i1 In this embodiment, p is set to 30%. For each image block where a key point is located, its embedded features have a 30% probability of being set to a zero vector, thereby no longer containing expression-related information in the area. Mask image I i1 like Figure 3 As shown in the second line of .

[0064] The mask processing layer sets the embedded features of the image blocks where all key points are located and the image blocks above, below, left, and right to all zero vectors, thereby obtaining the i-th face image I i The second mask image I i2 In this embodiment, the upper, lower, left and right image blocks are also set to all zero vectors, the purpose of which is to make the mask image I i2 It contains no information related to facial expressions. Mask image I i2 like Figure 3 As shown in the third row.

[0065] Step 3.3, construct a visual encoder E consisting of L encoding layers, where each encoding layer is composed of a multi-head self-attention sublayer and a feedforward sublayer connected in a residual manner; in this embodiment, the number of layers of encoder E is set to 24, the hidden layer size is 1024, the number of multi-heads is 16, and it is initialized using MAE pre-training parameters.

[0066] The visual encoder E uses formula (1) to encode the first mask image I i1 Processing is performed to obtain the hidden layer feature h of the l+1th encoding layer l+1 , and thus use formula (2) to obtain the encoding feature w output by the l+1th encoding layer l+1 , and finally the L-th encoding layer outputs the encoding feature w L , and as the visual encoder E for the first mask image I i1 The encoding feature f i1 :

[0067] h l+1 =LN(w l +MSA(w l )) (1)

[0068] w l+1 =LN(h l+1 +FF(h l+1 )) (2)

[0069] In formula (1), MSA(·) represents the multi-head self-attention sub-layer, LN(·) represents the layer regularization function, and w l Represents the encoding features output by the lth encoding layer. When l = 1, let w l =I i1 or I i2 In this embodiment, the multi-head self-attention module links each image patch with other image patches using certain weights, thereby achieving global image modeling. The layer regularization function is used to ensure the stability of feature distribution.

[0070] In formula (2), FF(·) represents the feedforward sublayer, which contains two fully connected layers and a ReLU activation function, and has:

[0071] FF(h l+1 )=max(0,h l+1 W1+b1)W2+b2 (3)

[0072] In formula (3), W1 and W2 represent the two weight parameters to be learned, b1 and b2 represent the two bias parameters to be learned;

[0073] The visual encoder E follows the same process to encode the second mask image I i2 Processing is performed to finally obtain the second mask image I i2 The encoding feature f i2 ;

[0074] Step 3.4: Construct a visual decoder D consisting of a spatial transformation layer, a mask permutation layer, M decoding layers, and an image reconstruction layer, where each decoding layer is composed of a multi-head self-attention sublayer and a feedforward sublayer connected in a residual manner; in this embodiment, the number of layers M is set to 8, the hidden layer size and the number of multi-heads of the decoding layer are 512 and 16, respectively, and are initialized with random parameters.

[0075] Step 3.4.1, the spatial transformation layer is a linear layer used to transform the encoded feature f i1 Mapping to a low-dimensional space to obtain the reduced-dimensional features In this embodiment, since the hidden layer size of the decoding layer is 512, it is necessary to first map the 1024-dimensional features obtained from the encoder E to a 512-dimensional feature space through a linear layer.

[0076] Step 3.4.2, the mask permutation layer will reduce the dimension of the features In the image, the part corresponding to the image block where the selected key point is located is replaced with the mask symbol, thereby obtaining the feature after mask replacement In this embodiment, the mask symbol is a learnable parameter with a dimension of 512.

[0077] Step 3.4.3. Features after mask replacement After processing through the M layers of decoding layers, the decoding features are obtained

[0078] Step 3.4.4, the image reconstruction layer is a linear layer used to transform the decoded features Mapped to the image space, the reconstructed image I' is obtained i In this embodiment, the reconstructed image I ' i like Figure 3 As shown in the fourth row.

[0079] Step 3.4.5: Calculate the reconstruction loss L of the visual decoder D using formula (4) rec :

[0080]

[0081] In formula (4), ||·||2 represents the second normal form of the matrix; in this embodiment, although the reconstruction is for the entire image, the reconstruction loss L rec Only the difference between the reconstruction of the image block where the selected key point is located and the corresponding position of the original image is considered. When the key point cannot be detected, the mask image I i1 No reconstruction is required, and the reconstruction loss L rec is zero, which is consistent with the overall optimization direction. Figure 3 The second and fourth rows of the last column are shown in the image.

[0082] Step 3.5, construct the expression classifier R which is a fully connected layer;

[0083] Step 3.5.1, encode the feature f i1 and f i2 Input them into the classifier R respectively, and get two expression probability distribution logit accordingly i1 and logit i2 ;

[0084] Step 3.5.2: Use equations (5) and (6) to construct the expression classification loss L of the expression classifier R. cls and KL divergence loss L kl :

[0085]

[0086]

[0087] In formula (5), CEL(·,·) represents the cross entropy loss; and Represents the i-th face image I i Expression category e i and expression probability distribution logit i1 The probability value of the kth category in .

[0088] In formula (6), KLL(·,·) represents KL divergence loss, and ud represents uniform distribution; and ud k Represents the probability distribution logit of expression i2 And the probability value of the kth category in the uniform distribution ud. In this embodiment, the KL divergence loss makes the probability distribution logit i2 Approximate to a uniform distribution. When the key point cannot be detected, the mask image I i2 All expression-related image blocks in are set to all-zero vectors, which do not contain any expression-related information. Figure 3 The third row of the last column is shown in the image.

[0089] Step 3.6: Use formula (7) to construct the total classification loss L all :

[0090] L all =α×L rec +β×L cls +γ×L kl (7)

[0091] In formula (7), α, β and γ represent three weight factors;

[0092] Step 4: Use the gradient descent algorithm to train the expression recognition network and calculate the total classification loss L all To update the network parameters until the total classification loss L all The training process continued until convergence was achieved, resulting in a trained expression recognition and classification model for classifying images of different facial expressions. For all experiments, the batch size and training epochs were set to 8 and 50, respectively. The network was optimized using the AdamW optimizer with an initial learning rate of 3.125e-5, 5 warmup epochs, and a decreasing cosine learning rate. The weight factors in the total loss function were set to 1, 2, and 1.5, respectively.

[0093] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0094] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are executed.

[0095] Example:

[0096] To verify the effectiveness of the proposed method, experiments were conducted on the datasets BU-3DFE and RAF-DB. Tables 1 and 2 show the comparison results of the proposed method and the existing methods on the two datasets, respectively.

[0097] Table 1 Expression recognition accuracy compared with existing methods on the BU-3DFE dataset

[0098]

[0099] Table 2 Expression recognition accuracy compared with existing methods on the RAF-DB dataset

[0100]

[0101]

[0102] The experimental results in Table 1 and Table 2 show that the present invention has better effects than other methods, thereby proving the feasibility of the method proposed in the present invention.

Claims

1. A facial expression recognition method based on image block perception based on representation learning, characterized in that: The steps are as follows: Step 1: Construct facial expression image data set D; Obtain a real face image dataset and preprocess it using cropping and normalization methods to obtain an expression image dataset ,in, , Respectively indicate the size The i-th face image and its corresponding expression category, , N is the number of images in the facial expression image dataset D, and C is the total number of facial expression categories; Step 2: Use the facial key point detection algorithm to detect the i-th face image The positions of the J key points related to the expression are located to obtain the position coordinates of the J key points ,in, , Represents the i-th face image The longitudinal and transverse coordinates of the j-th key point in ; Step 3: Construct an expression recognition network, including: image block embedding layer, mask processing layer, visual encoder E, visual decoder, and expression classifier R; Step 3.1, the image block embedding layer is a convolution kernel size of , a convolutional layer with a step size of a, used to transform the i-th face image Divide evenly into image blocks, and map each image block to the feature space to obtain the embedded features of each image block; make The corresponding dimension of the image block is , then the position of the jth key point is In the matrix Row, No. Column, thereby establishing the relationship between key points and image blocks; Step 3.2: Mask processing layer processing: The mask processing layer sets the embedded features of the image block where each key point is located to a full zero vector with a certain probability p, thereby obtaining the i-th face image The first mask image ; The mask processing layer sets the embedded features of the image blocks where all key points are located and the image blocks above, below, left and right to all zero vectors, thereby obtaining the i-th face image The second mask image ; Step 3.3, the visual encoder E respectively encodes the first mask image and the second mask image Processing is performed to obtain the first mask image The encoding features and the second mask image The encoding features ; Step 3.4: The visual decoder decodes the encoded features Process and obtain the reconstructed image ; Use formula (4) to calculate the reconstruction loss of the visual decoder : (4) In formula (4), Represents the second normal form of a matrix; Step 3.5, construct the expression classifier R which is a fully connected layer; Step 3.5.1: The coding feature and are input into the classifier R respectively, and two expression probability distributions are obtained accordingly and ; Step 3.5.2: Use equations (5) and (6) to construct the expression classification loss of the expression classifier R. and KL divergence loss : (5) (6) In formula (5), represents the cross entropy loss; and Represents the i-th face image Emoji category and the expression probability distribution The probability value of the kth category in; In formula (6), represents the KL divergence loss, represents uniform distribution; and Respectively represent the expression probability distribution and uniform distribution The probability value of the kth category in; Step 3.6: Use formula (7) to construct the total classification loss : (7) In formula (7), , and represents three weight factors; Step 4: Use the gradient descent algorithm to train the expression recognition network and calculate the total classification loss. To update the network parameters until the total classification loss Until convergence is reached, a trained expression recognition and classification model is obtained, which is used to classify images of different facial expressions.

2. The facial expression recognition method based on image block perception based on representation learning according to claim 1, characterized in that The visual encoder E is composed of L encoding layers, where each encoding layer is composed of a multi-head self-attention sublayer and a feedforward sublayer connected in a residual manner; The visual encoder E uses formula (1) to encode the first mask image Process it and get Hidden features of the encoding layer , and thus use formula (2) to get the The encoded features of the layer encoding layer output , and finally the Lth encoding layer outputs the encoding features , and as the visual encoder E for the first mask image The encoding features : (1) (2) In formula (1), represents the multi-head self-attention sub-layer, Representation layer regularization function, Indicates the The encoding features output by the layer encoding layer, when l=1, let = or ; In formula (2), Represents the feedforward sublayer, and contains two fully connected layers and a ReLU activation function, and has: (3) In formula (3), , Represents the two weight parameters to be learned, , Represents the two bias parameters to be learned; The visual encoder E follows the same process to encode the second mask image Processing is performed to finally obtain the second mask image The encoding features .

3. The facial expression recognition method based on image block perception of representation learning according to claim 2, characterized in that The visual decoder consists of a spatial transformation layer, a mask permutation layer, M decoding layers, and an image reconstruction layer, where each decoding layer consists of a multi-head self-attention sublayer and a feedforward sublayer connected in a residual manner; Step 3.4.1, the spatial transformation layer is a linear layer used to transform the encoded features Mapping to a low-dimensional space to obtain the reduced-dimensional features ; Step 3.4.2, the mask permutation layer transforms the reduced dimension features In the image, the part corresponding to the image block where the selected key point is located is replaced with the mask symbol, thereby obtaining the feature after mask replacement ; Step 3.4.3: Features after mask replacement After processing through the M layers of decoding layers, the decoding features are obtained ; Step 3.4.4, the image reconstruction layer is a linear layer for transforming the decoded features Mapped to the image space, the reconstructed image is obtained .

4. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the expression recognition method according to any one of claims 1 to 3, and the processor is configured to execute the program stored in the memory.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the expression recognition method according to any one of claims 1 to 3 are executed.

Images