An expression recognition method and system based on image block position-aware Transformer

Abstract

The present invention discloses an expression recognition method and system based on image block position-aware transformer, the method comprising the following steps: constructing an expression recognition model based on image block position-aware transformer, the model consisting of an image preprocessing unit, an initial feature extraction unit, a feature block position selection unit, a visual transformer unit and a classifier; training the expression recognition model using samples in a facial expression image library; inputting the face image to be tested into the trained expression recognition model for expression recognition. The present invention selects the position of feature blocks at an iterative and progressive non-fixed interval, excludes feature blocks that interfere with expression recognition, and strengthens feature blocks that play a key role in expression recognition, so that the expression recognition model can extract more discriminative expression features, enhance the model's robustness to facial occlusion, head posture changes, and uneven illumination, and effectively improve the accuracy and generalization performance of expression recognition.

Description

Technical Field

[0001] The invention relates to an expression recognition method and system based on an image block position perception Transformer, belonging to the technical field of expression recognition and artificial intelligence. Background Art

[0002] Facial expressions are an important way for humans to communicate emotions in daily life. By analyzing people's facial expressions, we can infer people's inner emotional state. In recent years, with the development of artificial intelligence technology, people have also put forward higher requirements for the "intelligence" of machines. Giving machines emotional cognition can make machines serve people more intelligently and friendly, such as achieving more humane interaction in human-machine interaction, assisting in judging the patient's condition in medical diagnosis, and improving the player experience in entertainment games. Automatic facial expression recognition, as a research branch of emotional computing, is the basis for realizing human-computer interaction and has become a research hotspot in the field of computer vision. Facial expression recognition technology has broad commercial application prospects in human-computer interaction fields such as social robots, education, medical diagnosis, and fatigue driving monitoring.

[0003] Existing facial expression recognition models have achieved good performance in controlled laboratory environments. However, in uncontrolled real-world environments, due to interference factors such as facial occlusion, head posture changes, and individual differences, the performance of existing expression recognition models has dropped sharply, posing a huge challenge to facial expression recognition. How to improve the accuracy, robustness, and generalization performance of facial expression recognition models is a key issue that needs to be addressed.

[0004] In daily life, people can judge other people's expressions or emotions by simply observing part of their faces. Focusing on local features of the face helps solve problems such as face occlusion and posture changes in natural scenes. Many researchers have introduced attention mechanisms to make the model pay more attention to areas in the face image that are closely related to the expression and suppress facial areas that are not related to the expression.

[0005] Convolutional neural networks (CNNs) have inductive bias capabilities such as translation invariance and local sensitivity, which can capture fine-grained image features and local information well, and show good performance in image classification and expression recognition. However, CNN has a limited receptive field and only focuses on local features within a small range. It does not have the ability to obtain global features, and the network weights are fixed and cannot dynamically adapt to changes in input. In 2017, the Google research team proposed the Transformer architecture based on the self-attention mechanism, which subsequently showed great advantages in tasks such as sequence modeling and machine translation in the field of natural language processing (NLP). Transformer can learn rich features from long sequence data with the self-attention mechanism and capture long-distance feature dependencies between data, solving the problem of limited receptive field of CNN. Inspired by the Transformer in the NLP field, some researchers directly applied the Transformer to the visual field and proposed the Visual Transformer (ViT) model. The ViT model first divides the input image into non-overlapping patches of fixed position and size, rearranges and flattens them into one-dimensional vectors, then linearly maps them and adds position encoding, and inputs them into multiple cascaded standard Transformer encoders for attention calculation and feature extraction. In this way, when performing self-attention calculations, the model is able to distinguish the information of patches at different positions and better understand the contextual relationship in the sequence. In addition, in order to achieve the classification task, an additional learnable category token is added to the embedding sequence of patches. The ViT model can well learn the feature dependencies between patches at different positions in the image, but ViT simply divides the input image into non-overlapping patches of fixed position and size, which destroys the inherent structural information in the image, especially the edge information of the patches, and does not consider the impact of interference factors such as face occlusion and head posture changes on the expression recognition model. Summary of the invention

[0006] The purpose of the present invention is to address the defects and shortcomings of the above-mentioned prior art and propose an expression recognition method based on image block position-aware Transformer, which selects the center position of the feature block at iterative and progressive non-fixed intervals, and extracts more discriminative expression features of unobstructed important areas in the face image through the position-aware visual Transformer, so as to enhance the robustness of the model to facial occlusion, head posture changes, and uneven lighting, and effectively improve the accuracy and generalization performance of expression recognition.

[0007] The technical solution adopted by the present invention to solve the technical problem is: an expression recognition method based on image block position perception Transformer, the method comprising the following steps:

[0008] Step 1: Build an expression recognition model based on image patch position-aware Transformer;

[0009] Step 2: Use samples in the facial expression image library to train the constructed expression recognition model, and adjust the parameters of the expression recognition model to the optimal value through the error back propagation algorithm;

[0010] Step 3: Input the face image to be tested into the trained expression recognition model for expression recognition.

[0011] Furthermore, the step 1 includes the feature block position selection unit selecting the center position coordinates of the feature block at an iterative and progressive non-fixed interval, and the specific steps are as follows:

[0012] Step 1-1: For the first iteration, as with the conventional visual Transformer model, the feature map F is divided into non-overlapping n×n feature blocks, where the center position coordinates of the i-th feature block are for:

[0013]

[0014] Where i represents the serial index of the feature block, i = 0, 1, 2, ..., n × n-1, i x and i y Respectively represent the row index and column index of the i-th feature block, Indicates the rounding down operation, i x =ii y *n,s w =W / n,s h =H / n, W and H represent the width and height of the feature map respectively; the center position coordinate sequence of all n×n feature blocks is

[0015] Step 1-2: For the first iteration, let the token corresponding to the i-th feature block be The token sequence corresponding to all n×n feature blocks is The center position coordinate sequence of n×n feature blocks Mapped to an embedding space, we get the position encoding matrix Add P1 and token sequence T′1 element by element and input them into Transformer encoder to get the token sequence output after the first iteration. Its expression is:

[0016] P1=W1p1

[0017]

[0018] T1=Transformer(X1)

[0019] in, It is the linear transformation matrix that maps the center position coordinate sequence p1 of the feature block to the position encoding matrix P1. represents element-by-element addition, Transformer(·) represents the Transformer encoder based on multi-head self-attention;

[0020] Step 1-3: For the first iteration, linearly map T1 through the fully connected layer to obtain the predicted coordinate offset sequence Its expression is:

[0021] o1=M1T1

[0022] in, It is a linear transformation matrix that can be trained and learned;

[0023] Step 1-4: For the tth iteration, t = 2, 3, ..., N, the center position coordinate sequence p of the n×n feature blocks t The center position coordinate sequence p at the t-1th iteration t-1 and the predicted coordinate offset sequence o at the t-1th iteration t-1 Add them together and the expression is:

[0024] p t =p t-1 +o t-1

[0025] Step 1-5: For the t-th iteration, t = 2, 3, ..., N, let the token corresponding to the i-th feature block be The token sequence corresponding to all n×n feature blocks is The center position coordinate sequence of n×n feature blocks Mapped to an embedding space, we get the position encoding matrix P t With token sequence T′ t and T t-1 After adding each element, input it into the Transformer encoder to get the token sequence output after the tth iteration Its expression is:

[0026] P t =W t p t

[0027]

[0028] T t = Transformer(X t )

[0029] in, The center position coordinate sequence p of the feature block is t Mapped to the position encoding matrix P t The linear transformation matrix of represents element-by-element addition, Transformer(·) represents the Transformer encoder based on multi-head self-attention;

[0030] Step 1-6: For the tth iteration, t = 2, 3, ..., N, set T t The predicted coordinate offset sequence is obtained by linear mapping through the fully connected layer Its expression is:

[0031] o t =M t T t

[0032] in, It is a linear transformation matrix that can be trained and learned.

[0033] The present invention also provides an expression recognition system based on image block position-aware Transformer, the system comprising:

[0034] A model building module, used to build an expression recognition model, wherein the model includes an image preprocessing unit, an initial feature extraction unit, a feature block position selection unit, a visual Transformer unit and a classifier;

[0035] The image preprocessing unit is used to perform face detection, cropping, alignment and resolution normalization processing on the input image;

[0036] The initial feature extraction unit is composed of a convolutional neural network and is used to extract features from the preprocessed image;

[0037] The feature block position selection unit is used to select the center position coordinates of the feature block at an iterative and progressive non-fixed interval;

[0038] The visual Transformer unit includes L Transformer encoders for feature extraction, and outputs the trained category token vector as the input of the classifier;

[0039] The classifier is used to classify the category token vector and output the facial expression category;

[0040] The model training module is used to train the constructed expression recognition model using samples in the facial expression image library, and adjust the parameters of the expression recognition model to the optimal value through the error back propagation algorithm;

[0041] The expression recognition module is used to use the trained expression recognition model to perform expression classification and recognition on the newly input face image to be tested.

[0042] Furthermore, the feature map output by the initial feature extraction unit is Where W, H, and C represent the width, height, and channel dimension of the feature map, respectively.

[0043] Furthermore, the number of iterations of the feature block position selection unit is N, and the value of N is an integer between 4 and 10; in each iteration, the feature map F is divided into n×n feature blocks, and the value of n is selected from 2, 4, and 8.

[0044] Furthermore, the value of L in the visual Transformer unit is an integer between 4 and 6. Suppose that after N iterations, the token sequence corresponding to the n×n feature blocks is Among them, C is the feature dimension, in the token sequence T N Concatenate a trainable category token vector Get the concatenated token sequence Input T into the visual Transformer unit for feature extraction, and output the token sequence after passing through L Transformer encoders from Extract the trained category token vector as input to the classifier.

[0045] Furthermore, the system includes at least one computing device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, the expression recognition method based on the image block position-aware Transformer is implemented.

[0046] Beneficial effects:

[0047] 1. In the existing expression recognition methods, the input face image is usually directly divided uniformly, and then local features are extracted from each patch, and expression recognition is performed through a classifier after feature fusion. The visual Transformer (ViT) model also first divides the input image into non-overlapping patches of fixed position and size, and rearranges and flattens them into one-dimensional vectors, and then linearly maps them and adds position coding, and inputs them into multiple cascaded standard Transformer encoders for attention calculation and feature extraction. The uniform division of the input image will destroy the inherent structural information inside the image, resulting in the loss of relevant information between patches, especially the edge information of the patches. The present invention takes into account the relevant information between patches, and does not divide the input face image uniformly, but uses an overlapping block strategy to divide the feature map output by the initial feature extraction unit to obtain overlapping feature blocks as the input of the Transformer encoder. In the feature map, each pixel is obtained by nonlinear transformation of the local area of ​​the input face image. Spatial block division of the feature map is conducive to retaining the integrity of local information.

[0048] 2. The present invention discloses an expression recognition method based on an image block position-aware Transformer. The method combines a feature block position selection unit with a visual Transformer (ViT) and applies them to expression recognition, overcoming the defect that the ViT model simply divides the input image into non-overlapping patches of fixed position and size, thereby destroying the inherent structural information inside the image. The expression recognition model pays more attention to the areas with significant expression features in the face image, thereby improving the accuracy of expression recognition.

[0049] 3. The feature block position selection unit proposed in the present invention selects the position of the feature block at an iterative and progressive non-fixed interval. In each iteration, the center position coordinate sequence of the current feature block is obtained by adding the center position coordinate sequence of the previous iteration and the predicted coordinate offset sequence. For face images with face occlusion, head posture changes, and uneven illumination, in the process of continuous iteration, the center position of the feature block corresponding to the occluded face area is gradually updated to the center position of the feature block with significant unobstructed features close to the corresponding face image, so as to exclude the feature blocks of the corresponding occluded face area that have an adverse effect on expression recognition, and strengthen the unobstructed feature blocks in the corresponding face image that play a key role in expression recognition, so that the expression recognition model can extract more discriminative expression features, enhance the model's robustness to face occlusion, head posture changes, and uneven illumination, and effectively improve the accuracy and generalization performance of expression recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The figure is a flow chart of a method according to an embodiment of the present invention.

[0051] Figure 2 Schematic diagram of the structure of the expression recognition model constructed in an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and preferred embodiments of the specification.

[0053] like Figure 1 As shown, an expression recognition method based on image block position-aware Transformer disclosed in an embodiment of the present invention mainly includes the following steps:

[0054] Step (1): Construct an expression recognition model based on image block position-aware Transformer. The model consists of an image preprocessing unit, an initial feature extraction unit, a feature block position selection unit, a visual Transformer unit, and a classifier. Figure 2 shown.

[0055] Figure 2 The image preprocessing unit in is used to perform face detection, cropping, alignment and resolution normalization on the input image. In this embodiment, Python is used to import the dilb toolkit, read the image, and dilb is used to perform face detection on each read image; the detected face area is cropped out and the complex background is removed; the cropped face image is marked with the dilb dotting tool for face key points, the angle between the line connecting the two eyes and the horizontal line is calculated, and then the face image is rotated to the horizontal according to the angle for alignment; finally, the resolution of the aligned face image is normalized to 224×224.

[0056] Figure 2 The initial feature extraction unit in is composed of the ResNet-34 backbone network, which is used to extract features from the preprocessed image. The output feature map is Wherein W, H and C represent the width, height and channel dimension of the feature map respectively; in this embodiment, W=32, H=32, C=128.

[0057] Figure 2 The feature block position selection unit in the invention selects the center position coordinates of the feature block at an iterative and progressive non-fixed interval; the number of iterations is N, and the value of N in the invention is an integer between 4 and 10; in each iteration, the feature map F is divided into n×n feature blocks, and the value of n is selected from 2, 4, and 8; in the present embodiment, N=6, n=4.

[0058] The visual Transformer unit includes L Transformer encoders. In the present invention, the value of L is an integer between 4 and 6. Suppose that after N iterations, the token sequence corresponding to the n×n feature blocks is Among them, C is the feature dimension, in the token sequence T N Concatenate a trainable category token vector Get the concatenated token sequence Input T into the visual Transformer unit for feature extraction, and output the token sequence after passing through L Transformer encoders from Extract the trained category token vector As the input of the classifier; in this embodiment, L=4, N=6, n=4, C=128.

[0059] Classifier for category token vector T′ cls Classify and output facial expression categories.

[0060] As a further optimization scheme of the expression recognition method based on the image block position-aware Transformer of the present invention, the specific steps of the feature block position selection unit in step 1 selecting the center position coordinates of the feature block at an iterative and progressive non-fixed interval are as follows:

[0061] Step 1-1: For the first iteration, as with the conventional visual Transformer model, the feature map F is divided into non-overlapping n×n feature blocks, where the center position coordinates of the i-th feature block are for:

[0062]

[0063] Where i represents the serial index of the feature block, i = 0, 1, 2, ..., n × n-1, i x and i y Respectively represent the row index and column index of the i-th feature block, Indicates the rounding down operation, i x =ii y *n,s w =W / n,s h =H / n, W and H represent the width and height of the feature map respectively; the center position coordinate sequence of all n×n feature blocks is In this embodiment, n=4, W=32, and H=32.

[0064] Step 1-2: For the first iteration, let the token corresponding to the i-th feature block be The token sequence corresponding to all n×n feature blocks is The center position coordinate sequence of n×n feature blocks Mapped to an embedding space, we get the position encoding matrix Add P1 and token sequence T′1 element by element and input them into Transformer encoder to get the token sequence output after the first iteration. Its expression is:

[0065] P1=W1p1

[0066]

[0067] T1=Transformer(X1)

[0068] in, It is the linear transformation matrix that maps the center position coordinate sequence p1 of the feature block to the position encoding matrix P1. represents element-by-element addition, Transformer(·) represents a Transformer encoder based on multi-head self-attention; in this embodiment, n=4, C=128.

[0069] Step 1-3: For the first iteration, linearly map T1 through the fully connected layer to obtain the predicted coordinate offset sequence Its expression is:

[0070] o1=M1T1

[0071] in, is a linear transformation matrix that can be trained and learned; in this embodiment, n=4, C=128.

[0072] Step 1-4: For the tth iteration, t = 2, 3, ..., N, the center position coordinate sequence p of the n×n feature blocks t The center position coordinate sequence p at the t-1th iteration t-1 and the predicted coordinate offset sequence o at the t-1th iteration t-1 Add them together and the expression is:

[0073] p t =p t-1 +o t-1

[0074] In this embodiment, n=4, N=6.

[0075] Step 1-5: For the t-th iteration, t = 2, 3, ..., N, let the token corresponding to the i-th feature block be The token sequence corresponding to all n×n feature blocks is The center position coordinate sequence of n×n feature blocks Mapped to an embedding space, we get the position encoding matrix P t With token sequence T′ t and T t-1 After adding each element, input it into the Transformer encoder to get the token sequence output after the tth iteration Its expression is:

[0076] P t =W t p t

[0077]

[0078] T t = Transformer(X t )

[0079] in, The center position coordinate sequence p of the feature block is t Mapped to the position encoding matrix p t The linear transformation matrix of represents element-by-element addition, Transformer(·) represents a Transformer encoder based on multi-head self-attention; in this embodiment, n=4, N=6, C=128.

[0080] Step 1-6: For the tth iteration, t = 2, 3, ..., N, set T t The predicted coordinate offset sequence is obtained by linear mapping through the fully connected layer Its expression is:

[0081] o t =M t T t

[0082] in, is a linear transformation matrix that can be trained and learned; in this embodiment, n=4, N=6.

[0083] Step 2: Use samples in the facial expression image library to train the constructed expression recognition model, and adjust the parameters of the expression recognition model to the optimal value through the error back propagation algorithm.

[0084] In this embodiment, facial images in the facial expression image set SFEW2.0 in an uncontrolled environment are used as training samples. The facial images on the SFEW (Static Facial Expressions in the Wild) image set are selected from single-frame images on the AFEW (Acted Facial Expressions in the Wild) video set. The AFEW video set has been used as the official data set for the EmotiW (Emotion Recognition In The Wild) competition since 2013, and its samples are collected from the scenes of characters with spontaneous expressions in different movie materials. SFEW is created by selecting static frame images from the AFEW video set based on the calculation of key frames based on facial point clustering. Compared with the traditional facial expression image set collected under laboratory conditions, the SFEW image set images are closer to actual natural scenes, which is equivalent to facial images taken in an unrestricted natural environment. Therefore, different head postures, different age groups, various real-life occlusions, resolutions, and lighting changes are fully reflected in this data set. The SFEW2.0 image set contains 1766 720×576 pixel facial expression images, which have been divided into three subsets: training set (958 samples), validation set (436 samples) and test set (372 samples). Each facial image is labeled with one of the seven expressions: anger, disgust, fear, neutral, happy, sad and surprised. Since the test set is used for competition, its labels are not public. Therefore, this embodiment uses the training set and 1394 samples of the validation set of the SFEW2.0 image set as training samples to train the expression recognition model.

[0085] Step 3: Input the face image to be tested into the trained expression recognition model for expression recognition.

[0086] Based on the same inventive concept, an embodiment of the present invention provides an expression recognition system based on an image block position-aware Transformer, comprising: a model construction module, used to construct an expression recognition model, the model being composed of an image preprocessing unit, an initial feature extraction unit, a feature block position selection unit, a visual Transformer unit and a classifier; the structure and function of each unit of the model refer to the above-mentioned method embodiment; a model training module, used to train the constructed expression recognition model using samples in a facial expression image library, and adjusting the parameters of the expression recognition model to the optimum through an error back propagation algorithm; and an expression recognition module, used to use the trained expression recognition model to perform expression classification and recognition on a newly input facial image to be tested.

[0087] Based on the same inventive concept, an embodiment of the present invention provides an expression recognition system based on an image block position-aware Transformer, comprising at least one computing device, the computing device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when loaded into the processor, implements the above-mentioned expression recognition method based on an image block position-aware Transformer.

[0088] The above scheme is only a description of the preferred implementation method of the present application, but the protection scope of the present application is not limited to this. Any person familiar with the technology can easily implement it within the scope of the description of the present application without changing the changes or substitutions involved in the basic principles of the claims, which should be covered by the protection scope of the present application, that is, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An expression recognition method based on image block position-aware Transformer, characterized in that: The steps include: Step 1: Build an expression recognition model based on image block position-aware Transformer, including a feature block position selection unit that selects the center position coordinates of feature blocks at iterative and progressive non-fixed intervals. The specific steps are as follows: Step 1-1: For the first iteration, as with the conventional visual Transformer model, the feature map F is divided into non-overlapping n×n feature blocks, where the center position coordinates of the i-th feature block are for: Where i represents the serial index of the feature block, i = 0, 1, 2, ..., n × n-1, i x and i y Respectively represent the row index and column index of the i-th feature block, Indicates the rounding down operation, i x =ii y *n,s w =W / n,s h =H / n, W and H represent the width and height of the feature map respectively; the center position coordinate sequence of all n×n feature blocks is Step 1-2: For the first iteration, let the token corresponding to the i-th feature block be The token sequence corresponding to all n×n feature blocks is The center position coordinate sequence of n×n feature blocks Mapped to an embedding space, we get the position encoding matrix Add P1 and token sequence T′1 element by element and input them into Transformer encoder to get the token sequence output after the first iteration. Its expression is: P1=W1P1 T1=Transformer(x1) in, It is the linear transformation matrix that maps the center position coordinate sequence p1 of the feature block to the position encoding matrix P1. represents element-by-element addition, Transformer(·) represents the Transformer encoder based on multi-head self-attention; Step 1-3: For the first iteration, linearly map T1 through the fully connected layer to obtain the predicted coordinate offset sequence Its expression is: o1=M1T1 in, It is a linear transformation matrix that can be trained and learned; Step 1-4: For the tth iteration, t = 2, 3, ..., N, the center position coordinate sequence p of the n×n feature blocks t The center position coordinate sequence p at the t-1th iteration t-1 and the predicted coordinate offset sequence o at the t-1th iteration t-1 Add them together and the expression is: p t =p t-1 +o t-1 Step 1-5: For the t-th iteration, t = 2, 3, ..., N, let the token corresponding to the i-th feature block be The token sequence corresponding to all n×n feature blocks is The center position coordinate sequence of n×n feature blocks Mapped to an embedding space, we get the position encoding matrix P t With token sequence T′ t and T t-1 After adding each element, input it into the Transformer encoder to get the token sequence output after the tth iteration Its expression is: P t =W t p t T t =Transformer(x t ) in, The center position coordinate sequence p of the feature block is t Mapped to the position encoding matrix P t The linear transformation matrix of represents element-by-element addition, Transformer(·) represents the Transformer encoder based on multi-head self-attention; Step 1-6: For the tth iteration, t = 2, 3, ..., N, set T t The predicted coordinate offset sequence is obtained by linear mapping through the fully connected layer Its expression is: o t =M t T t in, It is a linear transformation matrix that can be trained and learned; Step 2: Use samples in the facial expression image library to train the constructed expression recognition model, and adjust the parameters of the expression recognition model to the optimal value through the error back propagation algorithm; Step 3: Input the face image to be tested into the trained expression recognition model for expression recognition.

2. An expression recognition system based on image block position-aware Transformer, characterized in that: include: A model building module, used to build an expression recognition model, wherein the model includes an image preprocessing unit, an initial feature extraction unit, a feature block position selection unit, a visual Transformer unit and a classifier; The image preprocessing unit is used to perform face detection, cropping, alignment and resolution normalization processing on the input image; The initial feature extraction unit is composed of a convolutional neural network and is used to extract features from the preprocessed image; The feature block position selection unit is used to select the center position coordinates of the feature block at an iterative and progressive non-fixed interval; The visual Transformer unit includes L Transformer encoders for feature extraction, and outputs the trained category token vector as the input of the classifier; The classifier is used to classify the category token vector and output the facial expression category; The model training module is used to train the constructed expression recognition model using samples in the facial expression image library, and adjust the parameters of the expression recognition model to the optimal value through the error back propagation algorithm; The expression recognition module is used to use the trained expression recognition model to perform expression classification and recognition on the newly input face image to be tested.

3. The expression recognition system based on image block position-aware Transformer according to claim 2, characterized in that: The feature map output by the initial feature extraction unit is Where W, H, and C represent the width, height, and channel dimension of the feature map, respectively.

4. The expression recognition system based on image block position-aware Transformer according to claim 2, characterized in that: The number of iterations of the feature block position selection unit is N, and the value of N is an integer between 4 and 10; in each iteration, the feature map F is divided into n×n feature blocks, and the value of n is selected from 2, 4, and 8.

5. The expression recognition system based on image block position-aware Transformer according to claim 2, characterized in that: The value of L in the visual Transformer unit is an integer between 4 and 6. Suppose that after N iterations, the token sequence corresponding to the n×n feature blocks is Among them, C is the feature dimension, in the token sequence T N Concatenate a trainable category token vector Get the concatenated token sequence T = [T clc , T N ], Input T into the visual Transformer unit for feature extraction, and output the token sequence after passing through L Transformer encoders from Extract the trained category token vector as input to the classifier.

6. The expression recognition system based on image block position-aware Transformer according to claim 2, characterized in that: The system includes at least one computing device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, an expression recognition method based on an image block position-aware Transformer according to any one of claims 1 to 2 is implemented.

Images