Micro-expression image pair generation method based on linear attention mechanism and boundary balance mechanism

Abstract

The invention discloses a method for generating a micro-expression image pair based on a linear attention mechanism and a boundary balance mechanism, and the method comprises the following steps: (1), obtaining a video clip, obtaining a start frame image and a peak frame image when a micro-expression occurs through a FlowNet optical flow method, carrying out the preprocessing, and outputting a real micro-expression image pair; (2) inputting high-dimensional random noise and category label conditions, and generating and outputting a synthesized micro-expression image pair by using a branch network and a linear attention mechanism; (3) judging whether the image is true or false, extracting global and local features through a global block division convolutional neural network, calculating the difference between the synthesized micro-expression image pair and the real micro-expression image pair, and outputting a judgment result; (4) updating corresponding network weights and parameters according to a judgment result; constructing and calculating a boundary balance loss function based on a boundary balance mechanism, carrying out iterative training until the loss function converges, and outputting a synthesized image pair; according to the invention, the micro-expression image pair with good enough quality can be stably generated.

Description

Technical Field

[0001] The present invention relates to the technical field of micro-expression recognition, and in particular to a method for generating micro-expression image pairs based on a linear attention mechanism and a boundary balance mechanism. Background Art

[0002] Micro-expressions are short-lived and unconscious facial expressions that can reflect people's true inner emotional state. Since micro-expressions usually last very short (usually less than 0.5 seconds) and have a small amplitude, they are difficult to detect with the naked eye. Therefore, they are of great significance in psychology, behavior and related application fields (such as lie detection, sentiment analysis, etc.). However, there are still several challenges in the existing micro-expression recognition task: it is difficult to locate, and the amplitude of micro-expressions is very subtle, which makes it difficult for traditional image processing algorithms to effectively extract effective features of micro-expressions; the data set sample size is scarce and difficult to annotate, and accurate annotation of micro-expression samples requires professional knowledge, which leads to high data set construction costs; the technology for automatically generating micro-expression image sequences is immature. On the one hand, the use of traditional generative adversarial neural networks to generate micro-expression images not only has poor image quality, but also the generator and discriminator are very likely to be strong and weak during the training process, resulting in the inability to effectively train the model. On the other hand, although the related work of adding self-attention to the generative adversarial neural network has improved the quality of the generated images, it still cannot meet the data sample standards required by the micro-expression recognition task, and the amount of calculation required has increased exponentially. Summary of the invention

[0003] Purpose of the invention: The purpose of the present invention is to provide a method for generating micro-expression image pairs based on a linear attention mechanism and a boundary balance mechanism to solve the problems existing in the background technology.

[0004] Technical solution: The method for generating micro-expression image pairs based on a linear attention mechanism and a boundary balance mechanism described in the present invention comprises the following steps:

[0005] (1) Obtain video clips, use the FlowNet optical flow method to obtain the starting frame and peak frame images when micro-expressions occur, perform preprocessing, and output them as true micro-expression image pairs;

[0006] (2) Input high-dimensional random noise and category label conditions, use the branch network and linear attention mechanism to generate and output synthetic micro-expression image pairs;

[0007] (3) Determine true or false: extract global and local features through global block partitioning convolutional neural network, calculate the difference between the synthesized micro-expression image pair and the real micro-expression image pair, and output the judgment result;

[0008] (4) Based on the judgment results, the corresponding network weights and parameters are updated; based on the boundary balance mechanism, a boundary balance loss function is constructed and calculated, and training is iterated until the loss function converges, and the synthesized image pair is output.

[0009] Furthermore, step (1) includes the following steps:

[0010] (11) Extract continuous static image frames at a fixed frequency, use the directional histogram to perform face detection on each frame, and then crop the area where the face is located; calculate the displacement change of facial pixels through the FlowNet optical flow algorithm, so as to locate and extract the starting frame and peak frame image when the micro-expression occurs;

[0011] (12) The extracted micro-expression start frame and peak frame images are preprocessed including resizing, facial alignment, and normalization, and output as the true micro-expression image pair.

[0012] Furthermore, step (2) includes the following steps:

[0013] (21) inputting high-dimensional random noise and category label conditions, converting the labels into high-dimensional feature representations through one-hot encoding; then concatenating the high-dimensional noise vector and the label condition vector to form a new feature vector;

[0014] (22) The new feature vector is linearly transformed through the fully connected layer to output a low-resolution feature map; the low-dimensional feature map is converted into a high-dimensional feature map through several layers of transposed convolutional networks until it reaches the branch network;

[0015] (23) The branch structure includes two independent branch networks of the starting frame and the peak frame, each branch performs feature extraction and is used to generate a synthetic starting frame image and a peak frame image respectively;

[0016] (24) In the two branch networks, a linear attention mechanism is introduced at the same time to assign different weights;

[0017] (25) The results of the two branch networks are input again into several layers of transposed convolutional networks, and finally an image pair of the target size is output.

[0018] Furthermore, step (3) includes the following steps:

[0019] (31) Extract global and local features by partitioning the convolutional neural network into global blocks;

[0020] (32) Input the global feature map and the block feature map into the linear attention module to generate an attention weight matrix; perform weighted summation on the weighted global features and local features to obtain a fused feature map;

[0021] (33) The feature maps of the synthetic image pair and the feature maps of the real image pair are judged using a classification and discrimination method.

[0022] Furthermore, step (31) is specifically as follows: input a synthetic image pair and a real image pair and corresponding category labels, and extract features through a global block partitioning convolutional neural network; after the input set of image pairs is subjected to optical flow calculation, its horizontal component, vertical component and optical flow strain map are output; each image is evenly divided into a number of blocks of the same size, and features are extracted for each block, and all block features are spliced ​​to obtain global information; at the same time, the key area is divided into blocks again, and separate convolution and pooling operations are performed on each block of the secondary division; all the secondary divided blocks are first individually feature spliced, and then all the block features are spliced ​​according to the channel dimension to obtain local information; finally, the global feature map and local feature map extracted by the horizontal component, vertical component and optical flow strain are spliced ​​respectively, and then the category label information is converted into high-dimensional features and combined with the global and local feature maps through channel splicing.

[0023] Furthermore, the key areas in step (31) include: eyebrow area, eye corner area, mouth corner area

[0024] Furthermore, step (33) is as follows: the feature map of the synthetic image pair and the real image are mapped to a high-dimensional space to generate a discriminant vector, and then the classification result is directly output through the classification layer. The output value is mapped to [0,1] using a Sigmoid activation function to generate a classification probability, which represents the probability that the generated image pair and the real image pair are judged as real.

[0025] Furthermore, step (4) is specifically as follows: the difference in classification results is calculated and measured through the cross entropy loss function, the weights and parameters of the generation module and the discrimination module are updated according to the cross entropy loss result, and the training process of the generation and discrimination image pairs is dynamically adjusted by the boundary balance mechanism, and the boundary loss function is constructed. The balance factor γ is automatically updated to ensure the stable operation of the model. When γ tends to be stable, the synthetic image is output; wherein, the loss of the real image pair is:

[0026] L real = -log(D real )

[0027] The loss for the synthetic image pair is:

[0028] L syn = -log(1-D syn )

[0029] Among them, D real is the output probability of the discriminator for the real image pair, D synis the output probability of the discriminator for the generated image. The boundary balance mechanism is introduced to correlate with each other through the balance factor. The loss of the generation module is L G :

[0030] L G =||D(G(X))-γG(X)||1

[0031] Among them, X is the real image, G(X) is the generated image;

[0032] The discriminant module includes: reconstruction loss of real image pairs and reconstruction loss of generated image pairs as well as denoising loss:

[0033] L D =ω noise ·L dn +||D(x)-x||1-γ·||D(G(X))-G(X)||1

[0034] Where x is the real image, ω noise It is the weighted coefficient of denoising loss, which is used to control the proportion of denoising loss; the denoising loss function is as follows:

[0035] L dn =||D(x+noise)-x||2 2 ;

[0036] γ is the dynamic coefficient of weight used to balance training, and is adaptively adjusted to:

[0037] γ t+1 =γ t +τ·(σ·L real -L syn )

[0038] Among them, τ is the learning rate, γ is the dynamically adjusted balance factor, and σ is a fixed ratio.

[0039] An electronic device described in the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, it implements any one of the methods for generating micro-expression image pairs based on a linear attention mechanism and a boundary balance mechanism.

[0040] A storage medium described in the present invention stores a computer program, and when the computer program is executed by a processor, it implements any one of the methods for generating micro-expression image pairs based on a linear attention mechanism and a boundary balance mechanism.

[0041] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: First, the generated image and the real image are both the starting frame and the peak frame image when the micro-expression occurs. There is a wealth of micro-expression feature information between these two frames (necessary), which can fully represent the image sequence of micro-expression fragments. Secondly, by introducing the linear attention mechanism in the branch network and the global block convolutional neural network, not only the required computing resources are very small, but also the effective features can be better extracted. Finally, the introduction of the boundary balance mechanism can dynamically and effectively prevent the model from overfitting and imbalance, thereby efficiently and stably generating micro-expression starting frames and peak frame images. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic diagram of the process of the present invention;

[0043] Figure 2 It is a schematic diagram of the feature extraction network of the present invention. DETAILED DESCRIPTION

[0044] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0045] like Figure 1 As shown, an embodiment of the present invention provides a method for generating micro-expression image pairs based on a linear attention mechanism and a boundary balance mechanism, comprising the following steps:

[0046] S1, obtaining the starting frame and peak frame images when the micro-expression occurs, including the following steps:

[0047] S11, the micro-expression recognition task mainly relies on two images: the starting frame and the peak frame of the micro-expression. The starting frame indicates the moment when the micro-expression begins to change, and the peak frame is the moment when the micro-expression changes most obviously. These two key frames contain almost all the effective information of the micro-expression, so their accurate positioning and acquisition are crucial for micro-expression recognition. Input a video clip containing micro-expression changes, and extract continuous static image frames at a fixed frequency of 60fps. Use the HOG features and SVM in the Dlib library to perform face detection on each frame image and crop the face area R(x,y) so that subsequent analysis is based only on the face. Then use the FlowNet optical flow algorithm to calculate the displacement of facial pixels and generate a two-channel optical flow field. By analyzing the optical flow field F(U,V), locate the starting frame and peak frame images when the micro-expression occurs, and save them as the basic images for subsequent input. The calculation formula for the change in the optical flow field is as follows:

[0048]

[0049] Among them, U(X,Y) and V(X,Y) represent the horizontal and vertical components of the optical flow field, respectively, indicating the horizontal and vertical motion of the pixel.

[0050] S12, first resize the extracted start frame and peak frame images to 640x480 pixels. Then, facial alignment is achieved through translation and scaling operations based on facial key points. Finally, the two frame images are normalized and output as a pair of pre-processed images, which are determined to be the true micro-expression start frame and peak frame images.

[0051] S2, generating image pairs, includes the following steps:

[0052] S21, input high-dimensional random noise Z and category label condition Y, convert Y into a high-dimensional feature representation through one-hot encoding, where the dimension of Z is 256 and the dimension of Y is 7, then concatenate the high-dimensional noise vector and the label condition vector to form a new feature vector X, whose dimension is 263. The feature vector X is transformed into a low-dimensional vector T through a fully connected layer, and T is reshaped and output as a low-dimensional feature map with an output resolution of 8×8. Then, the low-dimensional feature map is gradually converted into a high-dimensional feature map with a size of 128×128 through a 4-layer transposed convolutional network.

[0053] There are 7 micro-expression category labels, namely anger, surprise, sadness, happiness, fear, contempt and disgust. Let the noise vector Z = [Z1, Z2, Z3, ... Z n ], the category label Y = [0,1,0,0] (indicating the second type of surprise), concatenate the two vectors to get a new vector X = [Z1,Z2,Z3,...Z n, 0,1,0,0], the feature vector X is used to guide the generation of images corresponding to the label category.

[0054] S22, input the high-dimensional feature map into the branch network. Among them, the starting frame branch network focuses on the initial state when the micro-expression just starts, uses a larger convolution kernel and the linear attention mechanism will focus on global attention to capture the global change characteristics of the face. The peak frame branch network focuses on the local state when the micro-expression change reaches its peak. The linear attention mechanism will focus on local attention, assign higher weights to key areas, and the receptive field is more concentrated on local changes in key areas such as the corners of the mouth.

[0055] The linear attention mechanism is introduced in the branch network, which reduces the computational complexity and improves the computational accuracy and efficiency compared with the traditional self-attention mechanism. In the traditional self-attention mechanism, attention weights are calculated between each query vector and each key vector, and these weights are applied to the value vector. This usually involves calculating the dot product of all query and all key pairs, and the corresponding computational complexity is O(n 2), where n is the sequence length. The self-attention mechanism first calculates the similarity (i.e., attention score) between the query vector and the key vector. This similarity is used to determine the degree of attention on the value vector V. The specific calculation formula is:

[0056]

[0057] Among them, Q represents the degree of matching between the current input and other elements, K represents the features in the input sequence, and d k is the dimension of the key and query vectors, used to scale the dot product. In order to make the attention scores interpretable and normalized, the scores need to be converted into weights through the softmax function. The formula is as follows:

[0058]

[0059] The normalized weight matrix Applied to the value vector V, the weighted sum is used to get the output of attention. The formula is:

[0060]

[0061] The key to the linear attention mechanism is to reduce the complexity to O(n) by optimizing the dot product operation. Specifically, it obtains a new representation by linearly transforming the input query (Q), key (K), and value (V):

[0062]

[0063]

[0064]

[0065] in is a linear activation function, which is obtained by approximating the exponential function of Taylor expansion. In the traditional self-attention mechanism, the dot product attention is based on Q·K T Calculated. In the linear attention mechanism, by introducing the linear mapping function This allows the dot product operation to be decomposed through a linear function, thereby reducing the computational complexity:

[0066]

[0067] In the branch network, the global and local features of the image are extracted respectively through the feature extraction network. These features are used as inputs of queries, keys and values. The attention weights in the feature map are calculated using the optimization method of the linear attention mechanism. Then, the features are weighted summed with the obtained attention weights and input into the transposed convolutional network.

[0068] S23, the output of the branch network is input into the two-layer transposed convolutional network again, and finally an image pair with a target size of 640×480 is generated.

[0069] S3, judging true or false, includes the following steps:

[0070] S31, input the synthetic image pair and the real image pair and their corresponding category labels at the same time, and extract features through the global block partitioning convolutional neural network. After the input set of image pairs is calculated by optical flow, its horizontal component, vertical component and optical flow strain map are output. Each image will be evenly divided into 16 blocks of the same size, feature extraction will be performed on each block, and all block features will be spliced ​​to capture global information. At the same time, the blocks numbered 6, 7, 10, 11, 14, and 15 (corresponding to the eyebrow area, the corner of the eye area, and the corner of the mouth area, respectively) are divided again in equal amounts, and separate convolution and pooling operations are performed on each block of the secondary division. All the blocks after the secondary division are first feature spliced ​​individually, and then all the block features are spliced ​​according to the channel dimension to capture local information. Finally, the global feature map and local feature map extracted from the horizontal component, vertical component and optical flow strain map are spliced ​​separately, and then the category label information is converted into high-dimensional features and combined with the global and local feature maps through channel splicing. The global feature map and the block feature map are input into the linear attention module to generate the attention weight matrix. The weighted global features and local features are weighted summed to obtain the fused feature map;

[0071] S32, introduces a linear attention mechanism in the fusion process, so that the network can more intelligently decide which features are more important during fusion. The specific operation is as follows: Use the global feature map F global Generate an attention weight map A global , using the local feature map F local l Generate another attention weight map A local :

[0072]

[0073]

[0074] Where W g and W l are the weight matrices of the global and local feature maps respectively. The global features and local features are weighted and A global and A local Respectively with F global and F local Element-wise multiplication:

[0075]

[0076]

[0077] The weighted global and local features are concatenated to obtain a fused feature map.

[0078] S33, compare the synthetic image pair feature map with the real image pair feature map F real The classification discriminant method is used for judgment, and the discriminant vector is first mapped to the high-dimensional space to generate the discriminant vector.

[0079] V syn =MLP(F syn ),V real =MLP(F real )

[0080] The classification result is then directly output through a classification layer, and the output value is mapped to [0, 1] using the Sigmoid activation function.

[0081]

[0082] Generate classification probability P syn and P real , respectively represent the probability of the generated image pair and the real image pair being judged as real. If P real The value is close to 1, then the image pair is a true image pair. syn If the value is close to 0, it is a synthetic image pair.

[0083] S4, update the weights and iterate the training until the boundary balance loss function converges, and output the synthesized image pair.

[0084] The difference in classification results is calculated and measured through the cross entropy loss function. The weights and parameters of the generation module and the discrimination module are updated according to the cross entropy loss. At the same time, the boundary balance mechanism is used to dynamically adjust the training process of the generation and discrimination image pairs, construct the boundary loss function, and automatically update the balance factor γ to ensure the stable operation of the model. When γ tends to be stable, the synthetic image pairs are output.

[0085] For the discrimination results of generated image pairs and real image pairs, cross entropy loss is used to measure their differences. The loss of real image pairs is:

[0086] L real = -log(D real )

[0087] The loss for the synthetic image pair is:

[0088] L syn = -log(1-D syn )

[0089] Among them, D real is the output probability of the discriminator for the real image pair, which should ideally be close to 1, Dsyn is the output probability of the discriminator for the generated image, which should ideally be close to 0. In addition, L real and L syn They are not independent. After introducing the boundary balance mechanism, they are related to each other through a balance factor, and the loss of the generated module is L G :

[0090] L G =||D(G(X))-γG(X)||1

[0091] Where X is the real image, G(X) is the generated image, and the loss of the generation module is defined by calculating the L1 distance between the generated image pair and the real image pair. In particular, we add an additional denoising loss to the discriminator to improve the ability to distinguish between real and generated images by estimating the gradient of the logarithmic density of the real data distribution. The denoising loss function is as follows:

[0092] L dn =||D(x+noise)-x||2 2

[0093] The discriminant module consists of three parts: the reconstruction loss of the real image pair, the reconstruction loss of the generated image pair, and the denoising loss:

[0094] L D =ω noise ·L dn +||D(x)-x||1-γ·||D(G(X))-G(X)||1

[0095] Where x is the real image, ω noise is the weighted coefficient of the denoising loss, which is used to control the proportion of the denoising loss. γ is the dynamic coefficient of the weight used to balance the training, and its adaptive adjustment is:

[0096] γ t+1 =γ t +τ·(σ·L real -L syn )

[0097] Among them, τ is the learning rate, γ is the dynamically adjusted balance factor, and σ is a fixed ratio. This mechanism is used to maintain the balance of the training process and avoid the quality degradation of the generated image pairs or over-optimization of the discriminator. When the quality of the synthesized image pairs is poor, γ will increase. On the contrary, when the quality of the synthesized image pairs is good, γ will decrease. When the change of the balance factor tends to be stable, it can be considered that the training has reached a balanced state and the synthesized image pairs can be output.

[0098] An embodiment of the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is loaded into the processor, it implements any one of the methods for generating micro-expression image pairs based on a linear attention mechanism and a boundary balance mechanism.

[0099] An embodiment of the present invention also provides a storage medium storing a computer program, which, when executed by a processor, implements any one of the methods for generating micro-expression image pairs based on a linear attention mechanism and a boundary balance mechanism.

Claims

1. A method for generating micro-expression image pairs based on a linear attention mechanism and a boundary balance mechanism, characterized in that: The following steps are involved: (1) Obtain video clips, use the FlowNet optical flow method to obtain the starting frame and peak frame images when micro-expressions occur, perform preprocessing, and output them as true micro-expression image pairs; (2) Input high-dimensional random noise and category label conditions, use the branch network and linear attention mechanism to generate and output synthetic micro-expression image pairs; (3) Determine true or false: extract global and local features through global block partitioning convolutional neural network, calculate the difference between the synthesized micro-expression image pair and the real micro-expression image pair, and output the judgment result; (4) Based on the judgment results, the corresponding network weights and parameters are updated; based on the boundary balance mechanism, a boundary balance loss function is constructed and calculated, and training is performed iteratively until the loss function converges, and the synthesized image pair is output.

2. The method for generating micro-expression image pairs based on a linear attention mechanism and a boundary balance mechanism according to claim 1, characterized in that: Step (1) comprises the following steps: (11) Extract continuous static image frames at a fixed frequency, use the directional histogram to perform face detection on each frame, and then crop the area where the face is located; calculate the displacement change of facial pixels through the FlowNet optical flow algorithm, so as to locate and extract the starting frame and peak frame image when the micro-expression occurs; (12) The extracted micro-expression start frame and peak frame images are preprocessed including resizing, facial alignment, and normalization, and output as the true micro-expression image pair.

3. The method for generating micro-expression image pairs based on a linear attention mechanism and a boundary balance mechanism according to claim 1, characterized in that: Step (2) comprises the following steps: (21) inputting high-dimensional random noise and category label conditions, converting the labels into high-dimensional feature representations through one-hot encoding; then concatenating the high-dimensional noise vector and the label condition vector to form a new feature vector; (22) The new feature vector is linearly transformed through the fully connected layer to output a low-resolution feature map; the low-dimensional feature map is converted into a high-dimensional feature map through several layers of transposed convolutional networks until it reaches the branch network; (23) The branch structure includes two independent branch networks of the starting frame and the peak frame, each branch performs feature extraction and is used to generate a synthetic starting frame image and a peak frame image respectively; (24) In the two branch networks, a linear attention mechanism is introduced at the same time to assign different weights; (25) The results of the two branch networks are input again into several layers of transposed convolutional networks, and finally an image pair of the target size is output.

4. The method for generating micro-expression image pairs based on a linear attention mechanism and a boundary balance mechanism according to claim 1, characterized in that: Step (3) comprises the following steps: (31) Extract global and local features by partitioning the convolutional neural network into global blocks; (32) Input the global feature map and the block feature map into the linear attention module to generate an attention weight matrix; perform weighted summation on the weighted global features and local features to obtain a fused feature map; (33) The feature maps of the synthetic image pair and the feature maps of the real image pair are judged using a classification and discrimination method.

5. The method for generating micro-expression image pairs based on a linear attention mechanism and a boundary balance mechanism according to claim 4, characterized in that: Step (31) is as follows: input a synthetic image pair and a real image pair and the corresponding category labels, and extract features through a global block partitioning convolutional neural network; after the input set of image pairs is subjected to optical flow calculation, its horizontal component, vertical component and optical flow strain map are output; each image is evenly divided into a number of blocks of the same size, and features are extracted for each block, and all block features are spliced ​​to obtain global information; at the same time, the key area is divided into blocks again, and separate convolution and pooling operations are performed on each block of the secondary division; all the secondary divided blocks are first feature spliced ​​individually, and then all the block features are spliced ​​according to the channel dimension to obtain local information; finally, the global feature map and local feature map extracted by the horizontal component, vertical component and optical flow strain are spliced ​​respectively, and then the category label information is converted into high-dimensional features and combined with the global and local feature maps through channel splicing.

6. The method for generating micro-expression image pairs based on a linear attention mechanism and a boundary balance mechanism according to claim 5, characterized in that: In step (31), the key areas include: eyebrow area, eye corner area, and mouth corner area.

7. The method for generating micro-expression image pairs based on a linear attention mechanism and a boundary balance mechanism according to claim 4, characterized in that: Step (33) is as follows: Map the feature map of the synthetic image pair and the real image to a high-dimensional space to generate a discriminant vector, and then directly output the classification result through the classification layer. The output value is mapped to [0,1] using a Sigmoid activation function to generate a classification probability, which represents the probability that the generated image pair and the real image pair are judged as real.

8. The method for generating micro-expression image pairs based on a linear attention mechanism and a boundary balance mechanism according to claim 1, characterized in that: Step (4) is as follows: the difference in classification results is calculated and measured through the cross entropy loss function, the weights and parameters of the generation module and the discrimination module are updated according to the cross entropy loss result, and the training process of the generation and discrimination image pairs is dynamically adjusted by the boundary balance mechanism, and the boundary loss function is constructed. The balance factor γ is automatically updated to ensure the stable operation of the model. When γ tends to be stable, the synthetic image is output; wherein, the loss of the real image pair is: L real =-log(D real ) The loss for the synthetic image pair is: L syn =-log(1-D syn ) Among them, D real is the output probability of the discriminator for the real image pair, D syn is the output probability of the discriminator for the generated image. The boundary balance mechanism is introduced to correlate with each other through the balance factor. The loss of the generation module is L G : L G =||D(G(X))-γG(X)||1 Among them, X is the real image, G(X) is the generated image; The discriminant module includes: reconstruction loss of real image pairs and reconstruction loss of generated image pairs as well as denoising loss: L D =ω noise ·L dn +||D(x)-x||1-γ·||D(G(X))-G(X)||1 Where x is the real image, ω noise It is the weighted coefficient of denoising loss, which is used to control the proportion of denoising loss; the denoising loss function is as follows: L dn =||D(x+noise)-x||2 2 ; γ is the dynamic coefficient of weight used to balance training, and is adaptively adjusted to: c t+1 =c t +τ·(σ·L real -L syn ) Among them, τ is the learning rate, γ is the dynamically adjusted balance factor, and σ is a fixed ratio.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into a processor, a method for generating micro-expression image pairs based on a linear attention mechanism and a boundary balance mechanism according to any one of claims 1 to 8 is implemented.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, a method for generating micro-expression image pairs based on a linear attention mechanism and a boundary balance mechanism according to any one of claims 1 to 8 is implemented.

Images