Method and device for realizing fast micro-expression recognition processing based on bidirectional optical flow, processor and computer readable storage medium thereof

Abstract

The present application relates to a kind of based on two-way optical flow to realize the method for fast micro-expression recognition processing, comprising the following steps: according to visual system acquisition tester face micro-expression video segment information;Extract the emotional video segment in emotional memory library, and the facial muscle movement situation of micro-expression in emotional video segment is captured by positive and negative two-way optical flow;Extract method extracts key frame in emotional video segment, and the redundant frame in continuous sequence image is eliminated;Call the optical flow information between key frame in optical flow information memory library.The present application also relates to a kind of two-way optical flow to realize fast micro-expression recognition device, processor and storage medium.The method for fast micro-expression recognition processing based on two-way optical flow of the present application, device, processor and its computer readable storage medium, the facial muscle movement situation of micro-expression is captured by positive and negative two-way optical flow, the micro-expression of tester is identified using muscle movement trend, and the micro-expression recognition accuracy is improved.

Description

Technical Field

[0001] This invention relates to the field of lie detection, and more particularly to the field of micro-expression recognition technology, specifically to a method, apparatus, processor, and computer-readable storage medium for rapid micro-expression recognition processing based on bidirectional optical flow. Background Technology

[0002] Microexpressions are facial expressions generated by genuine inner emotions that accurately reflect a person's true feelings. A standard microexpression lasts between 1 / 5 and 1 / 25 of a second, typically occurs only in specific areas of the face, and involves extremely subtle movements, making it difficult to detect with the naked eye. Previously, analyzing microexpressions required extensive and meticulous observation by highly experienced professionals. In recent years, however, thanks to advancements in artificial intelligence and breakthroughs in computer vision and pattern recognition technologies in microexpression analysis, intelligent real-time rapid microexpression recognition has become a challenging yet valuable research area.

[0003] Currently, mature micro-expression recognition methods mainly fall into two categories: those based on facial feature points and those based on facial texture features. The former primarily utilizes local facial feature points for recognition, while the latter relies on the changing trends of facial texture features. Feature-point-based micro-expression recognition provides a relatively complete framework for micro-expression recognition research; however, it is susceptible to the selection of feature points and has significant limitations. Facial texture-based micro-expression recognition can capture subtle facial changes through global facial features and identify micro-expressions based on the dynamic state of facial textures. However, existing technologies struggle to accurately extract the dynamic state information of facial textures, affecting the accuracy of micro-expression recognition. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, apparatus, processor and computer-readable storage medium for fast micro-expression recognition processing based on bidirectional optical flow that is highly accurate, easy to operate and widely applicable.

[0005] To achieve the above objectives, the present invention provides a method, apparatus, processor, and computer-readable storage medium for fast micro-expression recognition processing based on bidirectional optical flow, as follows:

[0006] The main feature of this method for fast micro-expression recognition based on bidirectional optical flow is that the method includes the following steps:

[0007] (1) Based on the visual system, collect video clips of the test subject's facial micro-expressions and store the collected video clips in the emotional memory bank;

[0008] (2) Extract emotional video segments from the emotional memory bank, capture the facial muscle movements of micro-expressions in the emotional video segments through forward and reverse bidirectional optical flow, extract optical flow information through forward indexing, and correct sequential optical flow through reverse indexing to suppress erroneous distance and direction information; characterize the emotional video segments through facial optical flow information, and store the extracted bidirectional optical flow information in the optical flow information memory bank; evaluate the accuracy of the optical flow information in the optical flow information bank through coordinate error evaluation rules;

[0009] (3) Extract key frames from emotional video clips by using a key frame extraction method based on optical flow field variation effect, and remove redundant frames from continuous sequence images;

[0010] (4) Retrieve the optical flow information between key frames in the optical flow information memory bank and store the optical flow information between key frames in the perception template library; use the optical flow information to characterize the texture running state of facial micro-expressions, and use the optical flow information between key frames as the input of the support vector machine to classify the facial emotional state of the test subject.

[0011] Preferably, step (1) specifically comprises:

[0012] Based on the video clips of the test subject's facial micro-expressions collected by the vision system, the collected video clips are converted into a continuous image sequence. A Gaussian smoothing filter is used to perform distortion correction on the obtained continuous images. The processed micro-expression video clips collected by the vision system are then stored in the emotional memory bank.

[0013] Preferably, in step (2), the extraction of emotional video clips from the emotional memory bank is achieved by capturing the facial muscle movements of micro-expressions in the emotional video clips using bidirectional optical flow, specifically as follows:

[0014] When performing reverse optical flow, the forward frame interval and the reverse frame interval time are the same. The forward and reverse optical flow can be calculated using the following formula:

[0015]

[0016] Where n represents the nth frame of a continuous image sequence. The optical flow vector along the x-axis of the reversed (n+1)th to nth frame images. The optical flow vector along the x-axis of the images from frame n to frame (n+1) in ascending order. The optical flow vector along the y-axis of the reversed (n+1)th to nth frame images. The optical flow vector along the y-axis of the images from frame n to frame (n+1) in ascending order. The velocity components along the x-axis of the images from frame (n+1) to frame n in reverse order. For the velocity components along the x-axis of the images from frame n to frame (n+1) in forward sequence, The velocity components along the y-axis of the images from frame (n+1) to frame n in reverse order. This represents the velocity components along the y-axis of the images from frame n to frame (n+1) in forward sequence. Let t be the grayscale value of the nth frame of the image. n The time of the nth frame.

[0017] Preferably, in step (2), the forward index is used to extract optical flow information, and the reverse index is used to correct the sequential optical flow and suppress erroneous distance and direction information, specifically as follows:

[0018] Assuming that the optical flow velocity components obtained by solving in the forward and reverse directions are the same, that is... but

[0019]

[0020] A linear equation set was constructed using multiple optical flow information from a single frame image, and the singular value decomposition method was used to solve the constructed linear equation set to obtain the results. and

[0021] in, The velocity components along the x-axis of the images from frame (n+1) to frame n in reverse order. The optical flow vector along the x-axis of the reversed (n+1)th to nth frame images. The optical flow vector along the x-axis of the images from frame n to frame (n+1) in ascending order. This represents the velocity components along the y-axis of the images from frame n to frame (n+1) in forward sequence. The optical flow vector along the y-axis of the reversed (n+1)th to nth frame images. The optical flow vector along the y-axis of the images from frame n to frame (n+1) in ascending order. Let t be the grayscale value of the nth frame of the image. n The time of the nth frame.

[0022] Preferably, the step (2) of evaluating the accuracy of optical flow information in the optical flow information database using coordinate error evaluation rules specifically involves:

[0023] The offset between the calculated and actual optical flow values ​​is evaluated using coordinate error evaluation rules. The offset between the calculated and actual optical flow values ​​is evaluated using the following formula:

[0024]

[0025] Among them, I p (x, y, n) and, t(x, y, n) represents the predicted optical flow and the actual optical flow at coordinates (x, y) in the nth frame image. It is a two-dimensional vector, and W and H are the width and height of the optical flow field, respectively.

[0026] Preferably, step (3) specifically includes the following steps:

[0027] The system acquires continuous image frame data using the time axis as the coordinate, determines the first frame as the first keyframe and stores it in the keyframe library, calculates the vector change information (Δu, Δv) of optical flow between the current keyframe and the next ordinary frame, calculates the amount of optical flow change to check whether the amount of optical flow change reaches the set threshold, and if it reaches the set threshold, then the current ordinary frame is set as a relatively important keyframe.

[0028] The calculation and verification of the change in optical flow specifically involves:

[0029] The change in optical flow is calculated using the following formula:

[0030]

[0031]

[0032] kf j ={f i |([Δu>τ]∪[Δv>τ])};

[0033] Among them, f i Indicates a normal frame, kf j This represents a keyframe, where Δu is the cumulative sum of distance changes along the x-axis and Δv is the cumulative sum of distance changes along the y-axis. and This represents the change in optical flow along the x-axis and y-axis at position (m, n) in a normal frame. and The distance change of optical flow along the x-axis and y-axis at the keyframe position (m, n) is given by l, which is the number of optical flow information on a single frame image, and τ is a constant that sets a threshold for the optical flow change.

[0034] Preferably, step (4) of storing the optical flow information between keyframes into the perceptual template library specifically includes the following steps:

[0035] If the difference between the current ordinary frame and the previous keyframe (Δu or Δv) is greater than the set optical flow change threshold (τ), then the current frame is set as a new keyframe and stored in the keyframe library. Otherwise, if both Δu and Δv between the current ordinary frame and the previous keyframe are less than the set optical flow change threshold (τ), then the current ordinary frame and the previous keyframe have a high similarity, and the previous keyframe represents the current ordinary frame. The current ordinary frame is discarded, and the next ordinary frame is compared with the previous keyframe to check whether it meets the keyframe requirements. When a keyframe is successfully selected, the current keyframe and its corresponding optical flow field information are stored in the keyframe library.

[0036] Store the keyframes in the keyframe library according to the following formula:

[0037] U={(F1, kf1, t1), (F2, kf2, t2), (F3, kf3, t3)…(F j kf j , t j )}

[0038] Where U is the corresponding keyframe library, F j For the corresponding kf j Frame optical flow field information, t j This represents the recording time of the j-th keyframe.

[0039] Preferably, in step (4), the texture running state of facial micro-expressions is characterized by optical flow information, and the optical flow information between keyframes is used as input to a support vector machine to classify the facial emotional state of the test subject. Specifically:

[0040] Information from the perceptual template library is used as input to the support vector machine. Optical flow information from the perceptual template library is used to reduce the dimensionality of the data through principal component analysis. The dimensionality-reduced feature vectors are then input into the support vector machine to construct and solve a constrained optimization problem. The optimal solution is determined through optimization during the training process, and the separating hyperplane is obtained. The classification decision function constructed using the separating hyperplane is used to recognize micro-expressions.

[0041] The main feature of this device for realizing fast micro-expression recognition processing based on bidirectional optical flow is that the device comprises:

[0042] A processor is configured to execute computer-executable instructions;

[0043] The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the various steps of the method for fast micro-expression recognition processing based on bidirectional optical flow described above.

[0044] The processor used to implement fast micro-expression recognition processing based on bidirectional optical flow is characterized in that the processor is configured to execute computer-executable instructions, which, when executed by the processor, implement the various steps of the above-described method for fast micro-expression recognition processing based on bidirectional optical flow.

[0045] The main feature of this computer-readable storage medium is that it stores a computer program thereon, which can be executed by a processor to implement the various steps of the above-described method for fast micro-expression recognition processing based on bidirectional optical flow.

[0046] This invention employs a method, apparatus, processor, and computer-readable storage medium based on bidirectional optical flow for rapid micro-expression recognition. It proposes a bidirectional optical flow tracking strategy model that captures facial muscle movements in micro-expressions through forward and reverse optical flow, and uses muscle movement trends to identify the micro-expressions of the test subject, thus improving the accuracy of micro-expression recognition. Addressing the issue of distance and angle errors in traditional optical flow methods during extraction, this invention proposes a reverse optical flow tracking method to correct forward optical flow, suppressing erroneous distance and direction information, improving the accuracy of optical flow information, and enabling better acquisition of facial texture movement information. Considering the large amount of video data, high similarity between consecutive image frames, and numerous redundant frames that consume significant system computing resources, this invention proposes a keyframe extraction method based on optical flow field variation effects to extract effective image frames from the video, improving system real-time performance. This invention uses coordinate error evaluation rules to assess the offset between the calculated and actual optical flow values. These rules consider both distance and angle errors when evaluating errors, reflecting the overall and angular error levels of optical flow in the optical flow field. This invention characterizes the texture state of facial micro-expressions using optical flow information, and uses the optical flow information between keyframes as input to a support vector machine to classify the facial emotional state of the test subject. This invention constructs a novel micro-expression recognition algorithm framework, which not only improves the system's accuracy but also its real-time performance. This method can accurately and quickly identify the facial micro-expressions of the test subject. Attached Figure Description

[0047] Figure 1 This is a flowchart illustrating the method for achieving rapid micro-expression recognition based on bidirectional optical flow according to the present invention.

[0048] Figure 2 This is a schematic diagram of the bidirectional optical flow tracking strategy of the method for fast micro-expression recognition processing based on bidirectional optical flow according to the present invention.

[0049] Figure 3 This is a schematic diagram of the keyframe selection process of the method for fast micro-expression recognition based on bidirectional optical flow according to the present invention.

[0050] Figure 4 This is a flowchart of the keyframe selection strategy for the method of fast micro-expression recognition processing based on bidirectional optical flow according to the present invention.

[0051] Figure 5 The results and magnified views of the LK algorithm, FlowNet2 algorithm, and the algorithm in this paper are extracted on the SMIC dataset.

[0052] Figure 6 The graph shows the feature extraction results of the LBP algorithm, LK algorithm, FlowNet2 algorithm, and the algorithm in this paper on the CASME2 dataset.

[0053] Figure 7 To compare the recognition confusion matrix of the algorithm on the CASME2 dataset.

[0054] Figure 8 The recognition time graphs for the LBP-TOP algorithm, LK algorithm, FlowNet2 algorithm, and the algorithm presented in this paper are shown on the CASMEII dataset. Detailed Implementation

[0055] To more clearly describe the technical content of the present invention, the following description is provided in conjunction with specific embodiments.

[0056] The method for fast micro-expression recognition based on bidirectional optical flow of the present invention includes the following steps:

[0057] (1) Based on the visual system, collect video clips of the test subject's facial micro-expressions and store the collected video clips in the emotional memory bank;

[0058] (2) Extract emotional video segments from the emotional memory bank, capture the facial muscle movements of micro-expressions in the emotional video segments through forward and reverse bidirectional optical flow, extract optical flow information through forward indexing, and correct sequential optical flow through reverse indexing to suppress erroneous distance and direction information; characterize the emotional video segments through facial optical flow information, and store the extracted bidirectional optical flow information in the optical flow information memory bank; evaluate the accuracy of the optical flow information in the optical flow information bank through coordinate error evaluation rules;

[0059] (3) Extract key frames from emotional video clips by using a key frame extraction method based on optical flow field variation effect, and remove redundant frames from continuous sequence images;

[0060] (4) Retrieve the optical flow information between key frames in the optical flow information memory bank and store the optical flow information between key frames in the perception template library; use the optical flow information to characterize the texture running state of facial micro-expressions, and use the optical flow information between key frames as the input of the support vector machine to classify the facial emotional state of the test subject.

[0061] In a preferred embodiment of the present invention, step (1) specifically comprises:

[0062] Based on the video clips of the test subject's facial micro-expressions collected by the vision system, the collected video clips are converted into a continuous image sequence. A Gaussian smoothing filter is used to perform distortion correction on the obtained continuous images. The processed micro-expression video clips collected by the vision system are then stored in the emotional memory bank.

[0063] In a preferred embodiment of the present invention, step (2) involves extracting emotional video clips from the emotional memory bank and capturing the facial muscle movements of micro-expressions in the emotional video clips using bidirectional optical flow. Specifically, this involves:

[0064] When performing reverse optical flow, the forward frame interval and the reverse frame interval time are the same. The forward and reverse optical flow can be calculated using the following formula:

[0065]

[0066] Where n represents the nth frame of a continuous image sequence. The optical flow vector along the x-axis of the reverse sequence from frame (n+1) to frame n. The optical flow vector along the x-axis of the images from frame n to frame (n+1) in ascending order. The optical flow vector along the y-axis of the reverse sequence of frames n+1 to n. The optical flow vector along the y-axis of the images from frame n to frame (n+1) in ascending order. The velocity components along the x-axis of the images from frame (n+1) to frame n in reverse order. For the velocity components along the x-axis of the images from frame n to frame (n+1) in forward sequence, The velocity components along the y-axis of the images from frame (n+1) to frame n in reverse order. The velocity components along the y-axis are the images from frame n to frame (n+1) in forward sequence. Let t be the grayscale value of the nth frame of the image. n The time of the nth frame.

[0067] In a preferred embodiment of the present invention, step (2) uses forward indexing to extract optical flow information and reverse indexing to correct sequential optical flow and suppress erroneous distance and direction information, specifically as follows:

[0068] Assuming that the optical flow velocity components obtained by solving in the forward and reverse directions are the same, that is... but

[0069]

[0070] A linear equation set was constructed using multiple optical flow information from a single frame image, and the singular value decomposition method was used to solve the constructed linear equation set to obtain the desired result. and

[0071] in, The velocity components along the x-axis of the images from frame (n+1) to frame n in reverse order. The optical flow vector along the x-axis of the reverse sequence from frame (n+1) to frame n. The optical flow vector along the x-axis of the images from frame n to frame (n+1) in ascending order. The velocity components along the y-axis are the images from frame n to frame (n+1) in forward sequence. The optical flow vector along the y-axis of the reverse sequence of frames n+1 to n. The optical flow vector along the y-axis of the images from frame n to frame (n+1) in ascending order. Let t be the grayscale value of the nth frame of the image. n The time of the nth frame.

[0072] In a preferred embodiment of the present invention, step (2) of evaluating the accuracy of optical flow information in the optical flow information database using coordinate error evaluation rules specifically includes:

[0073] The offset between the calculated and actual optical flow values ​​is evaluated using coordinate error evaluation rules. The offset between the calculated and actual optical flow values ​​is evaluated using the following formula:

[0074]

[0075] Among them, I p (x, y, n) and I t (x, y, n) represents the predicted optical flow and the actual optical flow at coordinates (x, y) in the nth frame image. It is a two-dimensional vector, and W and H are the width and height of the optical flow field, respectively.

[0076] In a preferred embodiment of the present invention, step (3) specifically includes the following steps:

[0077] The system acquires continuous image frame data using the time axis as the coordinate, determines the first frame as the first keyframe and stores it in the keyframe library, calculates the vector change information (Δu, Δv) of optical flow between the current keyframe and the next ordinary frame, calculates the amount of optical flow change to check whether the amount of optical flow change reaches the set threshold, and if it reaches the set threshold, then the current ordinary frame is set as a relatively important keyframe.

[0078] The calculation and verification of the change in optical flow specifically involves:

[0079] The change in optical flow is calculated using the following formula:

[0080]

[0081]

[0082] kf j ={f i |([Δu>τ]∪[Δv>τ])};

[0083] Among them, f i Indicates a normal frame, kf j This represents a keyframe, where Δu is the cumulative sum of distance changes along the x-axis and Δv is the cumulative sum of distance changes along the y-axis. and This represents the change in optical flow along the x-axis and y-axis at position (m, n) in a normal frame. and The distance change of optical flow along the x-axis and y-axis at the keyframe position (m, n) is given by l, which is the number of optical flow information on a single frame image, and τ is a constant that sets a threshold for the optical flow change.

[0084] In a preferred embodiment of the present invention, the step (4) of storing the optical flow information between keyframes into the perception template library specifically includes the following steps:

[0085] If the difference between the current ordinary frame and the previous keyframe (Δu or Δv) is greater than the set optical flow change threshold (τ), then the current frame is set as a new keyframe and stored in the keyframe library. Otherwise, if both Δu and Δv between the current ordinary frame and the previous keyframe are less than the set optical flow change threshold (τ), then the current ordinary frame and the previous keyframe have a high similarity, and the previous keyframe represents the current ordinary frame. The current ordinary frame is discarded, and the next ordinary frame is compared with the previous keyframe to check whether it meets the keyframe requirements. When a keyframe is successfully selected, the current keyframe and its corresponding optical flow field information are stored in the keyframe library.

[0086] Store the keyframes in the keyframe library according to the following formula:

[0087] U={(F1, kf1, t1), (F1, kf2, t2), (F3, kf3, t3)…(F j kf j , t j )}

[0088] Where U is the corresponding keyframe library, F j For the corresponding kf j Frame optical flow field information, t j This represents the recording time of the j-th keyframe.

[0089] In a preferred embodiment of the present invention, step (4) involves using optical flow information to characterize the texture running state of facial micro-expressions and using the optical flow information between keyframes as input to a support vector machine to classify the facial emotional state of the test subject. Specifically:

[0090] Information from the perceptual template library is used as input to the support vector machine. Optical flow information from the perceptual template library is used to reduce the dimensionality of the data through principal component analysis. The dimensionality-reduced feature vectors are then input into the support vector machine to construct and solve a constrained optimization problem. The optimal solution is determined through optimization during the training process, and the separating hyperplane is obtained. The classification decision function constructed using the separating hyperplane is used to recognize micro-expressions.

[0091] The apparatus of the present invention for realizing fast micro-expression recognition processing based on bidirectional optical flow, wherein the apparatus comprises:

[0092] A processor is configured to execute computer-executable instructions;

[0093] The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the various steps of the method for fast micro-expression recognition processing based on bidirectional optical flow described above.

[0094] The present invention provides a processor for implementing fast micro-expression recognition processing based on bidirectional optical flow, wherein the processor is configured to execute computer-executable instructions, which, when executed by the processor, implement the various steps of the method for implementing fast micro-expression recognition processing based on bidirectional optical flow described above.

[0095] The computer-readable storage medium of the present invention stores a computer program thereon, which can be executed by a processor to implement the various steps of the above-described method for fast micro-expression recognition processing based on bidirectional optical flow.

[0096] Considering the weak intensity and localized nature of micro-expression movements, existing algorithms struggle to achieve fast and accurate recognition. This invention proposes a fast micro-expression recognition method based on bidirectional optical flow. This method captures facial movements using forward and reverse optical flow to identify micro-expressions, improving the accuracy of micro-expression recognition. Furthermore, a keyframe extraction method based on optical flow field effects is introduced to remove redundant frames from video clips, improving system real-time performance. This invention achieves keyframe extraction from emotional video clips and captures the dynamic state of facial textures, enhancing the real-time performance and accuracy of facial micro-expression recognition.

[0097] In specific embodiments of the present invention, the following two examples are provided:

[0098] like Figure 1-8 As shown, this invention provides a mobile robot map construction method based on closed-loop detection and correction, with the following specific steps:

[0099] Step S1: Collect video clips of the test subject's facial micro-expressions using the visual system, and store the collected video clips in the emotional memory bank;

[0100] Specifically, this includes: acquiring video clips of micro-expressions from the test subject's face using a vision system; converting the acquired video clips into a continuous image sequence; and using a Gaussian smoothing filter to perform distortion correction on the continuous images obtained in step S1 to eliminate image distortion caused by changes in the external environment. Based on the working principle of the vision system, the processed micro-expression video clip information is then stored in the emotional memory bank.

[0101] In step S2, emotional video clips are extracted from the emotional memory bank. Facial muscle movements related to micro-expressions in the emotional video clips are captured using bidirectional optical flow. Forward indexing is used to extract optical flow information, while reverse indexing corrects the sequential optical flow, suppressing erroneous distance and direction information. The specific method is as follows:

[0102] Suppose that the gray value of pixel (x,y) in the image at time t is I(x,y,t); at time (t+dt), this pixel moves to the point (x+dx,y+dy,t+dt). According to the gray value invariance theorem, that is, when the image time interval is short, the gray value in the image remains unchanged, and optical flow tracing satisfies dI(x,y,t) / dt=0. Based on the principle of image pixel gray value conservation, optical flow tracing can be expressed as:

[0103] I(x,y,t)=I(x+dx,y+dy,t+dt)

[0104] Assuming the amount of exercise is very small, expanding the right side of the equation using Taylor's formula yields:

[0105]

[0106] Where τ is a higher-order infinitesimal, ignoring the infinitesimal terms in Equation 2, and substituting Equation 2 into Equation 1, we get:

[0107]

[0108] in, and These are the horizontal component x and the vertical component y of the optical flow vector I(x, y, t), respectively.

[0109] Use respectively Let G represent the horizontal component x and the vertical component y of the optical flow vector I(x, y, t). x G y Substituting into Equation 2 and performing a Taylor expansion, ignoring higher-order infinite terms (second order and above), we obtain:

[0110]

[0111] in, Let t be the grayscale value of the nth frame of the image. n The time of the nth frame.

[0112] When performing reverse optical flow, the forward frame interval and the reverse frame interval time are the same. The methods for solving the forward and reverse optical flow are as follows:

[0113]

[0114] Where n represents the nth frame of a continuous image sequence. The optical flow vector along the x-axis of the reversed (n+1)th to nth frame images. The optical flow vector along the x-axis of the images from frame n to frame (n+1) in ascending order. The optical flow vector along the y-axis of the reversed (n+1)th to nth frame images. The optical flow vector along the y-axis of the images from frame n to frame (n+1) in ascending order. The velocity components along the x-axis of the images from frame (n+1) to frame n in reverse order. For the velocity components along the x-axis of the images from frame n to frame (n+1) in forward sequence, The velocity components along the y-axis of the images from frame (n+1) to frame n in reverse order. This represents the velocity components along the y-axis of the images from frame n to frame (n+1) in forward sequence. Let t be the grayscale value of the nth frame of the image. n The time of the nth frame.

[0115] Assuming that the optical flow velocity components obtained by solving in the forward and reverse directions are the same, that is... but

[0116]

[0117] Specifically, a linear equation set is constructed using multiple optical flow information from a single frame image, and the linear equation set is solved using singular value decomposition to obtain the solution. and

[0118] The design scheme of this invention can correct forward optical flow by reverse optical flow tracing, suppress erroneous distance and direction information, improve the accuracy of optical flow information, and better capture the muscle movements of facial micro-expressions.

[0119] Preferably, in step S2, the accuracy of the optical flow information in the optical flow information database is evaluated using the coordinate error evaluation rule constructed in this patent. Specifically:

[0120] The accuracy of optical flow is affected by distance and angle errors. This patent uses a coordinate error evaluation rule to assess the deviation between the calculated and actual optical flow values. This rule takes both distance and angle errors into account when evaluating errors. It reflects the overall error and angle error level of the optical flow in the optical flow field. The calculation formula is as follows:

[0121]

[0122] Among them, I p (x, y, n) and I t (x, y, n) represents the predicted optical flow and the actual optical flow at coordinates (x, y) in the nth frame image, and is a two-dimensional vector. W and H are the width and height of the optical flow field, respectively.

[0123] In step S3, weights are learned according to the self-perception inverse mapping network learning rules to obtain the response values ​​of the grid cells, as follows:

[0124] Keyframes in emotional video clips are extracted using a keyframe extraction method based on optical flow field variation effects, and redundant frames in continuous image sequences are removed. The specific steps for obtaining keyframes are as follows:

[0125] To address the challenges of large amounts of video data, high similarity between consecutive image frames, and numerous redundant frames that consume significant system computing resources, this paper proposes a keyframe extraction method based on optical flow field variation effects. This method extracts effective image frames from the video, improving system real-time performance. A schematic diagram of the keyframe selection process is shown below. Figure 3As shown. Using the time axis as the coordinate system, continuous image frame data is acquired. First, the first frame is determined as the first keyframe and stored in the keyframe library. The vector change information (Δu, Δv) of optical flow between the current keyframe and the next ordinary frame is calculated. The optical flow change is checked to see if it reaches a set threshold. If it does, the current ordinary frame is designated as a more important keyframe. The calculation method for checking the optical flow change is as follows:

[0126]

[0127]

[0128] kf j ={f i |([Δu>τ]∪[Δv>τ])}

[0129] Among them, f i Indicates a normal frame, kf j This represents a keyframe, where Δu is the cumulative sum of distance changes along the x-axis and Δv is the cumulative sum of distance changes along the y-axis. and This represents the change in optical flow along the x-axis and y-axis at position (m, n) in a normal frame. and The distance change of optical flow along the x-axis and y-axis at the keyframe position (m, n) is denoted by l, where l is the number of optical flow information items in a single frame image. τ is a threshold value set for the optical flow change, which is a constant. In this invention, it is set to 8.5. If this threshold is set too small, the extracted keyframes will have many similar frame images, resulting in more redundant frames and affecting the real-time performance of the micro-expression recognition system. If it is set too large, some keyframes will be difficult to represent ordinary frames adjacent to the keyframes, affecting the accuracy of micro-expression recognition.

[0130] In step S4, optical flow information between keyframes is retrieved from the optical flow information memory, and this information is stored in the perceptual template library. Optical flow information is used to represent the texture running state information of facial micro-expressions, and the optical flow information between keyframes is used as input to a support vector machine to classify the facial emotional state of the test subject. The specific steps for storing the optical flow information between keyframes in the perceptual template library are as follows:

[0131] If the difference between the current ordinary frame and the previous keyframe (Δu or Δv) is greater than the set optical flow change threshold (τ), the current frame is set as a new keyframe and stored in the keyframe library. Conversely, if both Δu and Δv between the current ordinary frame and the previous keyframe are less than the set optical flow change threshold (τ), it indicates that the current ordinary frame and the previous keyframe have a high similarity, and the previous keyframe can represent the current ordinary frame. In this case, to save computation, the current ordinary frame will be discarded, and the next ordinary frame will be compared with the previous keyframe to check if it meets the keyframe requirements. When a keyframe is successfully selected, the current keyframe and its corresponding optical flow field information are stored in the keyframe library. The storage method is shown in the following formula, and the keyframe extraction strategy flowchart is shown below. Figure 4 As shown.

[0132] U={(F1, kf1, t1), (F2, kf2, t2), (F3, kf3, t3)…(F j kf j , t j )}

[0133] Where U is the corresponding keyframe library, F j For the corresponding kf j Frame optical flow field information, t j This represents the recording time of the j-th keyframe.

[0134] The design scheme of this invention can be used to extract effective image frames from video data with large amounts of information and high similarity between consecutive image frames. It can also remove redundant information from the system and improve the real-time performance of the system by using a key frame extraction method based on the optical flow field variation effect.

[0135] In step S4, optical flow information is used to characterize the texture state information of facial micro-expressions. The optical flow information between keyframes is used as input to a support vector machine (SVM) to classify the facial emotional state of the test subject. Specifically, this paper uses information from a perceptual template library as input to the SVM. The optical flow information from the perceptual template library is used to reduce the dimensionality of the data through principal component analysis (PCA). The dimensionality-reduced feature vectors are then input into the SVM to construct and solve a constrained optimization problem. The optimal solution is determined through optimization during the training process, and a separating hyperplane is obtained. Finally, the classification decision function constructed using the separating hyperplane is used to recognize micro-expressions.

[0136] This invention constructs a novel micro-expression recognition algorithm framework, which not only improves the system's accuracy but also its real-time performance. This method can accurately and quickly identify the facial micro-expressions of test subjects.

[0137] The following section will illustrate the above-mentioned fast micro-expression recognition method based on bidirectional optical flow with specific experiments.

[0138] The computer used in this experiment had the following configuration: an i5-9400F CPU, an 8-core processor with a clock speed of 2.9GHz, and 8GB of RAM. The accuracy of micro-expression recognition was verified using the SMIC, CASME, and CASMEII micro-expression datasets.

[0139] Figure 5 The images show the optical flow field information obtained from the LK algorithm, FlowNet2 algorithm, and the proposed algorithm on SMIC data. The micro-expression in this image is a positive one, with a slightly pursed mouth, a slightly raised chin, and little other facial texture movement, indicating dissatisfaction with the current scene or event. Figures (a), (e), and (i) show the global optical flow maps obtained by the LK algorithm, FlowNet2 algorithm, and the proposed algorithm, respectively. The right-hand columns, regions A, B, and C, are magnified views of parts of the LK algorithm (a), FlowNet2 (e), and the proposed algorithm (i), respectively. Region A is a magnified view near the mouth, region B is a magnified view near the eyebrows, and region C is a magnified view of the background area. As shown in Figure (b), the traditional LK algorithm failed to capture the texture motion of the mouth area effectively. Similarly, Figure (c) reveals significant chaotic optical flow information around the eyes. Figure (d), a magnified view of the background, shows no motion change in this area, yet the LK algorithm extracted incorrect uniform optical flow information, mistakenly identifying it as a small-scale background movement. The proposed algorithm, compared to the traditional LK algorithm, demonstrates superior texture trajectory extraction capabilities. Figures (j), (k), and (l) show that the proposed algorithm effectively captures texture motion in the mouth and chin areas, and avoids chaotic erroneous optical flow information in the eyebrows and background area C. This is due to the introduction of reverse optical flow to correct sequential erroneous optical flow. While the deep learning-based FlowNet2 network also exhibits good texture extraction capabilities in the mouth and chin areas, it still contains significant erroneous optical flow information in areas B and C.

[0140] Figure 6The images show feature extraction results of the LBP, LK, FlowNet2, and proposed algorithms on the CASME2 dataset. The four rows of images from top to bottom are selected from the EP01_11f, EP04_02f, EP02_31, and EP06_01 sequences in the CASMEII dataset. LBP features can capture the texture of micro-expressions, but they cannot describe the changes in texture. Optical flow features can characterize the texture movement of micro-expressions by measuring the distance and angle of optical flow. However, the LK algorithm still contains many erroneous optical flow information in static areas (such as the background wall). While the FlowNet2 optical flow method achieves better results than the LK algorithm, it still contains many errors in capturing the optical flow information of the eyebrow area in the EP04_02f sequence. The proposed algorithm, after improvements, demonstrates good feature extraction capabilities across all sequences in the CASME dataset.

[0141] Table 1 shows the comparison results of various algorithms on the SMIC, CASME, and CASMEII datasets. The table shows that the proposed method achieves the highest UAR (Ultimate Average Recognition) and UF1 on all three datasets. The proposed algorithm and other algorithms show slightly lower UAR and UF1 on the CASME and CASMEII datasets compared to the SMIC dataset. This is because the SMIC dataset classifies expressions into three categories: positive, negative, and normal, while the CASME and CASMEII datasets classify them into five categories, which can easily lead to classification confusion and lower accuracy. However, the proposed algorithm still performs best on the SMIC and CASME datasets. The proposed algorithm, by constructing a bidirectional optical flow tracing model, can correct erroneous optical flow information generated by the forward index through reverse indexing, effectively extracting the motion texture information of facial micro-expressions and improving the accuracy of micro-expression recognition. The table shows that the proposed method improves UAR by approximately 11.2% and UF1 by approximately 10.6% compared to the BDCNN method. This demonstrates that the proposed algorithm has good micro-expression recognition capabilities.

[0142] Table 2 Comparison of results for various algorithms on the SMIC, CASME, and CASMEII datasets.

[0143]

[0144] The recognition confusion matrix diagrams of various algorithms on the CASME2 dataset are as follows: Figure 7As shown, the columns of the matrix represent the predicted categories, the rows represent the predicted results for each category, and the diagonal of the matrix represents the proportion of correctly classified emotions out of those emotions. Here, Hap.Dig.Sur.Rep.Oth. represent happiness, disgust, surprise, sadness, and other micro-expressions, respectively. The figure shows that traditional LBP and LBP-TOP algorithms perform poorly in recognition. This is because while these algorithms can extract the texture of micro-expressions, they cannot characterize their motion. The LK algorithm, while capable of characterizing the texture of micro-expressions, extracts a significant amount of erroneous optical flow information, affecting recognition accuracy. Therefore, micro-expression recognition algorithms based on the LK approach have low accuracy. The SpareMDMO algorithm divides the face into 36 regions of interest (ROIs) using 66 facial feature points and identifies micro-expressions based on the principal direction optical flow characteristics of each RIO, reducing the influence of redundant facial features on micro-expressions and improving recognition accuracy. FlowNet, STSTNet, and RCA-N are deep learning-based micro-expression recognition methods. These methods have high micro-expression recognition rates, especially the RCA-N algorithm, which achieves a recognition rate of 0.86 for happy expressions. However, these deep learning-based micro-expression recognition methods have poor real-time performance, making it difficult to run online in real time. Furthermore, they have poor recognition rates for certain individual expressions; for example, the RCA-N algorithm only achieves a recognition rate of 0.57 for disgust, and the FlowNet2 algorithm only achieves a recognition rate of 0.58 for happiness. Our proposed algorithm, through a bidirectional optical flow tracing model and a keyframe selection strategy, not only improves the real-time performance but also the accuracy. As shown in the figure, compared with other algorithms, our proposed algorithm has a higher recognition rate for individual expressions, with the highest recognition rate for sadness reaching 0.87.

[0145] Figure 8 The figure shows a comparison of recognition time for the LBP algorithm, LK algorithm, FlowNet2 algorithm, and the algorithm presented in this paper on the CASMEII dataset. The algorithm presented in this paper reduces redundant frames by introducing a key frame selection strategy based on bidirectional optical flow, which reduces the recognition time by approximately 25.9% compared to the micro-expression recognition algorithm based on the LK algorithm and by approximately 36.5% compared to the micro-expression recognition algorithm based on the FlowNet2 network.

[0146] The experiments above demonstrate that the method of this invention reduces the error rate of optical flow capture and improves the accuracy of micro-expression recognition by introducing a bidirectional optical flow strategy model. Furthermore, the algorithm proposes a keyframe selection strategy model, reducing redundant frames in video clips and improving the real-time performance of the system. In terms of accuracy, the method of this invention improves by approximately 12.5% ​​compared to the Sparse MDMO algorithm. In terms of computational speed, the algorithm reduces computation by approximately 36.5% compared to the FlowNet2-based micro-expression recognition algorithm, indicating that the proposed algorithm exhibits excellent micro-expression recognition capabilities.

[0147] Example 2:

[0148] Corresponding to Embodiment 1 of the present invention, Embodiment 2 of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0149] Step S1: Collect video clips of the test subject's facial micro-expressions using the visual system, and store the collected video clips in the emotional memory bank;

[0150] Step S2: Extract emotional video segments from the emotional memory bank, and use the bidirectional optical flow method constructed in this patent to extract the optical flow information of the emotional video segments;

[0151] Step S3: Extract key frames from emotional video clips using a key frame extraction method based on optical flow field variation effect, and remove redundant frames from continuous sequence images;

[0152] Step S4: Retrieve bidirectional optical flow information between keyframes, use bidirectional optical flow vector information to characterize the texture running state of facial micro-expressions, and use the optical flow between keyframes as input to support vector machine to classify the facial emotional state of the test subject.

[0153] The aforementioned storage media include: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), optical discs, and other media capable of storing program code.

[0154] The specific limitations regarding the steps implemented after program execution in a computer-readable storage medium can be found in Embodiment 1, and will not be described in detail here.

[0155] Example 3

[0156] Corresponding to Embodiment 1 of the present invention, Embodiment 3 of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps:

[0157] Step S1: Collect video clips of the test subject's facial micro-expressions using the visual system, and store the collected video clips in the emotional memory bank;

[0158] Step S2: Extract emotional video segments from the emotional memory bank, and use the bidirectional optical flow method constructed in this patent to extract the optical flow information of the emotional video segments;

[0159] Step S3: Extract key frames from emotional video clips using a key frame extraction method based on optical flow field variation effect, and remove redundant frames from continuous sequence images;

[0160] Step S4: Retrieve bidirectional optical flow information between keyframes, use bidirectional optical flow vector information to characterize the texture running state of facial micro-expressions, and use the optical flow between keyframes as input to support vector machine to classify the facial emotional state of the test subject.

[0161] The specific limitations regarding the implementation steps of the computer device mentioned above can be found in Embodiment 1, and will not be described in detail here.

[0162] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0163] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.

[0164] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0165] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution device. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0166] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The corresponding program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0167] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0168] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0169] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0170] This invention employs a method, apparatus, processor, and computer-readable storage medium based on bidirectional optical flow for rapid micro-expression recognition. It proposes a bidirectional optical flow tracking strategy model that captures facial muscle movements in micro-expressions through forward and reverse optical flow, and uses muscle movement trends to identify the micro-expressions of the test subject, thus improving the accuracy of micro-expression recognition. Addressing the issue of distance and angle errors in traditional optical flow methods during extraction, this invention proposes a reverse optical flow tracking method to correct forward optical flow, suppressing erroneous distance and direction information, improving the accuracy of optical flow information, and enabling better acquisition of facial texture movement information. Considering the large amount of video data, high similarity between consecutive image frames, and numerous redundant frames that consume significant system computing resources, this invention proposes a keyframe extraction method based on optical flow field variation effects to extract effective image frames from the video, improving system real-time performance. This invention uses coordinate error evaluation rules to assess the offset between the calculated and actual optical flow values. These rules consider both distance and angle errors when evaluating errors, reflecting the overall and angular error levels of optical flow in the optical flow field. This invention characterizes the texture state of facial micro-expressions using optical flow information, and uses the optical flow information between keyframes as input to a support vector machine to classify the facial emotional state of the test subject. This invention constructs a novel micro-expression recognition algorithm framework, which not only improves the system's accuracy but also its real-time performance. This method can accurately and quickly identify the facial micro-expressions of the test subject.

[0171] In this specification, the invention has been described with reference to specific embodiments thereof. However, it will be apparent that various modifications and variations can be made without departing from the spirit and scope of the invention. Therefore, the specification and drawings should be considered illustrative rather than restrictive.

Claims

1. A method for rapid micro-expression recognition processing based on bidirectional optical flow, characterized in that, The method includes the following steps: (1) Based on the visual system, collect video clips of the test subject's facial micro-expressions and store the collected video clips in the emotional memory bank; (2) Extract emotional video segments from the emotional memory bank, capture the facial muscle movements of micro-expressions in the emotional video segments through forward and reverse bidirectional optical flow, extract optical flow information through forward indexing, and correct sequential optical flow through reverse indexing to suppress erroneous distance and direction information; characterize the emotional video segments through facial optical flow information, and store the extracted bidirectional optical flow information in the optical flow information memory bank; evaluate the accuracy of the optical flow information in the optical flow information bank through coordinate error evaluation rules; (3) Extract key frames from emotional video clips using a key frame extraction method based on optical flow field variation effect, and remove redundant frames from continuous sequence images; (4) Retrieve the optical flow information between key frames in the optical flow information memory bank and store the optical flow information between key frames in the perception template library; use the optical flow information to characterize the texture running state of facial micro-expressions, and use the optical flow information between key frames as the input of the support vector machine to classify the facial emotional state of the test subject. Step (2) involves extracting emotional video clips from the emotional memory bank and capturing the facial muscle movements of micro-expressions in the emotional video clips using bidirectional optical flow. Specifically: When performing reverse optical flow, the forward frame interval and the reverse frame interval time are the same. The forward and reverse optical flow can be calculated using the following formula: Where n represents the nth frame of a continuous image sequence. The optical flow vector along the x-axis of the reverse sequence from frame (n+1) to frame n. The optical flow vector along the x-axis of the images from frame n to frame (n+1) in ascending order. The optical flow vector along the y-axis of the reverse sequence of frames n+1 to n. The optical flow vector along the y-axis of the images from frame n to frame (n+1) in ascending order. The velocity components along the x-axis of the images from frame (n+1) to frame n in reverse order. For the velocity components along the x-axis of the images from frame n to frame (n+1) in forward sequence, The velocity components along the y-axis of the images from frame (n+1) to frame n in reverse order. The velocity components along the y-axis are the images from frame n to frame (n+1) in forward sequence. Let be the grayscale value of the nth frame image. The time of the nth frame.

2. The method for fast micro-expression recognition processing based on bidirectional optical flow according to claim 1, characterized in that, The specific steps (1) are as follows: Based on the video clips of the test subject's facial micro-expressions collected by the vision system, the collected video clips are converted into a continuous image sequence. A Gaussian smoothing filter is used to perform distortion correction on the obtained continuous images. The processed micro-expression video clips collected by the vision system are then stored in the emotional memory bank.

3. The method for fast micro-expression recognition processing based on bidirectional optical flow according to claim 1, characterized in that, In step (2), the forward index is used to extract optical flow information, and the reverse index is used to correct the sequential optical flow and suppress erroneous distance and direction information. Specifically: Assuming that the optical flow velocity components obtained by solving in the forward and reverse directions are the same, that is... = , = ,but ； A linear equation set was constructed using multiple optical flow information from a single frame image, and the singular value decomposition method was used to solve the constructed linear equation set to obtain the desired result. and ; in, The velocity components along the x-axis of the images from frame (n+1) to frame n in reverse order. The optical flow vector along the x-axis of the reversed (n+1)th to nth frame images. The optical flow vector along the x-axis of the images from frame n to frame (n+1) in ascending order. This represents the velocity components along the y-axis of the images from frame n to frame (n+1) in forward sequence. The optical flow vector along the y-axis of the reversed (n+1)th to nth frame images. The optical flow vector along the y-axis of the images from frame n to frame (n+1) in ascending order. Let be the grayscale value of the nth frame image. The time of the nth frame.

4. The method for fast micro-expression recognition processing based on bidirectional optical flow according to claim 1, characterized in that, The step (2) described above, which uses coordinate error evaluation rules to assess the accuracy of optical flow information in the optical flow information database, specifically involves: The offset between the calculated and actual optical flow values ​​is evaluated using coordinate error evaluation rules. The offset between the calculated and actual optical flow values ​​is evaluated using the following formula: in, and Represents the coordinates of the nth frame image. The predicted optical flow and the actual optical flow at a given location are two-dimensional vectors, where W and H are the width and height of the optical flow field, respectively.

5. The method for fast micro-expression recognition processing based on bidirectional optical flow according to claim 1, characterized in that, Step (3) specifically includes the following steps: Continuous image frame data is acquired using the time axis as the coordinate system. The first frame is identified as the first keyframe and stored in the keyframe library. The vector change information of optical flow between the current keyframe and the next ordinary frame is calculated. The optical flow change is calculated and checked to see if it reaches a set threshold. If it does, the current ordinary frame is set as a key frame that is relatively important. The calculation and verification of the change in optical flow specifically involves: The change in optical flow is calculated using the following formula: ； ； ； in, Indicates a normal frame. Indicates a keyframe. This is the cumulative sum of the changes in optical flow along the x-axis. This is the cumulative sum of the changes in optical flow along the y-axis. and This represents the change in optical flow along the x-axis and y-axis at position (m,n) in a normal frame. and Let be the distance change along the x-axis and y-axis of the optical flow at the keyframe position (m,n), and l be the number of optical flow information items in a single frame image. The threshold value set for the change in optical flow is a constant.

6. The method for fast micro-expression recognition processing based on bidirectional optical flow according to claim 1, characterized in that, Step (4) of storing the optical flow information between keyframes into the perceptual template library specifically includes the following steps: If the current normal frame is between the previous keyframe and or Greater than the set optical flow change threshold If the current frame is set as the new keyframe and stored in the keyframe library, then the current frame is set as the new keyframe and stored in the keyframe library; otherwise, if the current normal frame is close to the previous keyframe... and All are less than the set optical flow change threshold. If the current ordinary frame is highly similar to the previous key frame, the previous key frame represents the current ordinary frame; discard the current ordinary frame and compare the next ordinary frame with the previous key frame to check if it meets the key frame requirements; when the key frame is successfully selected, store the current key frame and the corresponding optical flow field information in the key frame library. Store the keyframes in the keyframe library according to the following formula: in, For the corresponding keyframe library, For the corresponding Frame optical flow field information, This represents the recording time of the j-th keyframe.

7. The method for fast micro-expression recognition processing based on bidirectional optical flow according to claim 1, characterized in that, Step (4) involves using optical flow information to characterize the texture of facial micro-expressions and using the optical flow information between keyframes as input to a support vector machine to classify the facial emotional state of the test subject. Specifically: Information from the perceptual template library is used as input to the support vector machine. Optical flow information from the perceptual template library is used to reduce the dimensionality of the data through principal component analysis. The dimensionality-reduced feature vectors are then input into the support vector machine to construct and solve a constrained optimization problem. The optimal solution is determined through optimization during the training process, and the separating hyperplane is obtained. The classification decision function constructed using the separating hyperplane is used to recognize micro-expressions.

8. An apparatus for realizing fast micro-expression recognition processing based on bidirectional optical flow, characterized in that, The device includes: A processor is configured to execute computer-executable instructions; The memory stores one or more computer-executable instructions, which, when executed by the processor, implement the steps of the method for fast micro-expression recognition processing based on bidirectional optical flow as described in any one of claims 1 to 7.

9. A processor for implementing fast micro-expression recognition processing based on bidirectional optical flow, characterized in that, The processor is configured to execute computer-executable instructions, which, when executed by the processor, implement the steps of the method for fast micro-expression recognition processing based on bidirectional optical flow as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the steps of the method for fast micro-expression recognition processing based on bidirectional optical flow as described in any one of claims 1 to 7.

Images