Milk preference prediction method and device based on micro-expression recognition and medium

Abstract

The application discloses a milk preference degree prediction method and device based on micro-expression recognition and a medium, and relates to the technical field of image processing and behavior analysis. The method comprises the following steps: decomposing facial video data of a target when the target tastes milk into an image sequence and performing pretreatment; adopting a multi-scale optical flow method to capture facial micro-expression changes and extract facial space-time features; constructing a cross-modal time sequence module, processing horizontal and vertical optical flow features by using an encoder, and fusing the processed horizontal and vertical optical flow features to obtain fused features; constructing and training a micro-expression recognition model; constructing a milk preference degree prediction model based on the cross-modal time sequence module, and combining micro-expression features and the fused features to perform prediction; training the milk preference degree prediction model; pretreating a video to be recognized, inputting the pretreated video into the trained milk preference degree prediction model, and finally outputting a milk preference degree score of the target. The application can realize high-automation and high-accuracy milk preference degree prediction.

Description

Technical Field

[0001] This invention relates to the fields of image processing and behavior analysis technology, and more specifically, to a method, apparatus, and medium for predicting milk preference based on micro-expression recognition. Background Technology

[0002] Consumer preference testing plays a crucial role in product development and marketing strategies. Traditional consumer preference testing typically relies on explicit measurement methods, such as questionnaires and rating scales. To ensure the reliability of consumer tests, these tests require significant time to recruit consumers. Furthermore, these methods are susceptible to consumer subjective biases and social expectations, leading to measurement results that do not accurately reflect consumers' true preferences and limiting the accuracy of preference testing.

[0003] In recent years, with the development of artificial intelligence technology, implicit measurement methods based on facial expression analysis have gradually become a research hotspot. Facial expression analysis can capture consumers' unconscious emotional reactions when tasting food, thereby more accurately predicting their preferences. However, existing facial expression analysis technologies mainly target macro-expressions (MaEs), making it difficult to capture the subtle facial changes that consumers produce when tasting similar foods. Micro-expressions (MEs) are extremely short-lived (0.065–0.5 seconds) and uncontrolled facial expressions that can more accurately reflect consumers' true emotions.

[0004] Therefore, developing a highly automated and accurate method for automatically predicting milk preferences based on micro-expression recognition is of significant practical importance. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a method, apparatus, and medium for predicting milk preferences based on micro-expression recognition, thereby achieving highly automated and highly accurate prediction of milk preferences.

[0006] In a first aspect, the present invention provides a method for predicting milk preference based on micro-expression recognition, the method comprising:

[0007] Decompose micro-expression videos into micro-expression image sequences;

[0008] The facial video data of the target tasting milk was decomposed into an image sequence;

[0009] The image sequence is preprocessed to obtain a preprocessed image sequence; wherein, the preprocessing includes face detection, alignment and cropping;

[0010] Based on the preprocessed image sequence, a multi-scale optical flow method is used to capture facial micro-expression changes and extract facial spatiotemporal features, which include horizontal optical flow features and vertical optical flow features between each frame and the starting frame.

[0011] A cross-modal temporal module is constructed, and the horizontal and vertical optical flow features are processed separately using an encoder. The processed horizontal and vertical optical flow features are then fused to obtain a fused feature, which can characterize global temporal dependencies.

[0012] A micro-expression recognition model was constructed and trained, mapping basic emotion categories into three classes: negative, positive, and surprised. The trained micro-expression model was then used to extract micro-expression features from preprocessed image sequences.

[0013] A milk preference prediction model is constructed based on a cross-modal temporal module, and prediction is made by combining micro-expression features and fused features.

[0014] The milk preference prediction model is trained.

[0015] The video to be identified is preprocessed and input into the trained milk preference prediction model, and the final output is the target's preference rating for milk.

[0016] Further, the image sequence is preprocessed to obtain a preprocessed image sequence, including:

[0017] The Dlib library was used to detect facial features and multiple facial key points in each frame of the image sequence;

[0018] Affine transformation was used to align multiple facial key points;

[0019] Using the inner corner of the left eye as the base point, effective facial regions of consistent size are cropped out as preprocessed images. Multiple preprocessed images are then combined in their original order to obtain a preprocessed image sequence.

[0020] Furthermore, based on the preprocessed image sequence, a multi-scale optical flow method is used to capture facial micro-expression changes and extract facial spatiotemporal features, including:

[0021] The first frame of the preprocessed image sequence is used as the starting frame;

[0022] The TV-L1 optical flow estimation algorithm is used to calculate the horizontal and vertical optical flow characteristics between each frame and the starting frame;

[0023] The horizontal optical flow sequence and vertical optical flow sequence formed by the horizontal optical flow features between each frame and the starting frame are used as facial spatiotemporal features.

[0024] Furthermore, the horizontal and vertical optical flow features are processed separately using an encoder, and the processed horizontal and vertical optical flow features are fused to obtain fused features, including:

[0025] The Vision Mamba encoder was used to perform bidirectional temporal modeling of horizontal and vertical optical flow features, respectively.

[0026] Enhance the correlation between horizontal and vertical optical flow features by sharing the state transition matrix;

[0027] The processed horizontal and vertical optical flow features are fused to obtain fused features, thereby establishing the temporal relationship between local and global features.

[0028] Furthermore, the steps of the milk preference prediction model to predict milk preference include:

[0029] Optical flow characteristics are processed using a cross-modal timing module;

[0030] The micro-expression features are fused with the processed optical flow features to obtain the first fused feature. The VisionMamba encoder is used to perform temporal modeling on the first fused feature to obtain the temporal feature.

[0031] The temporal features are input into a fully connected layer, and the fully connected layer outputs a consumer's preference rating for milk.

[0032] Further, the milk preference prediction model is trained, including:

[0033] Use the video data corresponding to one of the targets as the test set, and the video data of the other targets as the training set;

[0034] The optimal combination of parameters for the model is determined by cross-validation, and the optimal combination of parameters includes multiple sets of optimal parameters.

[0035] Multiple prediction models are obtained by configuring the milk preference prediction model with different optimal parameters. All video data are used to train the multiple prediction models, and the best-performing prediction model is saved as the trained milk preference prediction model.

[0036] Secondly, the present invention provides a milk preference prediction device based on micro-expression recognition, the device comprising:

[0037] The video data decomposition module is configured to decompose the facial video data of the target tasting milk into an image sequence;

[0038] An image preprocessing module is configured to preprocess the image sequence to obtain a preprocessed image sequence; wherein, the preprocessing includes face detection, alignment, and cropping;

[0039] The facial spatiotemporal feature extraction module is configured to capture facial micro-expression changes based on the preprocessed image sequence using a multi-scale optical flow method, and extract facial spatiotemporal features, which include horizontal optical flow features and vertical optical flow features between each frame and the starting frame.

[0040] The feature fusion module is configured to construct a cross-modal temporal module, which uses an encoder to process horizontal and vertical optical flow features respectively, and then fuses the processed horizontal and vertical optical flow features to obtain fused features, which can characterize global temporal dependencies.

[0041] The micro-expression feature extraction module is configured to build and train a micro-expression recognition model, map basic emotion categories into three categories: negative, positive, and surprised, and use the trained micro-expression model to extract micro-expression features from preprocessed image sequences.

[0042] The prediction model building module is configured to build a milk preference prediction model based on the cross-modal time series module, combining micro-expression features and fused features for prediction;

[0043] The prediction model training module is configured to train the milk preference prediction model;

[0044] The preference prediction module is configured to preprocess the video to be identified, input it into the trained milk preference prediction model, and finally output the target's preference rating for milk.

[0045] Furthermore, the image preprocessing module is further configured as follows:

[0046] The Dlib library was used to detect facial features and multiple facial key points in each frame of the image sequence;

[0047] Affine transformation was used to align multiple facial key points;

[0048] Using the inner corner of the left eye as the base point, effective facial regions of consistent size are cropped out as preprocessed images. Multiple preprocessed images are then combined in their original order to obtain a preprocessed image sequence.

[0049] Furthermore, the facial spatiotemporal feature extraction module is further configured as follows:

[0050] The first frame of the preprocessed image sequence is used as the starting frame;

[0051] The TV-L1 optical flow estimation algorithm is used to calculate the horizontal and vertical optical flow characteristics between each frame and the starting frame;

[0052] The horizontal optical flow sequence and vertical optical flow sequence formed by the horizontal optical flow features between each frame and the starting frame are used as facial spatiotemporal features.

[0053] Thirdly, the present invention provides a readable storage medium storing one or more programs that can be executed by one or more processors to implement the method described above.

[0054] The present invention has at least the following beneficial effects:

[0055] This invention addresses the difficulty in capturing subtle facial expression changes in existing technologies by proposing an automatic prediction framework based on micro-expression recognition. First, multi-scale optical flow is used to extract spatiotemporal facial features, capturing the micro-expression changes of a target while tasting milk. Second, a cross-modal temporal module is constructed using a Vision Mamba encoder to establish global temporal dependencies. Finally, by fusing micro-expression features and optical flow features, the automatic prediction of the target's milk preference is achieved. This method effectively captures consumers' unconscious emotional responses, significantly improving prediction accuracy and automation. Attached Figure Description

[0056] Figure 1 A flowchart of a milk preference prediction method based on micro-expression recognition according to an embodiment of the present invention is shown;

[0057] Figure 2 A flowchart illustrating the preprocessing of an image sequence according to an embodiment of the present invention is shown;

[0058] Figure 3 A flowchart of facial spatiotemporal feature extraction according to an embodiment of the present invention is shown;

[0059] Figure 4 A flowchart illustrating the fusion of processed horizontal and vertical optical flow features according to an embodiment of the present invention is shown;

[0060] Figure 5 A structural diagram of a cross-modal timing module according to an embodiment of the present invention is shown;

[0061] Figure 6 A flowchart illustrating the milk preference prediction model implemented according to an embodiment of the present invention is shown.

[0062] Figure 7 A structural diagram of a milk preference prediction framework based on micro-expression recognition according to an embodiment of the present invention is shown.

[0063] Figure 8 A structural diagram of a milk preference prediction device based on micro-expression recognition according to an embodiment of the present invention is shown. Detailed Implementation

[0064] To enable those skilled in the art to better understand the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and specific examples, but this is not intended to limit the present invention. If there is no necessary sequential relationship between the various steps described herein, the order in which they are described as examples should not be considered a limitation. Those skilled in the art should understand that the order can be adjusted, as long as it does not disrupt the logical consistency between them and render the entire process impossible.

[0065] This invention provides a method for predicting milk preference based on micro-expression recognition, such as... Figure 1 As shown, the milk preference prediction method based on micro-expression recognition includes the following steps S100-S700.

[0066] S100, Decompose the facial video data of the target tasting milk into an image sequence.

[0067] It should be noted that the term "target" as used in this article refers to a target capable of tasting milk, including but not limited to humans and other animals. For example, the method proposed in this invention can determine the type of target based on the intended audience of the milk, such as consumers (humans) or animals. It can be used to evaluate consumers' preferences for milk as well as the preferences of some animals for milk.

[0068] In this embodiment, the facial video data is decomposed into an image sequence by frame-by-frame extraction. Existing software can be used for extraction, including but not limited to Adobe Premiere.

[0069] S200. The image sequence is preprocessed to obtain a preprocessed image sequence; wherein, the preprocessing includes face detection, alignment and cropping.

[0070] In some embodiments, such as Figure 2 As shown, preprocessing the image sequence to obtain a preprocessed image sequence includes the following steps:

[0071] S201. Use the Dlib library to detect facial features and multiple facial key points in each frame of the image sequence;

[0072] S202. Affine transformation is used to align multiple facial key points;

[0073] S203. Using the inner corner of the left eye as the base point of the facial features in each frame image, crop out the effective facial regions of the same size as preprocessed images, and combine multiple preprocessed images in the original order to obtain a preprocessed image sequence.

[0074] After the above steps, each preprocessed image in the resulting preprocessed image sequence contains a valid facial region. For example, when the target is a consumer, the valid facial region is the valid face region.

[0075] S300. Based on the preprocessed image sequence, a multi-scale optical flow method is used to capture facial micro-expression changes and extract facial spatiotemporal features. The facial spatiotemporal features include horizontal optical flow features and vertical optical flow features between each frame and the starting frame.

[0076] In some embodiments, such as Figure 3 As shown, based on the preprocessed image sequence, a multi-scale optical flow method is used to capture facial micro-expression changes and extract facial spatiotemporal features, including the following steps:

[0077] S301. Take the first frame of the preprocessed image sequence as the starting frame;

[0078] S302. The TV-L1 optical flow estimation algorithm is used to calculate the horizontal and vertical optical flow characteristics between each frame and the starting frame.

[0079] S303. The horizontal optical flow sequence and vertical optical flow sequence formed by the horizontal optical flow features and vertical optical flow features between each frame and the starting frame are used as facial spatiotemporal features.

[0080] S400. Construct a cross-modal timing module, use an encoder to process horizontal and vertical optical flow features respectively, and fuse the processed horizontal and vertical optical flow features to obtain fused features. The fused features can characterize global temporal dependencies.

[0081] In some embodiments, such as Figure 4 As shown, the encoder is used to process the horizontal and vertical optical flow features separately, and the processed horizontal and vertical optical flow features are fused to obtain the fused features. The steps include:

[0082] S401. Use the Vision Mamba encoder to perform bidirectional temporal modeling of horizontal optical flow features and vertical optical flow features respectively;

[0083] S402. Enhance the correlation between horizontal and vertical optical flow features by sharing the state transition matrix;

[0084] S403. The processed horizontal and vertical optical flow features are fused to obtain fused features, thereby establishing the temporal relationship between local and global features.

[0085] For example, such as Figure 5As shown, the cross-modal temporal module designed in this invention consists of two bidirectional Vision Mamba (VIM) encoders, used to process horizontal and vertical optical flow features respectively. The correlation between horizontal and vertical optical flow features is enhanced by sharing a state transition matrix. By integrating a bidirectional temporal state model (SSM) to extend Mamba, it allows modeling of forward and backward temporal features, enabling the module to capture the temporal dependencies of facial expressions. Specifically, the horizontal optical flow features at time t... The bidirectional SSM representation can be expressed as follows:

[0086]

[0087] in These represent the positive and negative states of the horizontal optical flow characteristics at time t, respectively. This represents the forward and backward outputs at time t. A, B, and C are the state transition matrices for the forward SSM, where A... b B b C b This is the state transition matrix for backward SSM. Horizontal optical flow features and vertical optical flow features share the state transition matrix A,A b .

[0088] S500: Construct and train a micro-expression recognition model, map basic emotion categories into three categories: negative, positive, and surprised, and use the trained micro-expression model to extract micro-expression features from preprocessed image sequences.

[0089] In some embodiments, such as Figure 6 As shown, the steps of the milk preference prediction model to predict milk preference include:

[0090] S501 employs a cross-modal timing module to process optical flow characteristics;

[0091] S502. The micro-expression features are fused with the processed optical flow features to obtain the first fused feature. The Vision Mamba encoder is used to perform temporal modeling on the first fused feature to obtain the temporal feature.

[0092] S503. Input the time series features into the fully connected layer, and output the consumer's preference rating for milk through the fully connected layer.

[0093] S600, based on the cross-modal time series module, constructs a milk preference prediction model, and combines micro-expression features and fusion features for prediction.

[0094] S700, Train the milk preference prediction model.

[0095] For example, when training the milk preference prediction model, leave-one-out consumer cross-validation is used to train the model, determine the optimal parameters, train using all video data, and save the best-performing weight model. The specific steps include:

[0096] S701. Use the data of each consumer as the test set and the data of the remaining consumers as the training set.

[0097] S702. Determine the optimal parameters of the model through cross-validation;

[0098] S703 uses all video data for training and saves the best-performing weight model.

[0099] The S800 preprocesses the video to be identified, inputs it into the trained milk preference prediction model, and finally outputs the target's preference rating for milk.

[0100] In step S800, the method of preprocessing the video to be identified can be achieved through steps S100 and S200, which will not be elaborated here.

[0101] The milk preference prediction method based on micro-expression recognition designed in this invention can be implemented through a milk preference prediction framework based on micro-expression recognition, the structure of which is as follows: Figure 7 As shown, the framework consists of a micro-expression recognition model and a milk preference prediction model, both of which are built based on cross-modal temporal modules. A bidirectional Vision Mamba encoder is used to fuse micro-expression emotional features with facial optical flow features of milk consumers, thereby enhancing the accuracy of milk preference prediction.

[0102] This invention also provides a milk preference prediction device based on micro-expression recognition, such as... Figure 8 As shown, the device includes:

[0103] The video data decomposition module 801 is configured to decompose the facial video data of the target tasting milk into an image sequence;

[0104] The image preprocessing module 802 is configured to preprocess the image sequence to obtain a preprocessed image sequence; wherein, the preprocessing includes face detection, alignment and cropping;

[0105] The facial spatiotemporal feature extraction module 803 is configured to capture facial micro-expression changes and extract facial spatiotemporal features based on the preprocessed image sequence using a multi-scale optical flow method. The facial spatiotemporal features include horizontal optical flow features and vertical optical flow features between each frame and the starting frame.

[0106] The feature fusion module 804 is configured to construct a cross-modal temporal module, which uses an encoder to process horizontal and vertical optical flow features respectively, and then fuses the processed horizontal and vertical optical flow features to obtain fused features, which can characterize global temporal dependencies.

[0107] The micro-expression feature extraction module 805 is configured to build and train a micro-expression recognition model, map basic emotion categories into three categories: negative, positive, and surprised, and use the trained micro-expression model to extract micro-expression features from preprocessed image sequences.

[0108] The prediction model building module 806 is configured to build a milk preference prediction model based on the cross-modal time series module, and combine micro-expression features and fusion features for prediction;

[0109] The prediction model training module 807 is configured to train the milk preference prediction model;

[0110] The preference prediction module 808 is configured to preprocess the video to be identified, input it into the trained milk preference prediction model, and finally output the target's preference rating for milk.

[0111] In some embodiments, the image preprocessing module is further configured to:

[0112] The Dlib library was used to detect facial features and multiple facial key points in each frame of the image sequence;

[0113] Affine transformation was used to align multiple facial key points;

[0114] Using the inner corner of the left eye as the base point, effective facial regions of consistent size are cropped out as preprocessed images. Multiple preprocessed images are then combined in their original order to obtain a preprocessed image sequence.

[0115] In some embodiments, the facial spatiotemporal feature extraction module is further configured to:

[0116] The first frame of the preprocessed image sequence is used as the starting frame;

[0117] The TV-L1 optical flow estimation algorithm is used to calculate the horizontal and vertical optical flow characteristics between each frame and the starting frame;

[0118] The horizontal optical flow sequence and vertical optical flow sequence formed by the horizontal optical flow features between each frame and the starting frame are used as facial spatiotemporal features.

[0119] It should be noted that the structures of the various milk preference prediction devices based on micro-expression recognition described in this embodiment belong to the same technical concept as the previously described milk preference prediction method based on micro-expression recognition, and achieve the same beneficial effects through the same principle, which will not be repeated here.

[0120] This invention also provides a readable storage medium storing one or more programs that can be executed by one or more processors to implement the methods described in any of the above embodiments.

[0121] The above description is intended to be illustrative and not restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. Other embodiments can be used by those skilled in the art when reading the above description. Furthermore, in the above detailed description, various features may be grouped together to simplify the invention. This should not be construed as an intention that a feature of an unclaimed invention is necessary for any claim. Rather, the subject matter of the invention may be less than all the features of a particular embodiment of the invention. Thus, the following claims are incorporated herein by reference as examples or embodiments, wherein each claim is an independent, separate embodiment, and these embodiments are contemplated to be combined with each other in various combinations or arrangements. The scope of the invention should be determined by reference to the appended claims and the full scope of their equivalents.

Claims

1. A method for predicting milk preference based on micro-expression recognition, characterized in that, The method comprises: decomposing facial video data of a target when tasting milk into an image sequence; preprocessing the image sequence to obtain a preprocessed image sequence; wherein the preprocessing comprises face detection, alignment and cropping; based on the preprocessed image sequence, using a multi-scale optical flow method to capture facial micro-expression changes and extract facial spatiotemporal features, wherein the facial spatiotemporal features comprise horizontal and vertical optical flow features between each frame and the starting frame; constructing a cross-modal timing module, using an encoder to process the horizontal and vertical optical flow features respectively, and fusing the processed horizontal and vertical optical flow features to obtain fused features, wherein the fused features can represent global timing dependencies; constructing and training a micro-expression recognition model, mapping basic emotional categories to three categories of negative, positive and surprise, and using the trained micro-expression model to extract micro-expression features from the preprocessed image sequence; based on the cross-modal timing module, constructing a milk preference prediction model, combining the micro-expression features and the fused features for prediction; training the milk preference prediction model; preprocessing a video to be recognized, inputting the trained milk preference prediction model, and finally outputting a preference score of the target for milk; the steps of the milk preference prediction model for milk preference prediction comprise: using the cross-modal timing module to process the optical flow features; fusing the micro-expression features with the processed optical flow features to obtain first fused features, using a Vision Mamba encoder to perform timing modeling on the first fused features to obtain timing features; inputting the timing features into a fully connected layer and outputting a preference score of the consumer for milk through the fully connected layer; training the milk preference prediction model, comprising: using video data of one target as a test set and video data of the remaining targets as a training set; determining the best parameter combination of the model through cross-validation, wherein the best parameter combination comprises multiple sets of best parameters; configuring the milk preference prediction model with different best parameters to obtain multiple prediction models, training the multiple prediction models using all video data, and saving the best-performing prediction model as the trained milk preference prediction model. 2.The milk preference prediction method based on micro-expression recognition according to claim 1, characterized in that, The preprocessing of the image sequence to obtain a preprocessed image sequence comprises: detecting the facial features and multiple facial key points of each frame of the image sequence using the Dlib library; aligning the multiple facial key points using affine transformation; cropping a consistent effective facial region from the left inner corner of the eye of the facial features in each frame of the image as a preprocessed image, and combining multiple preprocessed images in the original order to obtain a preprocessed image sequence. 3.The micro-expression recognition based milk preference prediction method according to claim 1, characterized in that, Based on the preprocessed image sequence, using a multi-scale optical flow method to capture facial micro-expression changes and extract facial spatiotemporal features, comprising: taking the first frame of the preprocessed image sequence as the starting frame; using a TV-L1 optical flow estimation algorithm to calculate the horizontal and vertical optical flow features between each frame and the starting frame; taking the horizontal and vertical optical flow sequences formed by the horizontal and vertical optical flow features between each frame and the starting frame as the facial spatiotemporal features. 4.The micro-expression recognition based milk preference prediction method of claim 3, wherein, The horizontal and vertical optical flow features are processed by an encoder respectively, and the processed horizontal and vertical optical flow features are fused to obtain fused features, including: The horizontal and vertical optical flow features are respectively bidirectionally time-series modeled using a Vision Mamba encoder; The correlation between the horizontal and vertical optical flow features is enhanced by sharing a state transition matrix; The processed horizontal and vertical optical flow features are fused to obtain fused features, thereby establishing the time-series relationship between the local and global features.

5. A micro-expression recognition based milk preference prediction device for implementing the method of any one of claims 1 to 4, characterized in that, The device comprises: a video data decomposition module configured to decompose facial video data of a target person tasting milk into an image sequence; an image preprocessing module configured to preprocess the image sequence to obtain a preprocessed image sequence; wherein the preprocessing includes face detection, alignment, and cropping; a facial spatiotemporal feature extraction module configured to capture facial micro-expression changes using a multi-scale optical flow method based on the preprocessed image sequence, and extract facial spatiotemporal features including horizontal and vertical optical flow features between each frame and the starting frame; a feature fusion module configured to construct a cross-modal time-series module, process horizontal and vertical optical flow features by an encoder respectively, and fuse the processed horizontal and vertical optical flow features to obtain fused features that can represent global time-series dependency; a micro-expression feature extraction module configured to construct and train a micro-expression recognition model, map basic emotion categories to three categories of negative, positive, and surprise, and extract micro-expression features from the preprocessed image sequence using the trained micro-expression model; a prediction model construction module configured to construct a milk preference prediction model based on the cross-modal time-series module, and predict by combining the micro-expression features and the fused features; a prediction model training module configured to train the milk preference prediction model; a preference prediction module configured to preprocess a video to be recognized, input the trained milk preference prediction model, and finally output a preference score of the target person for milk. 6.The micro-expression recognition based milk preference prediction device according to claim 5, characterized in that, The image preprocessing module is further configured to: detect facial features and a plurality of facial key points in each frame of the image sequence using a Dlib library; align the plurality of facial key points using an affine transformation; crop a uniform-size effective facial region from the left inner corner of the eye of the facial features in each frame of the image as a preprocessed image, and combine a plurality of preprocessed images in the original order to obtain a preprocessed image sequence. 7.The micro-expression recognition based milk preference prediction device according to claim 5, characterized in that, The facial spatiotemporal feature extraction module is further configured to: use the first frame of the preprocessed image sequence as the starting frame; calculate the horizontal and vertical optical flow features between each frame and the starting frame using a TV-L1 optical flow estimation algorithm; use the horizontal and vertical optical flow sequences formed by the horizontal and vertical optical flow features between each frame and the starting frame as the facial spatiotemporal features.

8. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, perform the method of any one of claims 1 to 4.

Images