Baijiu evaluation method and device based on multi-modal sensory data

Abstract

The invention relates to the technical field of baijiu evaluation, discloses a baijiu evaluation method and device based on multi-modal sensory data, and aims at solving the problem that an existing method is poor in accuracy. The scheme mainly comprises the steps that multi-modal sensory data, identity data and baijiu evaluation data of multiple evaluators for multiple baijiu samples are obtained, constructing a Baijiu evaluation data set according to the multi-modal sensory data, the identity data and the corresponding Baijiu evaluation data; training a neural network model according to the Baijiu evaluation data set to obtain a Baijiu evaluation model; inputting the multi-modal sensory data and the identity data of the evaluation personnel corresponding to the to-be-evaluated white spirit into the white spirit evaluation model to obtain a white spirit evaluation result of the evaluation personnel corresponding to the to-be-evaluated white spirit; and counting the Baijiu evaluation results of the to-be-evaluated Baijiu corresponding to the plurality of evaluators to obtain a final Baijiu evaluation result of the to-be-evaluated Baijiu. The Baijiu evaluation accuracy is improved, and the method is suitable for various Baijiu.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of liquor evaluation, and in particular to a liquor evaluation method and device based on multi-modal sensory data. BACKGROUND

[0002] As one of Chinese traditional drinks, the consumption market of liquor has been influenced by various factors for a long time, including brand recognition, cultural value, drinking scene, and consumer personal preference, etc. In the context of fierce market competition, accurately understanding the real feelings and evaluation of consumers on liquor products has become a key link to promote product research and development, market segmentation, and precise marketing.

[0003] Traditional liquor evaluation surveys mainly rely on subjective questionnaires, expert scoring or consumer face-to-face evaluation, etc. Such methods have many limitations: on the one hand, consumers are stripped of the real consumption scene in questionnaires and face-to-face evaluation, and often reconstruct the drinking experience through rational thinking, but the actual purchase decision is driven by immediate sensory stimulation and emotional state. The difference between the two neural mechanisms leads to bias between self-reporting and real behavior, and subjective evaluation is easily affected by personal emotions, environmental interference and other irrational factors, making it difficult to truly and stably reflect the real evaluation and purchase intention of consumers; on the other hand, single modal data (such as relying only on written questionnaires) can only capture part of the information at the conscious level, and it is difficult to fully capture and quantify the multi-dimensional sensory response of consumers in the actual drinking process, and cannot accurately reflect the real evaluation of consumers on liquor. SUMMARY

[0004] The present application aims to solve the problem of poor accuracy in existing liquor evaluation methods, and proposes a liquor evaluation method and device based on multi-modal sensory data.

[0005] The technical solution adopted by the present application to solve the above technical problems is: In a first aspect, the present application provides a liquor evaluation method based on multi-modal sensory data, which comprises: obtaining multi-modal sensory data, identity data and liquor evaluation data of a plurality of evaluation personnel on a plurality of liquor samples, constructing a liquor evaluation data set according to the multi-modal sensory data and identity data and their corresponding liquor evaluation data, the multi-modal sensory data including electroencephalogram data, facial expression data, skin electromyogram data and heart rate data at each drinking stage; training a neural network model according to the liquor evaluation data set to obtain a liquor evaluation model; inputting the multi-modal sensory data and identity data of the evaluation personnel corresponding to the liquor to be evaluated into the liquor evaluation model to obtain the liquor evaluation result of the evaluation personnel corresponding to the liquor to be evaluated; The white wine evaluation results of the to-be-evaluated white wine corresponding to multiple evaluators are counted to obtain the final white wine evaluation result of the to-be-evaluated white wine.

[0006] Further, the drinking stages include a resting stage, a smelling stage, a drinking stage and a post-drinking stage.

[0007] Further, the EEG data includes power spectrum data and power spectrum density data corresponding to each electrode during EEG acquisition, and image data of an EEG topographic map corresponding to the power spectrum data and the power spectrum density data. The facial expression data includes an angry time proportion, a contemptuous time proportion, an aversive time proportion, a fearful time proportion, a happy time proportion, a sad time proportion, a surprised time proportion, a positive emotional time proportion, a negative emotional time proportion, a neutral emotional time proportion, a closed-eye time proportion, an open-eye time proportion and a smiling time proportion during data acquisition. The skin electromyography data includes a peak electromyography frequency per minute and average electromyography peak amplitude data during data acquisition. The heart rate data include average heart rate data.

[0008] Further, the multiple evaluators include wine tasters and ordinary consumers of different genders and age groups, and the identity information of the evaluators includes gender data and age data.

[0009] Further, the white wine evaluation data is white wine purchase willingness evaluation data, white wine preference evaluation data or white wine aroma and taste evaluation data, and the data type of the white wine evaluation data is a standardized numerical value, a grading evaluation data or a label evaluation data.

[0010] Further, the white wine evaluation model includes a multi-modal feature extraction module and a multi-modal feature mixed prediction module; the multi-modal feature extraction module is used to extract multi-modal sensory data, and the multi-modal feature mixed prediction module is used to output the white wine evaluation result according to the spliced data of the multi-modal sensory data and the identity data.

[0011] Further, the training method of the white wine evaluation model includes: The white wine evaluation data set is divided into a training set and a data set, the neural network model is trained using the training set, the training effect of the neural network model is verified using the verification set, when the error between the prediction result of the neural network model corresponding to the verification set and the true white wine evaluation result is less than an error threshold, the training is completed, and the white wine evaluation model is obtained.

[0012] Further, in the training process of the white wine evaluation model, the loss function used is a mean square error loss function, a mean absolute error loss function or a Huber loss function.

[0013] Further, the method further comprises: adding the multi-modal sensory data and identity data of the to-be-evaluated baijiu corresponding to multiple evaluators and the corresponding baijiu evaluation results into a baijiu evaluation data set, and periodically updating the baijiu evaluation model according to the baijiu evaluation data set.

[0014] In a second aspect, the present application provides a baijiu evaluation device based on multi-modal sensory data, which is used to implement the baijiu evaluation method based on multi-modal sensory data as described in the first aspect, and the device comprises: An acquisition unit is configured to acquire multi-modal sensory data, identity data and baijiu evaluation data of multiple evaluators on multiple baijiu samples, and construct a baijiu evaluation data set according to the multi-modal sensory data and identity data and the corresponding baijiu evaluation data; A training unit is configured to train a neural network model according to the baijiu evaluation data set to obtain a baijiu evaluation model; An evaluation unit is configured to input the multi-modal sensory data and identity data of the to-be-evaluated baijiu corresponding to the evaluators into the baijiu evaluation model to obtain the baijiu evaluation results of the to-be-evaluated baijiu corresponding to the evaluators; A statistical unit is configured to statistically analyze the baijiu evaluation results of the to-be-evaluated baijiu corresponding to multiple evaluators to obtain the final baijiu evaluation results of the to-be-evaluated baijiu.

[0015] The baijiu evaluation method and device based on multi-modal sensory data provided by the present application make full use of the multi-modal sensory data of different evaluators, the multi-modal sensory data provides comprehensive sensory experience data, covers multiple drinking stages and multiple dimensions of sensory experience, can capture the multi-dimensional dynamic changes of the evaluators on baijiu, and improves the comprehensiveness and accuracy of baijiu evaluation; the neural network model is used to learn the complex nonlinear relationship between the multi-modal sensory data and identity data and the baijiu evaluation data, avoids the subjectivity of traditional sensory evaluation and the one-sidedness of a single data source, considers the evaluators with different identity data, considers the influence of identity characteristics on personal preferences, further improves the accuracy of baijiu evaluation, and makes the evaluation process more objective and reliable; the neural network model can process high-dimensional input in parallel, realizes end-to-end prediction, once the model is trained, the personal evaluation results can be output in real time by inputting the data of new evaluators, the evaluation efficiency is improved, and the time and labor costs are reduced. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 A flowchart of the baijiu evaluation method based on multi-modal sensory data provided for the embodiment is shown; Figure 2 A brain electrical topography diagram corresponding to the power spectral density data provided for the embodiment is shown; Figure 3This is a schematic diagram illustrating the change of the loss function during the training process of the first type of liquor evaluation model provided in the embodiment. Figure 4 A diagram illustrating the comparison between the prediction results and actual purchase intention scores of the test set provided for this embodiment; Figure 5 This is a schematic diagram illustrating the change of the loss function during the training process of the second type of liquor evaluation model provided in the embodiment; Figure 6 A schematic diagram comparing the predicted results and actual liking scores of the test set provided for the example; Figure 7 A schematic diagram illustrating the change of the loss function during the training process of the third type of liquor evaluation model provided in this embodiment; Figure 8 A schematic diagram comparing the predicted results of the test set provided for the example with the mellow aroma index in the actual aroma and taste scores. Figure 9 A schematic diagram of a baijiu (Chinese liquor) evaluation device based on multimodal sensory data is provided for an embodiment. Detailed Implementation

[0017] Currently, liquor evaluation surveys mainly rely on subjective questionnaires, expert scoring, or consumer interviews. These methods are easily affected by irrational factors such as personal emotions and environmental interference. Furthermore, single-modal data is difficult to fully capture and quantify consumers' multi-dimensional sensory reactions during actual drinking, making it difficult to reflect consumers' true evaluations in a real and stable manner.

[0018] Based on this, the technical solution of the present invention is proposed. In this invention, multimodal sensory data covers multiple physiological and behavioral levels. EEG data reflects cognitive and emotional states, facial expression data quantifies external emotional responses, electromyography (EMG) data monitors physiological arousal levels, and heart rate data reflects autonomic nervous system responses. This data is continuously collected throughout the entire drinking process, fully recording the dynamic process from baseline state to sensory stimulation, immediate reactions, and subsequent aftertaste. By integrating multimodal sensory data, the emotional, physiological, and cognitive responses of evaluators are integrated, providing multidimensional dynamic evidence for the quantitative assessment of the overall baijiu experience. A neural network model effectively fuses multimodal sensory data. Through objectively quantifying the physiological-behavioral expression chain of subjective experience, multimodal sensory data forms a causal relationship with the evaluation results. Furthermore, different modalities quantify sensory experiences from complementary dimensions, forming a three-dimensional correlation network. Identity data is also considered to accommodate individual differences. The neural network model learns the complex nonlinear relationships between multimodal sensory data, identity data, and baijiu evaluation data. By fusing complementary signals, mining cross-modal collaborations, and calibrating individual differences, the complex nonlinear relationships are transformed into predictable mathematical mappings, thereby more accurately predicting baijiu evaluations and improving the reliability and accuracy of the evaluation.

[0019] The technical solutions in the embodiments will be described clearly and completely in conjunction with the drawings in the embodiments. Obviously, the described embodiments are only part of the embodiments of the present application, not all.

[0020] Figure 1 A flowchart of a liquor evaluation method based on multi-modal sensory data is shown. Please refer to Figure 1 The method comprises the following steps: Step 1, obtaining multi-modal sensory data, identity data and liquor evaluation data of a plurality of evaluators on a plurality of liquor samples, and constructing a liquor evaluation data set according to the multi-modal sensory data and identity data and the corresponding liquor evaluation data.

[0021] In the embodiment, the multi-modal sensory data includes electroencephalogram data, facial expression data, skin electromyogram data and heart rate data in each drinking stage, and the each drinking stage includes a resting stage, a smelling stage, a drinking stage and a post-drinking stage.

[0022] In the embodiment, the plurality of evaluators includes wine tasters and ordinary consumers of different genders and age groups, and the identity information of the evaluators includes gender data and age data. The gender data includes male and female, and is represented by a one-hot vector, for example, [0 1] represents male and [1 0] represents female. The age data is represented by an integer.

[0023] In actual application, 40 wine tasters and ordinary consumers aged from 23 to 50 can be recruited as evaluators, and 200 groups of data are collected to construct a liquor evaluation data set. Before the data collection starts, the evaluators are first introduced in detail about the entire process to ensure the stability of data collection.

[0024] In the embodiment, the electroencephalogram data includes power spectrum data and power spectral density data corresponding to each electrode during electroencephalogram collection, and image data of electroencephalogram topography corresponding to the power spectrum data and the power spectral density data.

[0025] In actual application, the electroencephalogram data is collected by Enobio 8 instrument, and 8 channel electrodes are used in the embodiment, which are located at FPZ, FZ, F3, F4, CZ, PZ, F7 and F8, and the earlobe electrode is used as a reference electrode. Please refer to Figure 2In this embodiment, the EEG data of six bands, i.e., alpha, beta, delta, theta, low gamma and high gamma, are analyzed. Taking the power spectral density data of 8 electrode channels as an example, the EEG data is an 8x6 matrix, and when drawing the EEG topographic map, a separate EEG topographic map is drawn for each band, and each subject includes 6 EEG topographic maps (corresponding to 6 bands).

[0026] In this embodiment, the facial expression data includes the proportion of time of anger, contempt, disgust, fear, happiness, sadness, surprise, positive emotion, negative emotion, neutral emotion, closed eyes, wide open eyes and smiling during data collection. In actual application, the facial expression data is collected and sorted by using iMotions software combined with R language (iMotions software version 10.1).

[0027] In this embodiment, the skin myoelectricity data includes the peak value of myoelectricity per minute and the average myoelectricity peak value amplitude data during data collection. In actual application, the skin myoelectricity data is measured by using SHIMMER sensor, specifically measured by two electrodes placed on the fingers of the non-dominant hand of the evaluator.

[0028] In this embodiment, the heart rate data includes average heart rate data. In actual application, the skin myoelectricity data is measured by using SHIMMER sensor, specifically measured by 4 electrodes placed on the skin of the right arm, left arm, left leg and right leg of the evaluator.

[0029] In this embodiment, the liquor evaluation data is liquor purchase willingness evaluation data, liquor preference evaluation data or liquor aroma and taste evaluation data, and the data type of liquor evaluation data is standardized numerical value, grading evaluation data or label evaluation data.

[0030] The liquor purchase willingness evaluation data represents the degree of purchase willingness of the evaluator to the liquor; the liquor preference evaluation data represents the degree of preference of the evaluator to the liquor; and the liquor aroma and taste evaluation data represents the evaluation result of the evaluator to the aroma and taste of the liquor, including sweet aroma, mellow aroma, pungent taste, sweet taste, sour taste and bitter taste. If the data type of the liquor aroma and taste evaluation data is standardized numerical value, the higher the value, the stronger the aroma and taste, for example: the liquor aroma and taste evaluation data of a certain evaluator to a certain liquor sample is: sweet aroma: 5 points; mellow aroma: 7 points; pungent taste: 7 points; sweet taste: 5 points; sour taste: 5 points; bitter taste: 4 points.

[0031] In this embodiment, after obtaining the multi-modal sensory data, identity data and liquor evaluation data, the minimum-maximum value normalization method is used for normalization to eliminate the influence of dimension.

[0032] Step 2, training a neural network model according to the liquor evaluation data set to obtain a liquor evaluation model.

[0033] This step is used to learn the complex mapping of multi-modal sensory data and identity data to liquor evaluation data, eliminate redundant information, and extract cross-modal collaborative features. Specifically, the training method of the liquor evaluation model includes: The liquor evaluation data set is divided into a training set and a data set, the neural network model is trained using the training set, and the training effect of the neural network model is verified using the verification set. When the error between the prediction result of the neural network model corresponding to the verification set and the true liquor evaluation result is less than the error threshold, the training is completed, and the liquor evaluation model is obtained.

[0034] In actual application, the multi-modal sensory data and identity data of the evaluation personnel on the liquor sample and the liquor evaluation data of the evaluation personnel on the liquor sample are divided into a training set and a verification set. In this embodiment, the proportion of the training set and the verification set is 80% and 20% respectively. The neural network model is trained using the training set, and the training effect of the neural network model is verified using the verification set. When the error between the prediction result of the neural network model corresponding to the verification set and the true purchase intention evaluation result is less than the error threshold, the model is trained. The loss function used by the model during training is the mean square error loss function, the mean absolute error loss function or the Huber loss function.

[0035] In this embodiment, the mean square error loss function is used. When the liquor evaluation data is the liquor purchase intention evaluation data, the loss function change diagram during training is shown in Figure 3 ; the prediction result of the test set and the actual purchase intention score are compared in Figure 4 , the determination coefficient R² is about 0.778, proving that the model is reliable. When the liquor evaluation data is the liquor preference evaluation data, the loss function change diagram during training is shown in Figure 5 , the prediction result of the test set and the actual preference score are compared in Figure 6 , the determination coefficient R² is about 0.8, proving that the model is reliable. When the liquor evaluation data is the liquor aroma and taste evaluation data, the loss function change diagram during training is shown in Figure 7 , the prediction result of the test set and the actual aroma and taste score are compared in Figure 8 , the determination coefficient R² is about 0.78, proving that the model is reliable.

[0036] Step 3, input the multi-modal sensory data and identity data of the evaluation personnel corresponding to the to-be-evaluated liquor into the liquor evaluation model to obtain the liquor evaluation result of the evaluation personnel corresponding to the to-be-evaluated liquor.

[0037] In the embodiment, the liquor evaluation model comprises a multi-modal feature extraction module and a multi-modal feature mixing prediction module.

[0038] The multi-modal feature extraction module is configured to extract multi-modal sensory data, and comprises an EEG electrode feature extraction module, an EEG topographic map feature extraction module, and a facial expression feature extraction module.

[0039] The EEG electrode feature extraction module is one of MLP, LSTM, RNN, 1D CNN, FT-Transformer, etc., the input is the power spectrum data and the power spectrum density data corresponding to each electrode in the EEG data collected during EEG, and the output is the EEG electrode feature extraction result.

[0040] In the embodiment, the MLP neural network is adopted; the input data of the MLP neural network is the power spectrum data and the power spectrum density data, and the two kinds of data are 8x6 matrices, and the data correspond to four stages (rest stage, smelling stage, drinking stage, and post-drinking stage), therefore, in the input of the MLP network, the two matrix data are flattened and spliced, and thus 384x1 one-dimensional data (8x6x2x4) is obtained; the output data of the MLP neural network is set to 8x1 one-dimensional data in the embodiment.

[0041] The facial expression feature extraction module is one of MLP, LSTM, RNN, 1D CNN, FT-Transformer, etc., the input is the facial expression data, and the output is the facial expression feature extraction result.

[0042] In the embodiment, the MLP neural network is adopted; the input data of the MLP neural network is 204x1 one-dimensional data (54x4), and the output data of the MLP neural network is set to 8x1 one-dimensional data in the embodiment. The EEG topographic map feature extraction module is one of CNN, ResNet, VGG, EfficientNet, etc., the input is the image data of the EEG topographic map corresponding to the power spectrum data and the power spectrum density data in the EEG data, and the output is the EEG topographic map feature extraction result.

[0043] In this embodiment, a CNN neural network is adopted; in this embodiment, the electroencephalographic topographic map is first down-sampled into a 32x32 square image, and a 1-channel grayscale image is adopted for representation; the power spectrum data and the power spectrum density data each correspond to 6 bands, and both correspond to four stages, therefore, the input data of the CNN network is a 32x32x48 three-dimensional tensor; the output dimension of the CNN network is set to be a 4x4x1 three-dimensional tensor; then the 4x4x1 three-dimensional tensor is flattened into a 16x1 one-dimensional data.

[0044] The multi-modal feature mixed prediction module is used to output the Baijiu evaluation result according to the spliced data of the multi-modal sensory data and the identity data. That is, the input data is the spliced data (46x1 one-dimensional data) of the electroencephalographic electrode feature extraction result (8x1 one-dimensional data), the facial expression feature extraction result (8x1 one-dimensional data), the electroencephalographic topographic map feature extraction result (16x1 one-dimensional data), the skin electromyography data (8x1 one-dimensional data, 4 stages, 2 data for each stage), the heart rate data (4x1 one-dimensional data), and the identity data (2x1 one-dimensional data), and the output data is the Baijiu evaluation result.

[0045] When it is necessary to determine the Baijiu evaluation result of the Baijiu to be evaluated, first, the multi-modal sensory data and the identity data of the evaluation personnel corresponding to the Baijiu to be evaluated are obtained, and the obtaining method and step 1 are the same, then the minimum-maximum value normalization method is adopted for normalization, and then the normalized data is input into the Baijiu evaluation model, so that the Baijiu evaluation result of the evaluation personnel corresponding to the Baijiu to be evaluated is obtained, and finally the obtained Baijiu evaluation data is processed through the inverse process of the corresponding normalization, so that the Baijiu evaluation result is restored to the original data magnitude.

[0046] Step 4, the Baijiu evaluation results of the multiple evaluation personnel corresponding to the Baijiu to be evaluated are statistically processed, and the final Baijiu evaluation result of the Baijiu to be evaluated is obtained.

[0047] Specifically, after obtaining the Baijiu evaluation results of the Baijiu to be evaluated by different genders, different age groups, Baijiu sommeliers and ordinary consumers, statistical analysis is performed, such as weighted average, so that the final Baijiu evaluation result of the Baijiu to be evaluated is obtained. Thus, individual bias is eliminated, and a robust final evaluation is generated.

[0048] In this embodiment, the multi-modal sensory data and the identity data of the multiple evaluation personnel corresponding to the Baijiu to be evaluated and the corresponding Baijiu evaluation results are also added to the Baijiu evaluation data set, and the Baijiu evaluation model is periodically updated according to the Baijiu evaluation data set. Through the dynamic updating mechanism of the Baijiu evaluation model, the Baijiu evaluation model is periodically or real-timely updated based on the latest obtained data, so as to improve the prediction accuracy.

[0049] In summary, the liquor evaluation method based on multi-modal sensory data provided in the embodiment fully utilizes multi-modal sensory data of different evaluators, and the multi-modal sensory data provides comprehensive sensory experience data, covers multiple drinking stages and multiple dimensions of sensory experience, can capture multi-dimensional dynamic changes of the evaluators on the liquor, and improves the comprehensiveness and accuracy of the liquor evaluation; the neural network model is used to learn the complex nonlinear relationship between the multi-modal sensory data, the identity data and the liquor evaluation data, the subjectivity of the traditional sensory evaluation and the one-sidedness of the single data source are avoided, and the evaluators with different identity data are considered, the personal preference interference is isolated, the accuracy of the liquor evaluation is further improved, the evaluation process is more objective and reliable; the neural network model can process high-dimensional input in parallel, realizes end-to-end prediction, once the model training is completed, the personal evaluation result can be output in real time by inputting the data of the new evaluator, the evaluation efficiency is improved, and the time and labor cost is reduced.

[0050] Based on the above technical solution, the embodiment further provides a liquor evaluation device based on multi-modal sensory data, which is used to realize the liquor evaluation method based on multi-modal sensory data described in the embodiment, please refer to Figure 9 , the device comprises: An acquisition unit is configured to acquire multi-modal sensory data, identity data and liquor evaluation data of a plurality of evaluators on a plurality of liquor samples, and construct a liquor evaluation data set according to the multi-modal sensory data and the identity data and the corresponding liquor evaluation data; A training unit is configured to train a neural network model according to the liquor evaluation data set, and obtain a liquor evaluation model; An evaluation unit is configured to input multi-modal sensory data and identity data of an evaluator corresponding to a liquor to be evaluated into the liquor evaluation model, and obtain a liquor evaluation result of the evaluator corresponding to the liquor to be evaluated; A statistical unit is configured to statistically analyze the liquor evaluation results of a plurality of evaluators corresponding to the liquor to be evaluated, and obtain a final liquor evaluation result of the liquor to be evaluated.

[0051] It can be understood that the liquor evaluation device based on multi-modal sensory data described in the embodiment is a device for realizing the liquor evaluation method based on multi-modal sensory data described in the embodiment. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant part is described in the method part. Therefore, it will not be repeated here.

Claims

1. A method for evaluating baijiu (Chinese liquor) based on multimodal sensory data, characterized in that, The method includes: The process involves acquiring multimodal sensory data, identity data, and baijiu evaluation data from multiple evaluators on multiple baijiu samples. A baijiu evaluation dataset is constructed based on the multimodal sensory data, identity data, and corresponding baijiu evaluation data. The multimodal sensory data includes EEG data, facial expression data, skin electromyography data, and heart rate data at each drinking stage. A neural network model is trained based on the aforementioned liquor evaluation dataset to obtain a liquor evaluation model; The multimodal sensory data and identity data of the person responsible for evaluating the liquor to be evaluated are input into the liquor evaluation model to obtain the liquor evaluation results of the person responsible for evaluating the liquor to be evaluated. The evaluation results of the liquor to be evaluated from multiple evaluators are statistically analyzed to obtain the final evaluation result of the liquor to be evaluated.

2. The method for evaluating baijiu (Chinese liquor) based on multimodal sensory data according to claim 1, characterized in that, The drinking stages include the resting stage, the smelling stage, the drinking stage, and the post-drinking stage.

3. The method for evaluating baijiu (Chinese liquor) based on multimodal sensory data according to claim 2, characterized in that, The EEG data includes power spectrum data and power spectral density data corresponding to each electrode during EEG acquisition, as well as image data of the EEG topography corresponding to the power spectrum data and power spectral density data. The facial expression data includes the percentage of time spent in anger, contempt, disgust, fear, happiness, sadness, surprise, positive emotions, negative emotions, neutral emotions, closed eyes, wide-open eyes, and smiling during data collection. The skin electromyography data includes the number of peak electromyography times per minute and the average peak electromyography amplitude data at the time of data acquisition; The heart rate data includes average heart rate data.

4. The method for evaluating baijiu (Chinese liquor) based on multimodal sensory data according to claim 1, characterized in that, The evaluators included wine tasters and ordinary consumers of different genders and age groups. The evaluators' identity information included gender and age data.

5. The method for evaluating baijiu (Chinese liquor) based on multimodal sensory data according to claim 1, characterized in that, The baijiu evaluation data includes baijiu purchase intention evaluation data, baijiu liking evaluation data, or baijiu aroma and taste evaluation data. The data types of the baijiu evaluation data are standardized numerical values, graded evaluation data, or label evaluation data.

6. The method for evaluating baijiu (Chinese liquor) based on multimodal sensory data according to claim 1, characterized in that, The baijiu evaluation model includes a multimodal feature extraction module and a multimodal feature hybrid prediction module. The multimodal feature extraction module is used to extract multimodal sensory data, and the multimodal feature hybrid prediction module is used to output baijiu evaluation results based on the spliced ​​data of multimodal sensory data and identity data.

7. The method for evaluating baijiu (Chinese liquor) based on multimodal sensory data according to claim 6, characterized in that, The training method for the liquor evaluation model includes: The baijiu evaluation dataset is divided into a training set and a validation set. The training set is used to train a neural network model, and the validation set is used to verify the training effect of the neural network model. When the error between the prediction result of the neural network model corresponding to the validation set and the actual baijiu evaluation result is less than the error threshold, the training is completed and the baijiu evaluation model is obtained.

8. The method for evaluating baijiu (Chinese liquor) based on multimodal sensory data according to claim 7, characterized in that, During the training process of the baijiu evaluation model, the loss function used is the mean squared error loss function, the mean absolute error loss function, or the Huber loss function.

9. The method for evaluating baijiu (Chinese liquor) based on multimodal sensory data according to claim 1, characterized in that, The method further includes: adding the multimodal sensory data and identity data of multiple evaluators corresponding to the liquor to be evaluated and their corresponding liquor evaluation results to the liquor evaluation dataset, and periodically updating the liquor evaluation model based on the liquor evaluation dataset.

10. A baijiu (Chinese liquor) evaluation device based on multimodal sensory data, used to implement the baijiu evaluation method based on multimodal sensory data as described in any one of claims 1 to 9, the device comprising: The acquisition unit is used to acquire multimodal sensory data, identity data, and baijiu evaluation data from multiple evaluators on multiple baijiu samples, and to construct a baijiu evaluation dataset based on the multimodal sensory data, identity data, and their corresponding baijiu evaluation data. The training unit is used to train a neural network model based on the liquor evaluation dataset to obtain a liquor evaluation model. The evaluation unit is used to input the multimodal sensory data and identity data of the person being evaluated for the liquor to be evaluated into the liquor evaluation model to obtain the liquor evaluation results of the person being evaluated for the liquor to be evaluated. The statistical unit is used to statistically analyze the evaluation results of the liquor to be evaluated from multiple evaluators, and obtain the final evaluation result of the liquor to be evaluated.

Images