Smell recognition and emotion analysis system and method based on multi-modal physiological signals, terminal and medium

Abstract

The invention discloses a smell recognition and emotion analysis system and method based on a multi-modal physiological signal, a terminal and a medium, and relates to the technical field of sensory analysis, and the system comprises a multi-modal physiological signal acquisition device, a smell output device and a data processing control device. The multi-modal physiological signal acquisition equipment comprises an electroencephalogram signal acquisition device, a skin electric signal acquisition device and an electrocardiosignal acquisition device. While the smell output equipment outputs gas with smell, the multi-modal physiological signal acquisition equipment acquires multi-modal physiological signals. And the control module is connected with the multi-modal physiological signal acquisition equipment, analyzes and processes the acquired multi-modal physiological signals, and outputs the odor type and the emotional state information of the user. According to the method, the accuracy and the stability of odor-induced emotional state recognition are improved, joint decoding of odor and emotional response can be realized, and meanwhile, comprehensive and reliable odor sensory evaluation technical support can be provided for the food industry.

Description

Technical Field

[0001] The present invention relates to the field of sensory analysis technology, and in particular to an odor recognition and emotion analysis system, method, terminal and medium based on multimodal physiological signals. Background Art

[0002] Food odor is a key indicator for food sensory evaluation, quality control, and authenticity verification. Existing food odor assessment primarily relies on manual sensory evaluation and biomimetic sensing technologies such as electronic noses. Manual sensory evaluation relies on human perception and verbal expression, which is subject to high subjectivity and difficulty in quantification, making it difficult to meet the needs of large-scale industrial testing.

[0003] While existing machine perception technologies like electronic noses can quickly detect odor components, they are limited by the type, quantity, and sensitivity of sensors, resulting in insufficient ability to identify complex, weak, or mixed odors. Furthermore, they fail to reflect human emotional responses to odors. Consequently, these existing technologies struggle to provide high-precision, comprehensive, and standardized technical support for food odor identification, sensory quality evaluation, and personalized product development.

[0004] Therefore, the prior art still has defects. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to address the above-mentioned shortcomings of the prior art and provide a system, method, terminal and medium for odor recognition and emotion analysis based on multimodal physiological signals. The technical solution adopted by the present invention is as follows: In a first aspect, the present invention provides an odor recognition and emotion analysis system based on multimodal physiological signals, wherein the system comprises: The system comprises: Multimodal physiological signal acquisition equipment, comprising an electroencephalogram (EEG) signal acquisition device, a skin electrical signal acquisition device, and an electrocardiogram (ECG) signal acquisition device; An odor output device, comprising an odor storage device and an odor inducing device for controlling the odor storage device to output odor, wherein the odor output device outputs odorous gas while the multimodal physiological signal acquisition device acquires multimodal physiological signals; A data processing and control device is connected to the multimodal physiological signal acquisition device and the odor output device respectively, and analyzes and processes the collected multimodal physiological signals, and outputs the odor type and the user's emotional state information.

[0006] In one implementation, the EEG signal acquisition device includes: a helmet, a first control module disposed on the helmet, and a first electrode connected to the first control module, wherein the first electrode is disposed inside the helmet for acquiring EEG signals; The skin electrical signal acquisition device includes a wristband, a second control module provided on the wristband, and a second electrode connected to the second control module, wherein the second electrode is used to acquire skin electrical signals; The electrocardiogram signal acquisition device includes a waist belt, a third control module arranged on the waist belt, and a third electrode connected to the third control module, wherein the third electrode is used to acquire electrocardiogram signals; The first regulation module, the second regulation module and the third regulation module are all connected to the data processing and control device.

[0007] In one implementation, the odor storage device includes a plurality of odor bottles for storing gases with different odors, and the bottom of each odor bottle is connected to the odor inducing device through a pipe.

[0008] In one implementation, the odor-inducing device stores odorless ammonia, which enters the bottom of the odor bottle through the pipe, forming a disturbed airflow, which drives the gas in the odor bottle to volatilize.

[0009] In one implementation, the scent output device further includes a flexible conduit, which is in communication with each scent bottle and is configured to direct the scented gas into the user's nasal cavity.

[0010] In one implementation, the odor output device further includes a heating module and a humidity control module, which are disposed between the odor storage device and the flexible conduit to control the temperature and humidity of the gas entering the user's nasal cavity.

[0011] In a second aspect, an embodiment of the present invention further provides a method for odor recognition and emotion analysis based on multimodal physiological signals, wherein the method is applied to the odor recognition and emotion analysis system based on multimodal physiological signals described in the above solution, and the method includes: When the odor output device outputs odorous gas to the user's nasal cavity, a multimodal physiological signal is obtained, wherein the multimodal physiological signal includes: an electroencephalogram signal, a skin electrode signal, and an electrocardiogram signal; Preprocessing the multimodal physiological signal, and performing feature extraction based on the preprocessed multimodal physiological signal to obtain target features; Based on the target feature, odor type information and emotional state information are determined, and an odor perception dimensional space model is established. The odor perception dimensional space model is used to graphically present the odor type information, the emotional state information and the target feature.

[0012] In one implementation, the preprocessing includes: bandpass filtering, power frequency interference removal, physiological artifact removal, and bad channel repair.

[0013] In one implementation, feature extraction is performed based on the preprocessed multimodal physiological signal to obtain target features, including: Extracting the time domain features, frequency domain features, wavelet features and entropy features of the pre-processed multimodal physiological signals to obtain multidimensional features; Based on the multi-dimensional features, target features are determined.

[0014] In one implementation, based on the target features, odor type information and emotional state information are determined, and an odor perception dimensional space model is established, including: Obtaining a preset classification model, classifying the target feature based on the classification model, and outputting odor type information and emotional state information corresponding to the target feature, wherein the classification model includes an ensemble learning framework or a manifold classifier, and the emotional state information includes pleasure and arousal; Mapping the odor type information, the emotional state information, and the target feature to establish an association relationship between the odor, emotion, and physiological signal; Based on the association relationship, an odor perception dimensional space model is established.

[0015] In a third aspect, an embodiment of the present invention further provides a terminal, wherein the terminal includes a memory, a processor, and an odor recognition and emotion analysis program based on multimodal physiological signals stored in the memory and runnable on the processor. When the processor executes the odor recognition and emotion analysis program based on multimodal physiological signals, the steps of the odor recognition and emotion analysis method based on multimodal physiological signals of any one of the above-mentioned schemes are implemented.

[0016] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein a program for odor recognition and emotion analysis based on multimodal physiological signals is stored on the computer-readable storage medium, and the program for odor recognition and emotion analysis based on multimodal physiological signals implements the steps of the method for odor recognition and emotion analysis based on multimodal physiological signals described in any one of the above-mentioned schemes on the computer-readable storage medium.

[0017] Beneficial effects: Compared with the prior art, the present invention provides an odor recognition and emotion analysis system based on multimodal physiological signals, which includes: a multimodal physiological signal acquisition device, an odor output device, and a data processing and control device. Specifically, the multimodal physiological signal acquisition device includes an electroencephalogram (EEG) signal acquisition device, a skin electrical signal acquisition device, and an electrocardiogram (ECG) signal acquisition device; the odor output device includes an odor storage device and an odor inducing device for controlling the odor storage device to output odor; while the odor output device outputs odorous gas, the multimodal physiological signal acquisition device acquires multimodal physiological signals; the data processing and control device is connected to the multimodal physiological signal acquisition device and the odor output device, respectively, and analyzes and processes the acquired multimodal physiological signals, outputting information on the odor type and the user's emotional state. The present invention can realize comprehensive analysis of odor type information and emotional state information based on multimodal physiological signals, solving the problem of inaccurate results caused by subjective judgment of odor and emotion in the existing technology, and improving the accuracy and stability of identifying odor-induced emotional states, which is conducive to the joint decoding of odor and emotional response, and also helps to provide comprehensive and reliable technical support for odor sensory evaluation for the food industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a block diagram of the principles of the odor recognition and emotion analysis system based on multimodal physiological signals provided by an embodiment of the present invention.

[0019] Figure 2 A schematic diagram of the structure of an EEG signal acquisition device in an odor recognition and emotion analysis system based on multimodal physiological signals provided by an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of the structure of the skin electrical signal acquisition device in the odor recognition and emotion analysis system based on multimodal physiological signals provided by an embodiment of the present invention.

[0021] Figure 4 A schematic diagram of the structure of a central electrical signal acquisition device in an odor recognition and emotion analysis system based on multimodal physiological signals provided by an embodiment of the present invention.

[0022] Figure 5 A schematic diagram of the structure of an odor output device in an odor recognition and emotion analysis system based on multimodal physiological signals provided by an embodiment of the present invention.

[0023] Figure 6 This is a flowchart of a preferred embodiment of the method for odor recognition and emotion analysis based on multimodal physiological signals provided by the embodiment of the present invention.

[0024] Figure 7This is a functional block diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solution and effect of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0026] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents, operations, or steps, nor must they be executed in the order described. For example, some operations or steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0027] It should be understood that the terms used in this specification are for the purpose of describing particular embodiments only and are not intended to limit the present invention. As used in the specification and appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should be understood that, to facilitate a clear description of the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, terms such as "first" and "second" are used to distinguish between identical or similar items with substantially the same functions and effects. For example, the first control information and the second control information are merely used to distinguish different control information and do not limit their order. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different. It will also be understood that the term "and / or" used in the present description and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0028] In order to solve the problems of the existing technology, this embodiment provides an odor recognition and emotion analysis system based on multimodal physiological signals. The system based on this embodiment can solve the problem of inaccurate results caused by subjective judgment of odor and emotion in the existing technology, and improve the accuracy and stability of the recognition of odor-induced emotional states, which is conducive to the joint decoding of odor and emotional response, and also helps to provide comprehensive and reliable technical support for odor sensory evaluation for the food industry. In specific applications, such as Figure 1As shown, the system includes a multimodal physiological signal acquisition device, an odor output device, and a data processing and control device. The multimodal physiological signal acquisition device is used to acquire multiple types of physiological signals. For example, the multimodal physiological signal acquisition device includes an EEG signal acquisition device, a galvanic skin signal acquisition device, and an ECG signal acquisition device, respectively for acquiring EEG signals, galvanic skin signals, and ECG signals. The odor output device is used to output odor-containing gas into the user's nasal cavity. The data processing and control device of this embodiment is connected to the multimodal physiological signal acquisition device and the odor output device, respectively, to process and analyze the acquired physiological signals and determine the odor type of the gas output by the odor output device and the user's emotional state. In practical applications, the data processing and control device can be an intelligent terminal that controls parameters such as the concentration and flow rate of the gas output by the odor output device. To analyze the correlation between odor type and the user's emotional state, this embodiment can control the multimodal physiological signal acquisition device to acquire multimodal physiological signals while the odor output device is outputting odor-containing gas. This allows the odor output device and the multimodal physiological signal acquisition device to operate synchronously, ensuring accurate analysis.

[0029] Specifically, combined Figure 2 As shown in , the EEG signal acquisition device of this embodiment is in the form of a helmet, which includes a helmet 11, a first control module 12 provided on the helmet 11, and a first electrode 13 connected to the first control module 12. A plurality of first electrodes 13 can be provided, which are provided on the inside of the helmet for collecting EEG signals. The first control module 12 is connected to a data processing and control device. In actual application, the user can wear the EEG signal acquisition device on the user's head so that the scalp contacts the first electrode 13, thereby realizing the collection of EEG signals. The sampling rate of the EEG signal of this embodiment is 256 Hz, and the electrode impedance is ≤5 kΩ. It can collect the changes of EEG signals before, during, and after odor stimulation in real time. Combined with Figure 3 As shown in , the skin electrical signal acquisition device of this embodiment is set in the form of a wristwatch, and the skin electrical signal acquisition device includes a wristband 111, a second control module 112 provided on the wristband, and a second electrode 113 connected to the second control module 112. A plurality of second electrodes 113 can be provided. The second electrode 113 is used to collect skin electrical signals. The second control module 112 is also connected to the data processing and control device. In actual application, Figure 3 As shown in , the skin electrical signal acquisition device can be worn on the user's wrist, the second electrode 113 and the second control module 112 are connected by a conductive wire, and the second electrode 113 can be clamped on the user's fingertips to collect the user's skin electrical signals. Figure 4As shown in , the ECG signal acquisition device of this embodiment is configured in the form of a waist belt. The ECG signal acquisition device includes a waist belt 10, a third control module 20 provided on the waist belt 10, and a third electrode 30 connected to the third control module 20, and the third electrode 30 is used to acquire ECG signals. The third control module 20 is also connected to the data processing and control device. In actual application, the user can wear the ECG signal acquisition device around the waist. There are multiple third electrodes 30, and they are all connected to the third control module 20 through a conductive wire. The third electrode 30 is then attached to the user's chest near the heart to collect ECG signals. In this embodiment, the sampling frequency of the ECG signal acquisition device is generally set to ≥500Hz to ensure detection accuracy.

[0030] like Figure 5 As shown in FIG, the odor output device in this embodiment includes an odor storage device 310 and an odor induction device 320 for controlling the output of odor from the odor storage device 310. The odor storage device 310 includes several odor bottles 311 for storing gases with different scents, such as floral, herbal, woody, and citrus scents. The bottom of each odor bottle 311 is connected to the odor induction device 320 via a pipe. The odor output device also includes a flexible conduit 330, which is connected to each odor bottle 311 and is used to direct the scented gas into the user's nasal cavity. In this embodiment, the odor induction device 320 stores odorless ammonia gas. A driving device in the odor induction device 320 drives the odorless ammonia gas through the pipe and into the bottom of the odor bottle 311, creating a turbulent airflow that volatilizes the gas in the odor bottle 311. The odor-bearing gas is then directed into the user's nasal cavity via the flexible conduit 330. In one implementation, the scent output device of this embodiment further includes a heating module and a humidity control module, which can be positioned between the scent storage device and the flexible conduit. These modules are used to control the temperature and humidity of the gas entering the user's nasal cavity, ensuring that the gas maintains a constant temperature and humidity, improving the physiological adaptability of the scent stimulation and making the output gas temperature and humidity closer to natural breathing conditions. This reduces non-odor-related interference factors and further ensures the purity and comfort of the olfactory stimulation, as well as the consistency of the neural response. Furthermore, the scent bottle 311 of this embodiment can also be replaced with an electronically controlled sprayer or a microfluidic scent generator to achieve higher-precision scent stimulation.

[0031] Of course, the odor-inducing device 320 of the odor output device of this embodiment can also precisely control and release the gas output by the odor storage device 310. For example, odorless ammonia is introduced into the bottom of the odor bottle 311 at a constant flow rate (e.g., 1.5 L / min), simulating a "pseudo-boiling" state and continuously pushing the odor bottle 311 to volatilize the gas. After passing through the heating module and the humidity control module, the gas is introduced into the user's nasal cavity at a specific concentration (10%, 30%, 50%, 70%, 90%) and for a fixed duration (e.g., 4 seconds). This helps ensure the consistency and repeatability of the odor stimulation process and the validity of the experimental results. In addition, this embodiment can also provide a three-way solenoid valve at the end of the flexible catheter to precisely control the gas entering the user's nasal cavity. When the three-way solenoid valve is opened, the multimodal physiological signal acquisition device is simultaneously activated to collect multimodal physiological signals and record key time points to ensure the synchronous collection of multimodal physiological signals during the odor stimulation process. Furthermore, the EEG signal acquisition device of this embodiment features 32 channels, and the distal end of the flexible catheter is designed in tandem with the spatial structure of the device, enabling synchronous alignment of the stimulation pathway with the device's channels, enhancing the accuracy and spatial traceability of neural signal timing. The odor output device of this embodiment also features a roller at its base, allowing for movement and adaptability to a wider range of application scenarios.

[0032] Based on the above-mentioned odor recognition and emotion analysis system based on multimodal physiological signals, the present invention also provides an odor recognition and emotion analysis method based on multimodal physiological signals. The odor recognition and emotion analysis method based on multimodal physiological signals of this embodiment can be applied to terminals, which can be intelligent product terminals such as computers, smart TVs and mobile phones. In practical applications, such as Figure 6 As shown in , the odor recognition and emotion analysis method based on multimodal physiological signals includes the following steps: Step S100: When the odor output device outputs odorous gas to the user's nasal cavity, a multimodal physiological signal is obtained, where the multimodal physiological signal includes: an electroencephalogram signal, an electrical skin signal, and an electrocardiogram signal.

[0033] The multimodal physiological signal acquisition device of this embodiment includes an EEG signal acquisition device, a skin electrical signal acquisition device, and an ECG signal acquisition device, which are used to acquire EEG signals, skin electrical signals, and ECG signals, respectively. When the odor output device outputs odorous gas to the user's nasal cavity, the EEG signal acquisition device, the skin electrical signal acquisition device, and the ECG signal acquisition device are started to acquire the user's EEG signals, skin electrical signals, and ECG signals, so as to analyze the impact of odor stimulation on EEG signals, skin electrical signals, and ECG signals in subsequent steps, and then analyze the odor type information and emotional state information. This embodiment uniformly triggers the acquisition of EEG signals, skin electrical signals, and ECG signals, automatically and synchronously marks the acquisition period of each physiological signal, and aligns the timestamps of signal acquisition to ensure the precise correspondence of multimodal physiological signals under odor stimulation.

[0034] Step S200: preprocessing the multimodal physiological signal, and extracting features based on the preprocessed multimodal physiological signal to obtain target features.

[0035] After collecting the multimodal physiological signals within a preset time period, this embodiment can preprocess the multimodal physiological signals. The preprocessing includes: bandpass filtering, power frequency interference removal, physiological artifact removal, and bad channel repair. For example, a 1-70Hz bandpass filter is selected for filtering, and a 50Hz notch filter is selected to remove power frequency interference. Then, the time domain features, frequency domain features, wavelet features, and entropy features of the preprocessed multimodal physiological signals are extracted to obtain multidimensional features. That is to say, the time domain features, frequency domain features, wavelet features, and entropy features are extracted from the preprocessed EEG signals, skin electrical signals, and electrocardiogram signals, respectively. The time domain features in this embodiment include mean and variance, the frequency domain features include power spectral density, the wavelet features include wavelet coefficients and wavelet energy moments, and the entropy features include power spectral entropy, fuzzy entropy, and singular value decomposition entropy. This embodiment effectively captures the multidimensional information of neural responses after odor stimulation by extracting multidimensional features such as time domain features, frequency domain features, wavelet features, and entropy features, achieves efficient recognition of complex odors and subtle odor differences, and supports high-precision classification of different food odor types.

[0036] Specifically, the above mean is expressed as: (1) In formula (1), is the total number of sampling points of the physiological signal, For the The signal value of the sampling point, Indicates the sum of the signal values ​​of all collected physiological signals.

[0037] The above variance is expressed as: (2) In formula (2), is the mean of the signal; Indicates the The square of the deviation of a physiological signal from the mean is used to eliminate the offset of positive and negative deviations.

[0038] The above power spectral density is expressed as: (3) In formula (3), Represents the frequency of physiological signals, represents the observation length of the physiological signal, Represents the spectrum of a physiological signal.

[0039] The wavelet feature in this embodiment is the wavelet energy moment, which is expressed as: (4) In formula (4), It means that the physiological signal is decomposed by wavelet The coefficients of the subbands, is the coefficient index, is the number of layers for wavelet decomposition of the original physiological signal.

[0040] The entropy feature in this embodiment is fuzzy entropy, which is expressed as: (5) In formula (5), fuzzy entropy is used to calculate the similarity of patterns in physiological signals with pattern length. Change to The rate of change when , as well as There are three core parameters: and There are two key functions, Indicates the number of similarities of continuous physiological signals analyzed, usually an integer. To control the looseness of similarity, it is usually taken as a multiple of the standard deviation of the physiological signal, or a fixed value can be directly taken, for example, is 0.5, is the total number of original physiological signals, usually required . Used to indicate the presence of continuous The physiological signals satisfy the fuzzy similarity ratio. Indicates that there is a continuous The physiological signals satisfy the fuzzy similarity ratio.

[0041] After calculating the aforementioned multidimensional features, this embodiment can determine the target feature based on these multidimensional features. Specifically, the multidimensional features in this embodiment are extracted from three signals: EEG, galvanic skin, and electrocardiogram (ECG). Therefore, when determining the target feature, this embodiment can extract the optimal feature from the multidimensional features of each physiological signal and then use the optimal feature of each physiological signal as the target feature. Specifically, this embodiment can use the mRMR algorithm, the Relief-F algorithm, or the CD-FS-ILFS algorithm to extract the optimal feature from the multidimensional features of each physiological signal. The mRMR (Minimum Redundancy Maximum Relevance) algorithm aims to select features that are highly correlated with the target variable while minimizing redundancy between features. The Relief-F algorithm calculates the weighted distance difference between features in neighboring samples and can be used to assess feature importance. The CD-FS-ILFS (Class-Discriminative Feature Selection + Infinite Latent Feature Selection) algorithm combines discriminative scoring with unsupervised graph modeling and is suitable for high-dimensional feature screening. Based on the above algorithm, the optimal features of each physiological signal can be screened out separately, thus obtaining the optimal features of the EEG signal, the optimal features of the skin electrode signal, and the optimal features of the ECG signal. Finally, the optimal features of the EEG signal, the optimal features of the skin electrode signal, and the optimal features of the ECG signal are used as target features.

[0042] In another implementation, this embodiment can also filter out target features from the multidimensional features based on the degree of influence of the features on odor recognition and emotion analysis. Specifically, this embodiment first calculates the influence score corresponding to the time domain features, frequency domain features, wavelet features, and entropy features of each preprocessed physiological signal, that is, calculates the influence score corresponding to the time domain features, frequency domain features, wavelet features, and entropy features of the EEG signal, calculates the influence score corresponding to the time domain features, frequency domain features, wavelet features, and entropy features of the skin electrode signal, and calculates the influence score corresponding to the time domain features, frequency domain features, wavelet features, and entropy features of the ECG signal. Then, based on the influence score, the feature with the highest influence score in each preprocessed physiological signal is determined, and the feature with the highest influence score in each preprocessed physiological signal is used as the optimal feature of each preprocessed physiological signal. Finally, based on the optimal feature of each preprocessed physiological signal, the target feature is obtained.

[0043] Step S300: Based on the target feature, determine the odor type information and the emotional state information, and establish an odor perception dimensional space model, wherein the odor perception dimensional space model is used to graphically present the odor type information, the emotional state information, and the target feature.

[0044] After obtaining the target feature, this embodiment can obtain a preset classification model, classify the target feature based on the classification model, and output odor type information and emotional state information corresponding to the target feature. In this embodiment, the classification model includes an ensemble learning framework or a manifold classifier. The ensemble learning framework can be a random forest network, a long short-term memory network, or a particle swarm optimization network. The manifold classifier can be an MDRM (Minimum Distance to Riemannian Mean) classifier or a GFMDRM (Geometric Functional Minimum Distance to Riemannian Mean) classifier, which classifies odor type and emotional state. In this embodiment, emotional state information includes pleasantness and arousal. Pleasure is an indicator that measures the degree of positivity or negativity of an emotional experience, reflecting whether the emotion brings a person pleasure or unpleasantness. Arousal measures the level of physiological and psychological activation of the emotional experience, namely the degree of excitement or tension caused by the emotion, reflecting the intensity of physical and psychological activation.

[0045] The classification model of this embodiment is pre-configured and can integrate odor recognition functionality with a binary emotion classification model. This classification model can reflect the mapping relationship between physiological signal features and odor types, as well as arousal and pleasantness. That is, when the target features extracted from the physiological signals are input into the classification model, the classification model can automatically output the corresponding odor type information and pleasantness and arousal values ​​based on the input target features, thereby obtaining emotional state information. If the pleasantness value is high and the arousal value is low, the emotional state information is determined to be a relaxed and tranquil state. If the pleasantness value is low and the arousal value is high, the emotional state information is determined to be an anxious and alert state. For example, if the extracted target feature is the variance of the EEG signal, if the variance of the EEG signal is low or continuously decreasing, the classification model can automatically output the odor type information as acidic or bitter, and the pleasantness value is low and the arousal value is high, then the emotional state information can be determined to be a highly alert state. This embodiment utilizes an ensemble learning framework or manifold classifier to identify odor type and emotional state information, improving model stability and cross-subject generalization, enabling high-precision classification of different odor types and emotional states, making it suitable for large-scale industrial applications. The classification model of this embodiment utilizes an ensemble learning framework, which helps fully utilize the spatial structure of the covariance matrix, improves the accuracy of odor type and emotion classification, and enhances the adaptability of the classification model. Of course, in other application scenarios, this embodiment can also directly decode raw physiological signals in terms of data processing by introducing an end-to-end deep learning architecture (such as a convolutional neural network).

[0046] Furthermore, in other implementations, after determining the target feature, this embodiment can also obtain weight information for the optimal feature of each preprocessed physiological signal, as the target feature is the optimal feature of each preprocessed physiological signal. Then, based on the optimal feature of each preprocessed physiological signal and the corresponding weight information, a target feature weight value is determined. This target feature weight value can be used to comprehensively reflect the user's overall level of odor perception and emotional perception using their EEG, galvano-dermal, and ECG signals. Therefore, based on the target feature weight value, odor type information and emotional state information can be determined. Specifically, if the target feature weight value is higher than a preset value, it indicates that the user prefers the smell and is feeling relaxed and happy. Therefore, the odor type information is determined to be sweet or fruity, and the emotional state information is determined to be relaxed and happy. If the target feature weight value is lower than the preset value, it indicates that the user is feeling aversion to the smell and is feeling stressed. Therefore, the odor type information is determined to be sour or bitter, and the emotional state information is determined to be highly alert. It should be noted that, in this embodiment, the determination of odor type information and emotional state information based on physiological signals is based on the perception of most users on odors, excluding individual preferences for certain odors.

[0047] After obtaining the above-mentioned target features, this embodiment can also use the K-PCA (Kernel Principal Component Analysis) algorithm to map the target features to a high-dimensional feature space, making them linearly separable in the high-dimensional space, and then reduce the dimensionality to a three-dimensional space through the principal component analysis method. Next, the odor type information and the emotional state information are mapped to the target features to establish a correlation between odor, emotion, and physiological signals. Based on the correlation, a three-dimensional space model is formed. This three-dimensional space model is the odor perception dimensional space model. The odor perception dimensional space model of this embodiment can graphically present the odor type information, the emotional state information, and the target features, which is conducive to intuitively assisting food authenticity judgment, sensory quality control, and product development decision-making, and promotes the construction of food safety and neurosensory evaluation systems. Of course, this embodiment can also use other visualization methods to establish the odor perception dimensional space, and the present invention is not limited to this.

[0048] In other implementations, after obtaining the odor type information, this embodiment can also match the odor type information with the identification information of the gas output by the odor output device. The gas identification information reflects the description of the gas's odor. If the odor type information and the identification information successfully match, the analyzed odor type information is correct, and the odor identification result can be determined to be correct. If the odor type information and the identification information fail to match, the analyzed odor type information is incorrect, and the odor identification result can be determined to be incorrect. Based on this, this embodiment can also be applied to scenarios where food odor authenticity is verified. Specifically, the authenticity of the odor output by the odor output device can be determined by matching the odor type information with the identification information of the gas output by the odor output device. For example, if the odor type information and the identification information successfully match, it indicates that the odor of the gas output is consistent with the analyzed odor type information, and therefore the odor is authentic. This helps provide comprehensive and reliable technical support for odor sensory evaluation in the food industry.

[0049] In another implementation, this embodiment can also obtain the user's self-assessment of their emotions regarding the odor when the odor output device outputs odorous gas into the user's nasal cavity. Specifically, the user can obtain the self-assessment of their emotions based on the pleasure and arousal they enter after smelling the odor. This self-assessment of emotions is then compared with the analyzed emotional state information to determine the emotion recognition result. If the self-assessment of emotions is identical or similar to the analyzed emotional state information, the emotion recognition result can be determined to be accurate.

[0050] Furthermore, this embodiment can also be expanded to incorporate additional physiological data such as respiratory rate, skin temperature, and electromyography to build a multimodal deep learning model, mining the temporal and spatial correlations of multi-source signals, and improving the ability to identify complex and subtle odor differences. This is suitable for industries such as high-precision odor evaluation, new food research and development, and spice development. By recording multi-layered information from multimodal physiological information, this embodiment significantly improves the accuracy and robustness of odor perception and emotion decoding.

[0051] This embodiment can be further expanded to interdisciplinary fields such as healthcare, psychology, and intelligent interaction. Furthermore, with the advancement of artificial intelligence, micro-nano sensing technology, and wearable devices, the present invention can be deeply integrated with portable, low-power sensing hardware and adaptive learning algorithms to enable the development of consumer-grade odor-sensing products. Furthermore, the method of the present invention can also be applied to emerging fields such as cross-species olfactory pattern research and animal behavior experiments, promoting the integrated development of neurosensory science research and industrial applications.

[0052] The working principles of each step of the odor recognition and emotion analysis method based on multimodal physiological signals in this embodiment are the same as the principles of each device or module in the above-mentioned system embodiment, and will not be repeated here.

[0053] Each module in the aforementioned multimodal physiological signal-based odor recognition and emotion analysis system can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a terminal in hardware form, or stored in a memory in the terminal in software form, allowing the processor to call and execute the corresponding operations of each module.

[0054] Based on the above embodiment, the present invention further provides a terminal, the principle block diagram of the terminal can be as follows: Figure 7 The terminal may include one or more processors 100 ( Figure 7 Only one is shown in the figure), memory 101, and computer program 102 stored in memory 101 and executable on one or more processors 100. For example, a program for odor recognition and emotion analysis based on multimodal physiological signals. When one or more processors 100 execute computer program 102, each step of the embodiment of the odor recognition and emotion analysis method based on multimodal physiological signals can be implemented. Alternatively, when one or more processors 100 execute computer program 102, each module / unit in the embodiment of the odor recognition and emotion analysis system based on multimodal physiological signals can be implemented, without limitation.

[0055] In one embodiment, the processor 100 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0056] In one embodiment, memory 101 may be an internal storage unit of an electronic device, such as a hard drive or memory. Memory 101 may also be an external storage device of the electronic device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash memory card, etc. Furthermore, memory 101 may include both an internal storage unit of the electronic device and an external storage device. Memory 101 is used to store computer programs and other programs and data required by the terminal. Memory 101 may also be used to temporarily store data that has been output or is about to be output.

[0057] Those skilled in the art will understand that Figure 7 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present invention, and does not constitute a limitation on the terminal to which the solution of the present invention is applied. The specific terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0058] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, operation database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct RAM bus dynamic RAM (DRDRAM), and RAM bus dynamic RAM (RDRAM), etc.

[0059] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An odor recognition and emotion analysis system based on multimodal physiological signals, characterized in that: The system comprises: Multimodal physiological signal acquisition equipment, comprising an electroencephalogram (EEG) signal acquisition device, a skin electrical signal acquisition device, and an electrocardiogram (ECG) signal acquisition device; An odor output device, comprising an odor storage device and an odor inducing device for controlling the odor storage device to output odor, wherein the odor output device outputs odorous gas while the multimodal physiological signal acquisition device acquires multimodal physiological signals; A data processing and control device is connected to the multimodal physiological signal acquisition device and the odor output device respectively, and analyzes and processes the collected multimodal physiological signals, and outputs the odor type and the user's emotional state information.

2. The odor recognition and emotion analysis system based on multimodal physiological signals according to claim 1 is characterized in that: The EEG signal acquisition device includes: a helmet, a first control module arranged on the helmet, and a first electrode connected to the first control module, wherein the first electrode is arranged on the inside of the helmet for collecting EEG signals; The skin electrical signal acquisition device includes a wristband, a second control module provided on the wristband, and a second electrode connected to the second control module, wherein the second electrode is used to acquire skin electrical signals; The electrocardiogram signal acquisition device includes a waist belt, a third control module arranged on the waist belt, and a third electrode connected to the third control module, wherein the third electrode is used to acquire electrocardiogram signals; The first regulation module, the second regulation module and the third regulation module are all connected to the data processing and control device.

3. The odor recognition and emotion analysis system based on multimodal physiological signals according to claim 1 is characterized in that: The odor storage device comprises a plurality of odor bottles for storing gases with different odors, and the bottom of each odor bottle is connected to the odor inducing device through a pipeline.

4. The odor recognition and emotion analysis system based on multimodal physiological signals according to claim 3 is characterized in that: The odor inducing device stores odorless ammonia, which enters the bottom of the odor bottle through the pipe, forming a disturbed airflow to promote the volatilization of the gas in the odor bottle.

5. The odor recognition and emotion analysis system based on multimodal physiological signals according to claim 4 is characterized in that: The scent output device further comprises a flexible conduit, which is in communication with each scent bottle and is used to guide the gas with scent into the nasal cavity of the user.

6. The odor recognition and emotion analysis system based on multimodal physiological signals according to claim 5 is characterized in that: The odor output device further includes a heating module and a humidity control module, which are arranged between the odor storage device and the flexible conduit to control the temperature and humidity of the gas entering the user's nasal cavity.

7. A method for odor recognition and emotion analysis based on multimodal physiological signals, characterized in that: The method is applied to the odor recognition and emotion analysis system based on multimodal physiological signals according to any one of claims 1 to 6, and the method comprises: When the odor output device outputs odorous gas to the user's nasal cavity, a multimodal physiological signal is obtained, wherein the multimodal physiological signal includes: an electroencephalogram signal, a skin electrode signal, and an electrocardiogram signal; Preprocessing the multimodal physiological signal, and performing feature extraction based on the preprocessed multimodal physiological signal to obtain target features; Based on the target feature, odor type information and emotional state information are determined, and an odor perception dimensional space model is established. The odor perception dimensional space model is used to graphically present the odor type information, the emotional state information and the target feature.

8. The method for odor recognition and emotion analysis based on multimodal physiological signals according to claim 7, characterized in that: The preprocessing includes: band-pass filtering, power frequency interference removal, physiological artifact removal, and bad channel repair.

9. The method for odor recognition and emotion analysis based on multimodal physiological signals according to claim 8, characterized in that: Feature extraction is performed based on the preprocessed multimodal physiological signals to obtain target features, including: Extracting the time domain features, frequency domain features, wavelet features and entropy features of the pre-processed multimodal physiological signals to obtain multidimensional features; Based on the multi-dimensional features, target features are determined.

10. The method for odor recognition and emotion analysis based on multimodal physiological signals according to claim 9, characterized in that: Based on the target features, the odor type information and the emotional state information are determined, and an odor perception dimensional space model is established, including: Obtaining a preset classification model, classifying the target feature based on the classification model, and outputting odor type information and emotional state information corresponding to the target feature, wherein the classification model includes an ensemble learning framework or a manifold classifier, and the emotional state information includes pleasure and arousal; Mapping the odor type information, the emotional state information, and the target feature to establish an association relationship between the odor, emotion, and physiological signal; Based on the association relationship, an odor perception dimensional space model is established.

11. A terminal, characterized in that: The terminal includes a memory, a processor, and an odor recognition and emotion analysis program based on multimodal physiological signals stored in the memory and runnable on the processor. When the processor executes the odor recognition and emotion analysis program based on multimodal physiological signals, the steps of the odor recognition and emotion analysis method based on multimodal physiological signals as described in any one of claims 1 to 7 are implemented.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores an odor recognition and emotion analysis program based on multimodal physiological signals, and the odor recognition and emotion analysis program based on multimodal physiological signals implements the steps of the odor recognition and emotion analysis method based on multimodal physiological signals according to any one of claims 1 to 7 on the computer-readable storage medium.

Images