In-vehicle aromatherapy control method, device, and electronic equipment based on occupant emotion recognition

Abstract

This invention discloses a method, device, and electronic device for controlling in-vehicle aromatherapy based on occupant emotion recognition. The method includes: acquiring time-series data of the target occupant's voice, physiological state, and behavior; inputting these data into a pre-trained occupant emotion recognition model to obtain the target occupant's current emotion type and corresponding emotion intensity; determining the corresponding target aromatherapy type based on the current emotion type, and determining the corresponding aromatherapy release concentration and release rate based on the emotion intensity; and adjusting and controlling the in-vehicle aromatherapy device according to the target aromatherapy type, release concentration, and release rate. This invention improves the accuracy of in-vehicle aromatherapy control and enhances user comfort, and can be applied to the field of vehicle control technology.

Description

Technical Field

[0001] This invention relates to the field of vehicle control technology, and in particular to a method, device, and electronic device for controlling in-vehicle aromatherapy based on occupant emotion recognition. Background Technology

[0002] With increasing demands for automotive cabin comfort, in-car fragrance systems have become an important feature for enhancing the driving experience. Their core function is to improve air quality and create a comfortable atmosphere by releasing fragrances. However, existing in-car fragrance systems have the following drawbacks: 1) Existing in-vehicle aromatherapy adjustment technology only focuses on objective parameters such as environmental quality and time, and cannot recognize passengers' real-time emotions; solutions that rely on facial expression recognition are easily affected by light and posture, have low recognition accuracy, and single-dimensional recognition cannot guarantee the accuracy of emotion judgment. 2) The existing fragrance switching is manual or fixed preset, and cannot automatically switch to the appropriate fragrance according to the passenger's mood changes, thus failing to meet the core needs of mood relaxation and mood regulation through aromatherapy; 3) Parameters such as release concentration and release rate cannot be dynamically adjusted according to the intensity of emotions, resulting in poor adaptability.

[0003] In summary, existing in-vehicle aromatherapy control solutions cannot accurately identify passenger emotions or dynamically adapt aromatherapy scents, thus affecting the accuracy of in-vehicle aromatherapy control and user comfort. Summary of the Invention

[0004] The purpose of this invention is to at least partially solve one of the technical problems existing in the prior art.

[0005] Therefore, one objective of this invention is to provide a vehicle aromatherapy control method based on occupant emotion recognition. This method identifies the occupant's current emotion type and corresponding emotion intensity based on occupant voice time-series data, physiological state time-series data, and occupant behavior time-series data. Based on the current emotion type and corresponding emotion intensity, it matches the corresponding target aromatherapy type, aromatherapy release concentration, and aromatherapy release rate, thereby adaptively adjusting and controlling the vehicle aromatherapy device, improving the accuracy of vehicle aromatherapy control and the user's riding comfort.

[0006] Another objective of this invention is to provide an in-vehicle aromatherapy control device based on occupant emotion recognition.

[0007] To achieve the above-mentioned technical objectives, the technical solutions adopted in the embodiments of the present invention include: On one hand, embodiments of the present invention provide a method for controlling in-vehicle aromatherapy based on occupant emotion recognition, comprising the following steps: Acquire the target occupant's voice time-series data, physiological state time-series data, and occupant behavior time-series data; The passenger's speech time-series data, physiological state time-series data, and passenger behavior time-series data are input into a pre-trained passenger emotion recognition model to obtain the target passenger's current emotion type and corresponding emotion intensity. The target aromatherapy type is determined based on the current emotion type, and the corresponding aromatherapy release concentration and release rate are determined based on the emotion intensity. The in-vehicle aromatherapy device is adjusted and controlled according to the target aroma type, the aroma release concentration, and the aroma release rate.

[0008] Furthermore, in one embodiment of the present invention, the acquisition of the target occupant's occupant voice timing data, physiological state timing data, and occupant behavior timing data specifically includes: The target occupant's voice timing data is acquired through the vehicle's microphone. The heart rate, skin conductance signal, and body surface temperature of the target occupant are acquired through an in-vehicle wearable device to obtain the time-series data of the physiological state. The vehicle-mounted camera device acquires the occupant image information, and the occupant image information is used to detect limb movements to obtain the occupant behavior time sequence data.

[0009] Furthermore, in one embodiment of the present invention, the occupant emotion recognition model is trained through the following steps: We obtained time-series samples of the test occupants' speech, physiological state, and behavior, and determined the corresponding emotion type and emotion intensity labels through manual annotation. The passenger's speech time-series sample, physiological state time-series sample, and passenger behavior time-series sample are input into a pre-constructed CNN-LSTM hybrid neural network to obtain the predicted emotion type and predicted emotion intensity. The loss value is determined based on the predicted emotion type, the predicted emotion intensity, the emotion type label, and the emotion intensity label. The parameters of the CNN-LSTM hybrid neural network are updated based on the loss value to obtain the trained passenger emotion recognition model.

[0010] Further, in one embodiment of the present invention, the CNN-LSTM hybrid neural network includes a first CNN branch, a second CNN branch, a third CNN branch, a feature fusion layer, an LSTM layer, and a fully connected layer. The step of inputting the passenger's speech time-series samples, the physiological state time-series samples, and the passenger's behavior time-series samples into the pre-constructed CNN-LSTM hybrid neural network to obtain predicted emotion type and predicted emotion intensity specifically includes: The occupant speech time-series sample, the physiological state time-series sample, and the occupant behavior time-series sample are respectively input into the first CNN branch, the second CNN branch, and the third CNN branch for feature extraction to obtain speech time-series features, physiological state time-series features, and behavior time-series features. The feature fusion layer performs temporal alignment and feature fusion on the speech temporal features, the physiological state temporal features, and the behavioral temporal features, thereby fusing the temporal features. The fused temporal features are input into the LSTM layer to calculate the hidden state, and the corresponding hidden state vector is obtained. The hidden state vector is mapped to the predicted emotion type and the predicted emotion intensity through the fully connected layer.

[0011] Furthermore, in one embodiment of the present invention, the step of determining the corresponding target aromatherapy type based on the current emotion type, and determining the corresponding aromatherapy release concentration and release rate based on the emotion intensity, specifically includes: The target aromatherapy type is determined based on the current mood type and a preset aromatherapy type mapping table, and the aromatherapy release parameter mapping sub-table corresponding to the target aromatherapy type is obtained. The aroma release concentration and the aroma release rate are determined based on the emotional intensity and the aroma release parameter mapping sub-table.

[0012] Furthermore, in one embodiment of the present invention, the in-vehicle aromatherapy device includes multiple aromatherapy essential oil chambers, an atomizing chamber, a diffusion channel, and a fan. The aromatherapy essential oil chambers are used to store aromatherapy essential oils corresponding to various aromatherapy types. The atomizing chambers are used to atomize the aromatherapy essential oils. The diffusion channel is used to diffuse the atomized aromatherapy mist into the vehicle. The fan is used to adjust the diffusion rate of the aromatherapy mist.

[0013] Furthermore, in one embodiment of the present invention, the adjustment and control of the in-vehicle aromatherapy device based on the target aroma type, the aroma release concentration, and the aroma release rate specifically includes: The corresponding target aromatherapy essential oil chamber is determined according to the target aromatherapy type, and the valve of the target aromatherapy essential oil chamber is controlled to open, so that the target aromatherapy essential oil in the target aromatherapy essential oil chamber flows into the atomizing chamber. The target aromatherapy essential oil is atomized by the atomization module of the atomization chamber according to the aroma release concentration to obtain the target aromatherapy mist; The fan speed is controlled according to the aroma release rate, so that the target aroma mist diffuses into the vehicle through the diffusion channel according to the aroma release rate.

[0014] On the other hand, embodiments of the present invention provide a vehicle-mounted aromatherapy control device based on occupant emotion recognition, comprising: The data acquisition module is used to acquire the target occupant's voice time-series data, physiological state time-series data, and occupant behavior time-series data. An emotion recognition module is used to input the passenger's speech time-series data, physiological state time-series data, and passenger behavior time-series data into a pre-trained passenger emotion recognition model to obtain the current emotion type and corresponding emotion intensity of the target passenger. The parameter determination module is used to determine the corresponding target aromatherapy type based on the current emotion type, and to determine the corresponding aromatherapy release concentration and aromatherapy release rate based on the emotion intensity. The aromatherapy control module is used to adjust and control the in-vehicle aromatherapy device according to the target aromatherapy type, the aromatherapy release concentration, and the aromatherapy release rate.

[0015] On the other hand, embodiments of the present invention provide an electronic device, including: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the above-described in-vehicle aromatherapy control method based on occupant emotion recognition.

[0016] On the other hand, embodiments of the present invention also provide a computer-readable storage medium storing a processor-executable computer program that, when executed by a processor, implements the above-described in-vehicle aromatherapy control method based on occupant emotion recognition.

[0017] On the other hand, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the above-described in-vehicle aromatherapy control method based on occupant emotion recognition.

[0018] The advantages and beneficial effects of the present invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention: This invention acquires the time-series data of a target occupant's voice, physiological state, and behavior. This data is then input into a pre-trained occupant emotion recognition model to obtain the target occupant's current emotion type and corresponding emotion intensity. Based on the current emotion type, a corresponding target aromatherapy type is determined, and based on the emotion intensity, the corresponding aromatherapy release concentration and release rate are determined. The in-vehicle aromatherapy device is then adjusted and controlled according to the target aromatherapy type, release concentration, and release rate. This invention identifies the occupant's current emotion type and corresponding emotion intensity based on their voice, physiological state, and behavior time-series data. By matching the current emotion type and corresponding emotion intensity with the corresponding target aromatherapy type, release concentration, and release rate, the in-vehicle aromatherapy device can be adaptively adjusted and controlled, improving the accuracy of in-vehicle aromatherapy control and enhancing user comfort. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments of the present invention are described below. It should be understood that the drawings described below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating the steps of an in-vehicle aromatherapy control method based on occupant emotion recognition, provided in an embodiment of the present invention; Figure 2 This is a structural block diagram of an in-vehicle aromatherapy control device based on occupant emotion recognition, provided in an embodiment of the present invention. Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of this invention; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this invention as detailed in the appended claims.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0023] The in-vehicle aromatherapy control method based on occupant emotion recognition provided in this invention can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited thereto; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application implementing the in-vehicle aromatherapy control method based on occupant emotion recognition, but is not limited to the above forms.

[0024] This invention can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0025] It should be noted that in various specific embodiments of the present invention, when processing data related to user identity or characteristics, such as user information, user behavior data, user historical data, and user parking space location information, user permission or consent is obtained first. Furthermore, the collection, use, and processing of this data comply with relevant laws, regulations, and standards. In addition, when embodiments of the present invention require access to sensitive personal information of users, separate permission or consent from the user is obtained through pop-ups or redirection to a confirmation page. Only after obtaining the user's separate permission or consent is the necessary user-related data for the normal operation of the embodiments of the present invention acquired.

[0026] Reference Figure 1 This invention provides a method for controlling in-vehicle aromatherapy based on occupant emotion recognition, specifically including the following steps: S101. Obtain the target occupant's voice time-series data, physiological state time-series data, and occupant behavior time-series data; S102. Input the passenger's speech time-series data, physiological state time-series data, and passenger behavior time-series data into the pre-trained passenger emotion recognition model to obtain the current emotion type and corresponding emotion intensity of the target passenger. S103. Determine the target aromatherapy type based on the current mood type, and determine the corresponding aromatherapy release concentration and release rate based on the intensity of the mood. S104. Adjust and control the in-vehicle aromatherapy device according to the target aromatherapy type, aromatherapy release concentration, and aromatherapy release rate.

[0027] This invention identifies the current emotion type and corresponding emotion intensity of a passenger based on the passenger's voice time-series data, physiological state time-series data, and passenger behavior time-series data. Based on the current emotion type and corresponding emotion intensity, it matches the corresponding target aromatherapy type, aromatherapy release concentration, and aromatherapy release rate, thereby adaptively adjusting and controlling the in-vehicle aromatherapy device, improving the accuracy of in-vehicle aromatherapy control and the user's riding comfort.

[0028] As a further optional implementation, the acquisition of the target occupant's occupant voice time-series data, physiological state time-series data, and occupant behavior time-series data specifically includes: S1011. Acquire the occupant's voice timing data through the vehicle microphone; S1012. Obtain the target occupant's heart rate, skin conductance signal, and body surface temperature through an in-vehicle wearable device to obtain physiological state time-series data. S1013. Obtain occupant image information of the target occupant through the vehicle-mounted camera device, perform limb movement detection on the occupant image information, and obtain occupant behavior time sequence data.

[0029] Specifically, for acquiring the occupant's speech time sequence data, the system relies on an in-vehicle speech acquisition system to capture the target occupant's speech content in real time through a high-sensitivity microphone array distributed throughout the cabin. It not only records the speech text but also simultaneously collects the acoustic features of the speech, such as pitch, speech rate, volume, and pauses. These features are key auxiliary information for judging emotions. The acquired raw speech is then subjected to noise reduction processing to filter out interference such as engine noise inside the vehicle and external traffic noise, ensuring the clarity of the speech information.

[0030] The acquisition of physiological state time-series data includes three parts: heart rate, skin conductance signal, and body surface temperature. 1) Heart rate and heart rate variability: The heart rate value and heart rate variability index of the target occupant are monitored in real time through in-vehicle wearable devices (such as smart bracelets, steering wheel heart rate sensors) or physiological monitoring sensors built into the seat. Heart rate variability can reflect the activity of the autonomic nervous system and indirectly reflect emotional fluctuations.

[0031] 2) Electrodermal signal: Using electrodermal sensors on the seat armrest or steering wheel, the conductivity of the occupant's skin is measured. When a person experiences emotional fluctuations, the secretion of sweat glands changes, which in turn affects the electrodermal response.

[0032] 3) Body surface temperature: The seat uses an infrared body temperature sensor to monitor the occupant's body surface temperature. The body temperature may rise slightly when the occupant is emotionally agitated.

[0033] The acquisition of occupant behavior time-series data includes three parts: facial expressions, body movements, and driving operation behaviors (if any), among which: 1) Facial expression recognition: The high-definition camera in the cockpit captures the facial expressions of the target occupants and recognizes facial movements such as frowning, smiling, pouting, and staring, as well as details such as the direction of eye gaze and blinking frequency.

[0034] 2) Body movement monitoring: Using seat pressure sensors and in-vehicle motion capture cameras, the body movements of occupants are recorded, such as whether they frequently adjust their sitting posture, clench their fists, or tap the seat. Body movements can directly reflect emotional state.

[0035] 3) Driving behavior: If the target occupant is the driver, collect data on their steering wheel operation force, the frequency and force of pressing the accelerator and brake pedals, and the frequency of using the turn signals and horn. Changes in these behaviors can also reflect emotional changes.

[0036] The collected occupant speech time-series data, physiological state time-series data, and occupant behavior time-series data are input into a pre-trained occupant emotion recognition model. The model analyzes and processes the input data and outputs the current emotion type of the target occupant, such as happy, angry, anxious, calm, etc., and outputs the corresponding emotion intensity value. Emotion intensity is generally represented by a value of 0-1, with a higher value indicating a stronger emotion.

[0037] As an optional implementation, the occupant emotion recognition model is trained through the following steps: S201. Obtain the time-series samples of the test occupants' speech, physiological state, and behavior, and determine the corresponding emotion type and emotion intensity labels through manual annotation. S202. Input the passenger's speech time-series samples, physiological state time-series samples, and passenger behavior time-series samples into a pre-constructed CNN-LSTM hybrid neural network to obtain the predicted emotion type and predicted emotion intensity. S203. Determine the loss value based on the predicted emotion type, predicted emotion intensity, emotion type label, and emotion intensity label; S204. Update the parameters of the CNN-LSTM hybrid neural network based on the loss value to obtain the trained passenger emotion recognition model.

[0038] Specifically, the process involves acquiring temporal samples of passenger speech, physiological state, and behavior, and manually labeling them to determine corresponding emotion type and intensity labels. These samples are then input into a pre-built CNN-LSTM hybrid neural network to obtain predicted emotion type and intensity. A loss value is determined based on the predicted emotion type, intensity, emotion type label, and intensity label. The parameters of the CNN-LSTM hybrid neural network are updated according to the loss value, completing one iteration of training. Training stops when the number of iterations reaches a preset threshold or the loss value falls below the preset threshold, resulting in a well-trained passenger emotion recognition model.

[0039] As a further optional implementation, the CNN-LSTM hybrid neural network includes a first CNN branch, a second CNN branch, a third CNN branch, a feature fusion layer, an LSTM layer, and a fully connected layer. Passenger speech time-series samples, physiological state time-series samples, and passenger behavior time-series samples are input into the pre-constructed CNN-LSTM hybrid neural network to obtain predicted emotion type and predicted emotion intensity. Specifically, this includes: S2021. Input the passenger speech time-series sample, physiological state time-series sample and passenger behavior time-series sample into the first CNN branch, the second CNN branch and the third CNN branch respectively for feature extraction to obtain speech time-series features, physiological state time-series features and behavior time-series features. S2022. The feature fusion layer performs temporal alignment and feature fusion on speech temporal features, physiological state temporal features and behavioral temporal features, and fuses temporal features. S2023. Input the fused temporal features into the LSTM layer to calculate the hidden state and obtain the corresponding hidden state vector. S2024. The hidden state vector is mapped to the predicted emotion type and predicted emotion intensity through a fully connected layer.

[0040] Specifically, the CNN-LSTM hybrid neural network includes a first CNN branch, a second CNN branch, a third CNN branch, a feature fusion layer, an LSTM layer, and a fully connected layer. Specifically: the first CNN branch, the second CNN branch, and the third CNN branch are used to extract features from the occupant's speech temporal samples, physiological state temporal samples, and occupant behavior temporal samples, respectively, to obtain speech temporal features, physiological state temporal features, and behavior temporal features; the feature fusion layer is used to perform temporal alignment and feature fusion on the speech temporal features, physiological state temporal features, and behavior temporal features, fusing the temporal features; the LSTM layer is used to calculate the hidden state on the fused temporal features, obtaining the corresponding hidden state vector; and the fully connected layer is used to map the hidden state vector to the predicted emotion type and predicted emotion intensity.

[0041] As a further optional implementation, the target aromatherapy type is determined based on the current mood type, and the corresponding aromatherapy release concentration and release rate are determined based on the mood intensity, specifically including: S1031. Determine the target aromatherapy type based on the current mood type and the preset aromatherapy type mapping table, and obtain the aromatherapy release parameter mapping sub-table corresponding to the target aromatherapy type; S1032. Determine the aroma release concentration and aroma release rate based on the mapping sub-table of emotional intensity and aroma release parameters.

[0042] Specifically, based on a preset aromatherapy type mapping table, the target aromatherapy type matching the current emotion type is determined, and the aromatherapy release parameter mapping sub-table corresponding to the target aromatherapy type is obtained. For example, when the emotion type is anxiety, aromatherapy with soothing and relaxing effects such as lavender and chamomile is selected; when the emotion type is fatigue, aromatherapy with refreshing and invigorating effects such as lemon and peppermint is selected; when the emotion type is anger, aromatherapy with calming and soothing effects such as rose and jasmine is selected.

[0043] Based on the emotional intensity value and a defined aromatherapy release parameter mapping table, the aromatherapy release concentration and release rate are determined. Higher emotional intensity results in a higher aromatherapy release concentration to enhance the mood-regulating effect; lower emotional intensity results in a lower release concentration to avoid overstimulation. For example, when the emotional intensity is 0.8 (intense anxiety), the aromatherapy concentration is set to a high level; when the emotional intensity is 0.3 (mild anxiety), the aromatherapy concentration is set to a low level. The aromatherapy release rate is determined based on the emotional intensity and aromatherapy type. For scenarios requiring rapid mood regulation (such as intense anger), a faster release rate is set to quickly fill the cabin; for scenarios requiring sustained mood relief (such as mild anxiety), a slower release rate is set to maintain a stable aromatherapy concentration within the cabin.

[0044] As an optional implementation, the in-vehicle aromatherapy device includes multiple aromatherapy essential oil chambers, an atomizing chamber, a diffusion channel, and a fan. The aromatherapy essential oil chambers are used to store aromatherapy essential oils corresponding to various aromatherapy types. The atomizing chambers are used to atomize the aromatherapy essential oils. The diffusion channel is used to diffuse the atomized aromatherapy mist into the vehicle. The fan is used to adjust the diffusion rate of the aromatherapy mist.

[0045] As a further optional implementation, the in-vehicle air freshener device is adjusted and controlled according to the target fragrance type, fragrance release concentration, and fragrance release rate, specifically including: S1041. Determine the corresponding target aromatherapy essential oil chamber according to the target aromatherapy type, and control the valve of the target aromatherapy essential oil chamber to open so that the target aromatherapy essential oil in the target aromatherapy essential oil chamber flows into the atomization chamber. S1042. The target aromatherapy essential oil is atomized according to the aroma release concentration through the atomization module of the atomization chamber to obtain the target aromatherapy mist; S1043. Control the fan speed according to the aroma release rate so that the target aroma mist diffuses into the car through the diffusion channel according to the aroma release rate.

[0046] Specifically, depending on the target aromatherapy type, the aromatherapy device is controlled to switch to the corresponding aromatherapy essential oil storage module. For example, the device has multiple essential oil compartments, which store different types of aromatherapy essential oils. The system controls the valve to open the channel of the corresponding essential oil compartment, so that the target aromatherapy essential oil flows into the atomization layer.

[0047] The atomizing chamber adjusts the amount of essential oil atomized or evaporated by the aromatherapy device according to the concentration of aroma release. The higher the concentration, the greater the amount of atomization or evaporation; the lower the concentration, the smaller the amount of atomization or evaporation.

[0048] The fan speed of the aroma diffuser is controlled according to the release rate of the aroma. The faster the release rate, the faster the fan speed, thereby accelerating the diffusion of the aroma in the car; the slower the release rate, the slower the fan speed, to maintain the slow release of the aroma.

[0049] During the adjustment process, the concentration of the fragrance in the car is continuously monitored, and data is fed back in real time through the air quality sensor in the car to ensure that the fragrance concentration is maintained within the set range, so as to avoid the concentration being too high or too low and affecting the adjustment effect.

[0050] The method steps of the embodiments of the present invention have been described above. It can be understood that the embodiments of the present invention identify the current emotion type and corresponding emotion intensity of the occupant based on the occupant's voice time-series data, physiological state time-series data, and occupant behavior time-series data. Based on the current emotion type and corresponding emotion intensity, the corresponding target aromatherapy type, aromatherapy release concentration, and aromatherapy release rate are matched, thereby adaptively adjusting and controlling the in-vehicle aromatherapy device, improving the accuracy of in-vehicle aromatherapy control and the user's riding comfort.

[0051] Reference Figure 2 This invention provides a vehicle-mounted aromatherapy control device based on occupant emotion recognition, comprising: The data acquisition module is used to acquire the target occupant's voice time-series data, physiological state time-series data, and occupant behavior time-series data. The emotion recognition module is used to input the passenger's speech time-series data, physiological state time-series data, and passenger behavior time-series data into a pre-trained passenger emotion recognition model to obtain the target passenger's current emotion type and corresponding emotion intensity. The parameter determination module is used to determine the corresponding target aromatherapy type based on the current emotion type, and to determine the corresponding aromatherapy release concentration and release rate based on the emotion intensity. The aromatherapy control module is used to adjust and control the in-vehicle aromatherapy device according to the target aromatherapy type, aromatherapy release concentration, and aromatherapy release rate.

[0052] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0053] Reference Figure 3 This invention provides an electronic device, comprising: At least one processor; At least one memory for storing at least one program; When the above-mentioned at least one program is executed by the above-mentioned at least one processor, the above-mentioned at least one processor implements the above-mentioned in-vehicle aromatherapy control method based on occupant emotion recognition.

[0054] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0055] This invention also provides a computer-readable storage medium storing a processor-executable computer program that, when executed by a processor, implements the above-described in-vehicle aromatherapy control method based on occupant emotion recognition.

[0056] This invention provides a computer-readable storage medium that can execute a vehicle aromatherapy control method based on occupant emotion recognition provided in the method embodiments of this invention. It can execute any combination of the implementation steps of the method embodiments and has the corresponding functions and beneficial effects of the method.

[0057] This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described in-vehicle aromatherapy control method based on occupant emotion recognition.

[0058] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented by the embodiments of this program product are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0059] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0060] The embodiments described in this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0061] The terms "first," "second," "third," "fourth," etc. (if present) in the specification and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0062] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the aforementioned blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is changed and sub-operations described as part of a larger operation are executed independently.

[0063] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the aforementioned functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0064] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0065] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0066] More specific examples (a non-exhaustive list) of computer-readable media include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the aforementioned program can be printed, because the aforementioned program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0067] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0068] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0069] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0070] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A method for controlling an in-vehicle aromatherapy based on occupant emotion recognition, the method comprising: obtaining an occupant emotion; and controlling an in-vehicle aromatherapy based on the obtained occupant emotion. Includes the following steps: Acquire the target occupant's voice time-series data, physiological state time-series data, and occupant behavior time-series data; The passenger's speech time-series data, physiological state time-series data, and passenger behavior time-series data are input into a pre-trained passenger emotion recognition model to obtain the target passenger's current emotion type and corresponding emotion intensity. The target aromatherapy type is determined based on the current emotion type, and the corresponding aromatherapy release concentration and release rate are determined based on the emotion intensity. The in-vehicle aromatherapy device is adjusted and controlled according to the target aroma type, the aroma release concentration, and the aroma release rate.

2. The in-vehicle aromatherapy control method based on occupant emotion recognition according to claim 1, characterized in that, The acquisition of the target occupant's voice time-series data, physiological state time-series data, and occupant behavior time-series data specifically includes: The target occupant's voice timing data is acquired through the vehicle's microphone. The heart rate, skin conductance signal, and body surface temperature of the target occupant are acquired through an in-vehicle wearable device to obtain the time-series data of the physiological state. The vehicle-mounted camera device acquires the occupant image information, and the occupant image information is used to detect limb movements to obtain the occupant behavior time sequence data.

3. The in-vehicle aromatherapy control method based on occupant emotion recognition according to claim 1, characterized in that, The occupant emotion recognition model is trained through the following steps: We obtained time-series samples of the test occupants' speech, physiological state, and behavior, and determined the corresponding emotion type and emotion intensity labels through manual annotation. The passenger's speech time-series sample, physiological state time-series sample, and passenger behavior time-series sample are input into a pre-constructed CNN-LSTM hybrid neural network to obtain the predicted emotion type and predicted emotion intensity. The loss value is determined based on the predicted emotion type, the predicted emotion intensity, the emotion type label, and the emotion intensity label. The parameters of the CNN-LSTM hybrid neural network are updated based on the loss value to obtain the trained passenger emotion recognition model.

4. The in-vehicle aromatherapy control method based on occupant emotion recognition according to claim 3, characterized in that, The CNN-LSTM hybrid neural network includes a first CNN branch, a second CNN branch, a third CNN branch, a feature fusion layer, an LSTM layer, and a fully connected layer. The step of inputting the passenger's speech time-series samples, physiological state time-series samples, and passenger behavior time-series samples into the pre-constructed CNN-LSTM hybrid neural network to obtain predicted emotion type and predicted emotion intensity specifically includes: The occupant speech time-series sample, the physiological state time-series sample, and the occupant behavior time-series sample are respectively input into the first CNN branch, the second CNN branch, and the third CNN branch for feature extraction to obtain speech time-series features, physiological state time-series features, and behavior time-series features. The feature fusion layer performs temporal alignment and feature fusion on the speech temporal features, the physiological state temporal features, and the behavioral temporal features, thereby fusing the temporal features. The fused temporal features are input into the LSTM layer to calculate the hidden state, and the corresponding hidden state vector is obtained. The hidden state vector is mapped to the predicted emotion type and the predicted emotion intensity through the fully connected layer.

5. The in-vehicle aromatherapy control method based on occupant emotion recognition according to claim 1, characterized in that, The step of determining the corresponding target aromatherapy type based on the current emotion type, and determining the corresponding aromatherapy release concentration and release rate based on the emotion intensity, specifically includes: The target aromatherapy type is determined based on the current mood type and a preset aromatherapy type mapping table, and the aromatherapy release parameter mapping sub-table corresponding to the target aromatherapy type is obtained. The aroma release concentration and the aroma release rate are determined based on the emotional intensity and the aroma release parameter mapping sub-table.

6. The in-vehicle aromatherapy control method based on occupant emotion recognition according to claim 1, characterized in that, The in-vehicle aromatherapy device includes multiple aromatherapy essential oil chambers, an atomizing chamber, a diffusion channel, and a fan. The aromatherapy essential oil chambers are used to store aromatherapy essential oils corresponding to various aromatherapy types. The atomizing chamber is used to atomize the aromatherapy essential oils. The diffusion channel is used to diffuse the atomized aromatherapy mist into the vehicle. The fan is used to adjust the diffusion rate of the aromatherapy mist.

7. The in-vehicle aromatherapy control method based on occupant emotion recognition according to claim 6, characterized in that, The adjustment and control of the in-vehicle air freshener device based on the target fragrance type, the fragrance release concentration, and the fragrance release rate specifically includes: The corresponding target aromatherapy essential oil chamber is determined according to the target aromatherapy type, and the valve of the target aromatherapy essential oil chamber is controlled to open, so that the target aromatherapy essential oil in the target aromatherapy essential oil chamber flows into the atomizing chamber. The target aromatherapy essential oil is atomized by the atomization module of the atomization chamber according to the aroma release concentration to obtain the target aromatherapy mist; The fan speed is controlled according to the aroma release rate, so that the target aroma mist diffuses into the vehicle through the diffusion channel according to the aroma release rate.

8. A vehicle-mounted aromatherapy control device based on occupant emotion recognition, characterized in that, include: The data acquisition module is used to acquire the target occupant's voice time-series data, physiological state time-series data, and occupant behavior time-series data. An emotion recognition module is used to input the passenger's speech time-series data, physiological state time-series data, and passenger behavior time-series data into a pre-trained passenger emotion recognition model to obtain the current emotion type and corresponding emotion intensity of the target passenger. The parameter determination module is used to determine the corresponding target aromatherapy type based on the current emotion type, and to determine the corresponding aromatherapy release concentration and aromatherapy release rate based on the emotion intensity. The aromatherapy control module is used to adjust and control the in-vehicle aromatherapy device according to the target aromatherapy type, the aromatherapy release concentration, and the aromatherapy release rate.

9. An electronic device, characterized in that, include: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements a vehicle aromatherapy control method based on occupant emotion recognition as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements a vehicle aromatherapy control method based on occupant emotion recognition as described in any one of claims 1 to 7.

Images