Vehicle-mounted system for providing emotional value based on AI

Abstract

The invention discloses a vehicle-mounted system for providing emotional value based on AI, and particularly relates to the field of artificial intelligence, which comprises a driver facial expression acquisition module, an emotion diagnosis module, a vehicle-mounted AI control system and an emotion regulation module, facial expressions of a driver are captured through a camera, electroencephalogram activities are detected through a wireless EEG sensor, eyebrow challenge, eye expression, lip radian and facial muscle changes are analyzed, and the emotion of the driver is preliminarily judged; the system combines facial expressions and electroencephalogram data, utilizes a Bayesian network algorithm to comprehensively judge emotional states, and divides the emotional states into positive emotional states and negative emotional states; under the positive emotion, the system is kept silent; in case of negative emotion, relaxed music is played, positive voice prompts are provided through the bone conduction audio unit, and the environment in the vehicle is adjusted to improve the driving experience.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more particularly, to an in-vehicle system that provides emotional value based on AI. Background Art

[0002] With the rapid development of the automotive industry, cars have become more than just a means of transportation; they have gradually evolved into people's "third living space." This shift has driven a fundamental evolution in in-vehicle infotainment (IVI). From its initial roots as a radio and Bluetooth phone system to today's comprehensive platforms integrating multiple functional modules, including audio, video, navigation, social communication, ADAS driver assistance, body control, and human-computer interaction, IVI has become a crucial component of the intelligent cockpit.

[0003] Modern users' expectations for in-car systems are no longer limited to navigation and entertainment. They now desire a sense of care and understanding while driving. Intelligent assistants, for example, that can respond to users' emotional changes and provide relevant advice and support have become crucial factors in enhancing the user experience. Furthermore, with increasing awareness of mental health, providing emotional value has become an emerging trend, attracting continued market attention and research.

[0004] However, traditional in-vehicle systems still have some shortcomings in actual use. For example, they cannot recognize the user's emotional state and therefore cannot provide personalized services based on the user's emotional changes. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides an in-vehicle system for providing emotional value based on AI to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] Driver facial expression acquisition module: used to capture facial expressions through the camera and detect brain electrical activity through the wireless EEG sensor, and transmit the obtained information to the emotion diagnosis module;

[0008] Emotion diagnosis module: used to diagnose preliminary emotions based on the received information and send the diagnosed preliminary emotions to the on-board AI control system;

[0009] In-vehicle AI control system: used to analyze received emotions using in-vehicle AI;

[0010] Emotion regulation module: used to transmit the processed content of the conversation back to the driver through "special glasses" and bone conduction.

[0011] Preferably, in the driver's facial expression acquisition module, the camera captures the facial expression and carefully analyzes the eyebrows, eyes, lips curvature and subtle changes in facial muscles to acquire the emotional information conveyed by the passenger's face from 180 degrees;

[0012] The wireless EEG sensor is installed on the pillow. The brain electrical activity data is obtained through the wireless EEG sensor, and the signal changes of the EEG are converted into language information.

[0013] Preferably, in the emotion diagnosis module, the driver's initial emotion is first preliminarily judged by facial expressions. Facial expressions are divided into eyebrow raising, eyebrow expression, lip curvature and changes in facial muscles. Eyebrow raising is divided into eyebrow raising, eyebrow furrowing and other eyebrow fluctuations. The method of judging the eyebrow state is specifically as follows:

[0014] Raised eyebrows indicate surprise. When a driver encounters unexpected information or events, their eyebrows will be raised upwards, forming a surprised expression. This expression is accompanied by widened eyes and a slightly open mouth. This is an initial positive emotion.

[0015] Frowning eyebrows indicates dissatisfaction. When the eyebrows gather in the middle to form a frown, it means that the individual is experiencing negative emotions and is dissatisfied with what happened. This is the initial negative interest.

[0016] Other eyebrow fluctuations indicate the driver's state of happiness. During the conversation, such fluctuations are accompanied by smiles and friendly eye contact; this is an initial positive emotion.

[0017] Preferably, in the in-vehicle AI control system, the driver's emotions are determined by performing a secondary judgment based on the data obtained by the wireless EEG sensor and the emotions initially judged by the facial expressions; a Bayesian network structure is constructed using a structural learning method based on the preliminary judgment results of the facial expressions and EEG data; under the determined Bayesian network structure, a Bayesian estimation learning method is used to calculate the conditional probability table of each node; in the Bayesian network, the posterior probability of each emotion or emotional state is calculated using an inference algorithm based on the observed facial expressions and EEG data features; and the emotional state with the largest posterior probability is selected as the comprehensive judgment result;

[0018] Based on the comprehensive judgment results, the driver's emotions are divided into positive emotions and negative emotions; positive emotions include happiness, satisfaction, and calmness; negative emotions include sadness, anger, and anxiety.

[0019] Preferably, in the emotion regulation module, the special glasses integrate a bone conduction audio unit, which is located at the rear of the glasses; the bone conduction audio unit emits sound waves through a vibrating membrane, directly acting on the skull; the received information is converted into a specific audio signal, which generates vibrations through the bone conduction unit; the driver receives the vibration signals through bone conduction, and these signals are transmitted to the cochlea and perceived as sound; the sound feedback from bone conduction does not interfere with the sound of the driver's surrounding environment; among them, the special glasses can transmit the detected heart rate, blood pressure, and respiratory status data to the on-board AI.

[0020] The technical effects and advantages of the present invention are as follows:

[0021] 1. By monitoring the driver's emotional changes in real time, the system can issue warnings when the driver's emotions fluctuate significantly, reminding the driver to remain calm, thereby reducing driving errors caused by emotional fluctuations;

[0022] 2. In-vehicle AI systems can learn and adapt to user behavior patterns, provide more humane feedback, and enhance the interaction between driver and vehicle;

[0023] 3. By collecting and analyzing emotional data during driving, the system can help manufacturers better understand user needs, thereby optimizing product design and improving the overall performance and user experience of the vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Schematic diagram of module connection of the present invention.

[0025] Figure 2 It is a schematic diagram of the process of the present invention.

[0026] Figure 3 This is a schematic diagram of the sensor of the present invention placed on a pillow for back support. DETAILED DESCRIPTION

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0028] See also Figure 1 As shown, the present invention provides an in-vehicle system for providing emotional value based on AI, which includes a driver facial expression acquisition module, an emotion diagnosis module, an in-vehicle AI control module and an emotion regulation module:

[0029] Driver facial expression acquisition module: used to capture facial expressions through the camera and detect brain electrical activity through the wireless EEG sensor, and transmit the obtained information to the emotion diagnosis module;

[0030] In the driver's facial expression acquisition module, the camera captures facial expressions and carefully analyzes the raised eyebrows, the expression of the eyes, the curvature of the lips, and the subtle changes in facial muscles to obtain 180-degree emotional information conveyed by the passenger's face;

[0031] The wireless EEG sensor is installed on the pillow to obtain brain electrical activity data and convert the signal changes of the EEG into language information;

[0032] Monitoring of EEG activity can provide a more detailed picture of the driver's mental state by capturing brain waves in different frequency bands, including alpha waves, beta waves, and delta waves; for example, an increase in alpha waves is often associated with relaxation and pleasure, while beta waves are usually associated with high concentration and tension.

[0033] Emotion diagnosis module: used to diagnose preliminary emotions based on the received information and send the diagnosed preliminary emotions to the on-board AI control system;

[0034] In the emotion diagnosis module, the driver's initial emotion is first determined by facial expressions. Facial expressions are divided into eyebrow raising, eyebrow expression, lip curvature, and changes in facial muscles. Eyebrow raising is divided into eyebrow raising, eyebrow furrowing, and other eyebrow fluctuations. The method for determining the eyebrow state is specifically as follows:

[0035] Raised eyebrows indicate surprise. When a driver encounters unexpected information or events, their eyebrows will be raised upwards, forming a surprised expression. This expression is accompanied by widened eyes and a slightly open mouth. This is an initial positive emotion.

[0036] Frowning eyebrows indicates dissatisfaction. When the eyebrows gather in the middle to form a frown, it means that the individual is experiencing negative emotions and is dissatisfied with what happened. This is the initial negative interest.

[0037] Other eyebrow fluctuations indicate the driver's state of happiness. During the conversation, such fluctuations are accompanied by smiles and friendly eye contact; this is an initial positive emotion.

[0038] The expressions of the eyes can be divided into flickering and dull eyes. The methods for judging the expressions of the eyes are as follows:

[0039] Eye flickering indicates joy; when a driver feels joy, his eyes become bright and flickering, reflecting an active inner state; this is a preliminary positive emotion;

[0040] A dull look in the eyes indicates fatigue, which is a negative emotion. When the driver's eyes become dull and lack focus, it indicates that the current emotional state is low and lacks motivation, which is an initial negative emotion.

[0041] The curvature of the lips is divided into upturned corners and tightly pursed corners. The method for judging the curvature of the lips is as follows:

[0042] The upturned corners of the mouth indicate happiness; when the corners of the mouth are bent upward, it means that the driver is experiencing a positive emotional state and is satisfied with the current event; this is an initial positive emotion;

[0043] Tightly pursed lips indicate dissatisfaction. When the lips are tightly pursed together, it indicates that the driver is experiencing some negative emotions and is dissatisfied with the current event. This is a preliminary negative emotion.

[0044] Drooping mouth corners indicate sadness; when the mouth corners are bent downward, it indicates that the driver is experiencing a negative emotional state and is disappointed with the current event; this indicates a preliminary negative emotion;

[0045] Facial muscle changes are divided into facial tension and facial relaxation. The specific methods of facial muscle changes are:

[0046] Facial tension means that when a driver feels nervous, his facial muscles become tense and stiff. This state of tension is manifested by frowning eyebrows, narrowing eyes, and tightly closed lips. This indicates a preliminary negative emotion.

[0047] Facial relaxation means that when the driver is in a happy state, the facial muscles will become soft and relaxed; this state of relaxation may be manifested by the stretching of eyebrows, the brightness of eyes and the upturn of mouth corners; this indicates a preliminary positive emotion;

[0048] By analyzing the movement of eyebrows, the expression of eyes, the curvature of lips and changes in facial muscles, the driver's initial emotional state can be judged.

[0049] In-vehicle AI control system: used to analyze received emotions using in-vehicle AI;

[0050] In the in-vehicle AI control system, the data obtained by the wireless EEG sensor and the emotions initially determined by facial expressions are used for secondary judgment to determine the driver's emotions;

[0051] The specific method of converting EEG signal changes into language information is:

[0052] Step 1: Clean the collected EEG signal to remove artifacts and noise caused by muscle activity and blinking;

[0053] Step 2: Extract time domain features from the preprocessed signal.

[0054] Among them, time domain features include mean, variance, kurtosis, and skewness;

[0055] Step 3: Perform Fourier transform on the signal, analyze its spectrum characteristics, and extract power spectrum density and frequency bandwidth characteristics;

[0056] Step 4: Analyze the temporal and spatial distribution patterns of EEG signals to extract features for emotional state recognition;

[0057] Step 5: Use machine learning algorithms to train the extracted features and generate an EEG change model;

[0058] Step 6: Design a verbal feedback template based on the predicted EEG change model. For example, if the driver is detected to be in a nervous state, design the following template: “Please note that you seem a little nervous. Please try to take a deep breath and relax.

[0059] Step 7: wherein, using a text-to-speech engine to convert the generated language text into a speech signal; selecting speech synthesis parameters, which include speech rate, intonation, and volume;

[0060] The preliminary emotion based on facial expression judgment and EEG signal results are integrated and a comprehensive judgment is made using the Bayesian network algorithm. The specific method of comprehensive judgment is as follows:

[0061] Based on the preliminary judgment results of facial expressions and EEG data, a Bayesian network structure is constructed using a structural learning method. Within the determined Bayesian network structure, a Bayesian estimation learning method is used to calculate the conditional probability table of each node. Within the Bayesian network, an inference algorithm is used to calculate the posterior probability of each emotion or emotional state based on the observed facial expressions and EEG data features. The emotional state with the largest posterior probability is selected as the comprehensive judgment result.

[0062] Based on the comprehensive judgment results, the driver's emotions are divided into positive emotions and negative emotions; positive emotions include happiness, satisfaction, and calmness; negative emotions include sadness, anger, and anxiety.

[0063] Emotion Regulation Module: This module transmits the processed conversation content back to the driver through bone conduction via "special glasses";

[0064] In the emotion regulation module, the special glasses integrate a bone conduction audio unit located at the back of the glasses. The bone conduction audio unit emits sound waves through a vibrating membrane, acting directly on the skull. The received information is converted into a specific audio signal, which is then vibrated by the bone conduction unit. The driver receives the vibration signal through bone conduction, and these signals are transmitted to the cochlea and perceived as sound. The sound feedback from bone conduction does not interfere with the sound of the driver's surrounding environment. The special glasses can transmit heart rate, blood pressure, and respiratory status data to the in-vehicle AI.

[0065] When the driver's emotion is positive, the vehicle system remains silent;

[0066] When the driver's mood is negative, the in-vehicle system plays relaxing, cheerful and dynamic music to further enhance the pleasure of driving; the system can send positive voice prompts to the user through the bone conduction audio unit, such as "Your driving performance today is great!" "Let's enjoy this journey!" Such information enhances the driver's good state; when the driver is in a positive mood, the in-vehicle system adjusts the interior lighting and seat massage to create a more comfortable driving environment, making the driving experience more relaxed and enjoyable; in the navigation prompts, a relaxed and pleasant tone is used, and humorous guidance is added to reduce the driver's psychological pressure and enhance driving pleasure.

[0067] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An in-vehicle system for providing emotional value based on AI, characterized in that: include: Driver facial expression acquisition module: used to capture facial expressions through the camera and detect brain electrical activity through the wireless EEG sensor, and transmit the obtained information to the emotion diagnosis module; Emotion diagnosis module: used to diagnose preliminary emotions from the received information and send the diagnosed preliminary emotions to the on-board AI control system; In-vehicle AI control system: used to analyze the received emotions using in-vehicle AI; Emotion Regulation Module: It is used to transmit the processed content of the conversation to the driver through "special glasses" and then through bone conduction.

2. The vehicle-mounted system for providing emotional value based on AI according to claim 1, characterized in that: In the driver's facial expression acquisition module, the camera captures facial expressions and carefully analyzes the movement of eyebrows, the expression of eyes, the curvature of lips and the subtle changes of facial muscles to acquire the emotional information conveyed by the passenger's face at 180 degrees; The wireless EEG sensor is installed on the pillow, and the brain electrical activity signal is obtained through the wireless EEG sensor, and the signal changes of the EEG are converted into language information; The monitoring of electrical activity is done by capturing brain waves of different frequencies, including alpha waves, beta waves, and delta waves; for example, an increase in alpha waves is often associated with relaxation and pleasure, while beta waves are associated with high concentration and tension.

3. The vehicle-mounted system for providing emotional value based on AI according to claim 1, characterized in that: In the emotion diagnosis module, the driver's initial emotion is firstly judged by facial expressions, which are divided into the raising of eyebrows, the expression of eyes, the curvature of lips and the changes of facial muscles; Eyebrow movements can be divided into eyebrow raising, eyebrow locking, and other eyebrow fluctuations; among them, the specific methods for judging the eyebrow state are: Raised eyebrows indicate surprise. When a driver encounters unexpected information or events, his eyebrows will be raised upwards, forming a surprised expression. This expression will be accompanied by wide eyes and a slightly open mouth. This is a preliminary positive emotion. Frowning eyebrows indicates dissatisfaction. When eyebrows gather in the middle to form a frowning expression, it means that the individual is experiencing negative emotions and is dissatisfied with what happened. This is the initial negative emotion. Other eyebrow fluctuations indicate the driver's state of pleasure, which is accompanied by a smile and friendly eye contact during a conversation; this is a preliminary positive emotion.

4. The vehicle-mounted system for providing emotional value based on AI according to claim 3, characterized in that: The expressions of the eyes can be divided into flickering and dull eyes. The specific methods for judging the expressions of the eyes are: Eye flickering indicates joy; when the driver feels joy, his eyes become bright and flickering, reflecting an active inner state; this is a preliminary positive emotion; Dull eyes indicate fatigue, which is a negative emotion; when the driver's eyes become dull and lack focus, it indicates that the current emotional state is low and lacks motivation, which is an initial negative emotion.

5. The vehicle-mounted system for providing emotional value based on AI according to claim 3, characterized in that: The curvature of the lips is divided into upturned corners of the mouth and tightly pursed corners of the mouth. The specific method for judging the curvature of the lips is: The upturned corners of the mouth indicate happiness; when the corners of the mouth are bent upward, it means that the driver is experiencing a positive emotional state and is satisfied with the current event; this is a preliminary positive emotion; Tightly pursed lips indicate dissatisfaction. When the lips are tightly pursed together, it means that the driver is experiencing some negative emotions and is dissatisfied with the current event. This is a preliminary negative emotion. Drooping mouth corners indicate sadness; when the mouth corners are bent downward, it indicates that the driver is experiencing a negative emotional state and is disappointed with the current event; this indicates preliminary negative emotions.

6. The vehicle-mounted system for providing emotional value based on AI according to claim 3, characterized in that: Facial muscle changes are divided into facial tension and facial relaxation. The specific methods of facial muscle changes are: Facial tension means that when the driver is nervous, the facial muscles become tense and stiff; this state of tension is manifested by the frowning of eyebrows, squinting of eyes and tight lip closure; this is a preliminary negative emotion; Facial relaxation means that when the driver is in a happy state, the facial muscles will become soft and relaxed; this state of relaxation may be manifested by the stretching of eyebrows, the brightness of eyes and the upturned corners of the mouth; this is an initial positive emotion; By analyzing the movement of eyebrows, the expression of eyes, the curvature of lips and changes in facial muscles, the driver's initial emotional state can be judged.

7. The vehicle-mounted system for providing emotional value based on AI according to claim 1, characterized in that: In the vehicle-mounted AI control system, the data obtained by the wireless EEG sensor and the emotions initially judged by facial expressions are used for secondary judgment to determine the driver's emotions: The specific method of converting EEG signal changes into language information is: Step 1: Clean the collected EEG signals to remove artifacts and noise caused by muscle activity and blinking; Step 2: Extract time domain features from the preprocessed signal. Among them, the time domain features include mean, variance, kurtosis, and skewness; Step 3: Perform Fourier transform on the signal, analyze its spectrum characteristics, and extract power spectrum density and frequency bandwidth characteristics; Step 4: Analyze the distribution pattern of EEG signals in time and space to extract features for emotional state recognition; Step 5: Use machine learning algorithms to train the extracted features and generate an EEG change model; Step 6: Design a language feedback template based on the predicted EEG change model; Step 7: Convert the generated language text into speech signals using a text-to-speech engine.

8. The vehicle-mounted system for providing emotional value based on AI according to claim 7, characterized in that: The preliminary emotion based on facial expression judgment and the EEG signal results are integrated and a comprehensive judgment is made using the Bayesian network algorithm. The specific method of comprehensive judgment is as follows: Based on the preliminary judgment results of facial expressions and EEG data, the Bayesian network structure is constructed using the structural learning method; under the determined Bayesian network structure, the conditional probability table of each node is calculated using the Bayesian estimation learning method; In the Bayesian network, an inference algorithm is used to calculate the posterior probability of each emotion or emotional state based on the observed facial expressions and EEG data features; the emotional state with the largest posterior probability is selected as the comprehensive judgment result; Based on the comprehensive judgment results, the driver's emotions are divided into positive emotions and negative emotions; positive emotions include happiness, satisfaction, and calmness; negative emotions include sadness, anger, and anxiety.

9. The vehicle-mounted system for providing emotional value based on AI according to claim 1, characterized in that: In the emotion regulation module, the special glasses integrate a bone conduction audio unit, which is located at the back of the glasses; the bone conduction audio unit emits sound waves through a vibrating membrane, directly acting on the skull; the received information is converted into a specific audio signal, which generates vibrations through the bone conduction unit; the driver receives the vibration signals through bone conduction, and these signals are transmitted to the cochlea and perceived as sound; the sound feedback from bone conduction does not interfere with the sound of the driver's surrounding environment; the special glasses can transmit the detected heart rate, blood pressure, and breathing status data to the in-vehicle AI; When the driver's emotion is positive, the vehicle system remains silent; When the driver's emotions are negative, the system sends music and voice prompts to the user through the bone conduction audio unit.

Images