Driver emotion regulation method, device and storage medium

Abstract

This application discloses a method, device, and storage medium for driver emotion regulation, belonging to the field of vehicle control technology. The method includes: acquiring the power of theta and beta waves in purified EEG signals and the pure PPG waveform of the driver's facial ROI in infrared video; using the ratio of the theta wave power to the beta wave power as a fatigue index; calculating a stress index based on the pure PPG waveform of the facial ROI; calculating the product of blink frequency and grip force fluctuation coefficient on the steering wheel as attention distraction; determining music parameters for regulating the driver's emotions based on the fatigue index, stress index, and attention distraction; combining the music parameters, data on the vehicle's environment, and the driver's historical music preferences to generate a music prompt text for regulating the driver's emotions and converting it into buffered audio of a certain duration; arranging the buffered audio in the order of generation in a playlist and playing it, thereby performing effective emotion regulation.

Description

Technical Field

[0001] This application relates to the field of vehicle control technology, and in particular to a method, device and storage medium for regulating a driver's emotions. Background Technology

[0002] With the rapid development of intelligent vehicle technology, driver emotion management has become a key area for improving driving safety and the driving experience. In scenarios such as long-distance driving or traffic congestion, drivers are prone to negative emotions such as fatigue and anxiety, leading to distraction and decreased reaction time, thus increasing the risk of traffic accidents.

[0003] In related technologies, when providing emotional therapy to drivers, video images of the driver are captured and analyzed to determine their emotions. These images are then matched with tracks from a pre-set music library for emotional regulation. However, this approach suffers from limitations in accuracy. reliance on video images alone for emotional analysis and excessive dependence on music for intervention is simplistic and insufficiently effective. Therefore, accurately identifying driver emotions and implementing effective emotional regulation measures is crucial for improving driving safety and experience. Summary of the Invention

[0004] This application provides a method, device, and storage medium for regulating a driver's emotions, which can be used to improve driving safety and experience. The technical solution is as follows: On the one hand, embodiments of this application provide a method for regulating a driver's emotions, the method comprising: Acquire the driver's purified EEG signal, infrared video, blink frequency, grip force fluctuation coefficient on the vehicle's steering wheel, and data on the vehicle's environment; The power of theta waves and the power of beta waves in the purified EEG signal are obtained, wherein the frequency range of theta waves is lower than that of the beta waves; The pure PPG (Photo Plethysmo Graphy) waveform of the driver's face ROI (Region of Interest) is obtained from the infrared video. The ROI indicates the specific area of ​​the driver's face that is most suitable for extracting the PPG. The PPG waveform is used to reflect the periodic changes in the subcutaneous capillary blood volume of the driver with the heartbeat. The driver's fatigue index is obtained by calculating the ratio of the power of the theta wave to the power of the beta wave. The driver's stress index was calculated based on the pure PPG waveform of the driver's facial ROI. The driver's attentional distraction is obtained by multiplying the driver's blink frequency by the grip force fluctuation coefficient on the vehicle's steering wheel. Music parameters for regulating the driver's mood are determined based on the driver's fatigue index, the driver's stress index, and the driver's distractibility. By combining the music parameters, the data of the vehicle's environment, and the driver's historical music preferences, a music prompt text is generated to regulate the driver's emotions. The music prompt text includes sound effects, instruments used, tempo, frequency, and neuromodulation stimulation signals. A buffered audio of a certain duration is generated based on the music Prompt text; The buffered audio is arranged in the order it was generated in the playlist, and the music in the playlist is played in that order.

[0005] On the other hand, a driver's emotion regulation device is provided, the device comprising: The first acquisition module is used to acquire the driver's purified EEG signal, infrared video, blink frequency, grip force fluctuation coefficient on the vehicle's steering wheel, and data on the vehicle's environment. The second acquisition module is used to acquire the power of theta waves and the power of beta waves in the purified EEG signal, wherein the frequency range of theta waves is lower than the frequency range of the beta waves. The third acquisition module is used to acquire the pure PPG waveform of the driver's facial ROI from the infrared video. The ROI indicates the specific area of ​​the driver's face that is most suitable for extracting the PPG. The PPG waveform is used to reflect the periodic changes in the subcutaneous capillary blood volume of the driver with the heartbeat. The first calculation module is used to calculate the ratio of the power of the theta wave to the power of the beta wave to obtain the driver's fatigue index. The second calculation module is used to calculate the driver's stress index based on the pure PPG waveform of the driver's facial ROI; The third calculation module is used to calculate the product of the driver's blinking frequency and the grip force fluctuation coefficient on the vehicle steering wheel to obtain the driver's attention distraction. The first determining module is used to determine music parameters for regulating the driver's emotions based on the driver's fatigue index, the driver's stress index, and the driver's attention distraction. The first generation module is used to combine the music parameters, the data of the vehicle's environment, and the driver's historical music preferences to generate a music prompt text for regulating the driver's emotions. The music prompt text includes sound effects, instruments used, tempo, frequency, and neuromodulation stimulation signals. The second generation module is used to generate a buffered audio of a certain duration based on the music Prompt text; The playback module is used to arrange the buffered audio in the playlist according to the generation order, and to play the music in the playlist in sequence.

[0006] On the other hand, a non-transitory computer-readable storage medium is also provided, characterized in that the computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement any of the above-described driver emotion regulation methods.

[0007] On the other hand, a computer program product is also provided, the computer program product including computer instructions, which, when executed by a processor, implement the steps of any of the above-described driver emotion regulation methods.

[0008] The technical solution provided in this application brings at least the following beneficial effects: This application acquires purified EEG signals from the driver, extracting the power of theta waves and beta waves; acquires infrared video of the driver, obtaining the purified PPG waveform of the driver's facial ROI; calculates the driver's attentional distraction by acquiring blink frequency and grip force fluctuation coefficient on the vehicle's steering wheel; calculates the driver's fatigue index based on the power of theta waves and beta waves; and calculates the driver's stress index based on the purified PPG waveform of the driver's facial ROI. Based on the driver's fatigue index, stress index, and attentional distraction, music parameters for regulating the driver's emotions are determined. Combining data from the vehicle's environment and the driver's historical music preferences, a music prompt text for regulating the driver's emotions is generated. A buffered audio of a certain duration is then generated based on the music prompt text and played in the generated order. This achieves accurate identification of the driver's emotions and corresponding effective emotion regulation operations, thereby improving driving safety and experience. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application; Figure 2 This is a flowchart of a driver's emotion regulation method provided in an embodiment of this application; Figure 3This is a schematic diagram of the structure of a driver's emotion regulation device provided in an embodiment of this application. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0012] This application provides a method for regulating a driver's emotions. Please refer to the following embodiments. Figure 1 The diagram illustrates the implementation environment of the method provided in this application embodiment. This implementation environment may include: a CDC (Cockpit Domain Controller) 11, a steering wheel-embedded flexible dry electrode 12, a headrest active pressure textile electrode 13, an IMU (Inertial Measurement Unit) 14, an EPS (Electric Power Steering Control Unit) 15, an in-vehicle infrared camera 16, a cockpit camera 17, a steering wheel grip force sensor 18, a carbon dioxide sensor 19, a temperature sensor 20, a humidity sensor 21, a bandpass filter 22, an in-vehicle audio system 23, an in-vehicle fragrance device 24, and a large screen on the center console 25.

[0013] Optionally, the steering wheel embedded flexible dry electrode 12 is embedded in the grip portion or spoke area of ​​the steering wheel; the headrest active pressure textile electrode 13 is integrated on the front of the driver's headrest to collect the driver's raw EEG signals in a collaborative manner and send them to the CDC11; the IMU14 is used to acquire the longitudinal acceleration of the vehicle and send it to the CDC11; and the EPS15 is used to acquire the steering wheel angle of the vehicle and send it to the CDC11.

[0014] For example, the infrared camera 16 and the cockpit camera 17 can be installed at the front of the vehicle facing the driver's face. The vehicle-mounted infrared camera 16 is used to acquire infrared video and send it to CDC11; the cockpit camera 17 is used to acquire video of the driver's face, which is then recognized by the image recognition device and sent to CDC11; the steering wheel grip force sensor 18 is used to acquire the grip force fluctuation coefficient on the vehicle's steering wheel and send it to CDC11; the carbon dioxide sensor 19 is used to acquire the carbon dioxide concentration of the vehicle's environment and send it to CDC11; the temperature sensor 20 is used to acquire the temperature of the vehicle's environment and send it to CDC11; and the humidity sensor 21 is used to acquire the humidity of the vehicle's environment and send it to CDC11.

[0015] In one possible implementation, a bandpass filter 22 is used to receive the purified EEG signal sent by CDC11, extract theta waves and beta waves from the purified EEG signal and send them to CDC11; a car audio system 23 is used to receive instructions from CDC11 and play music from the playlist in sequence; a car fragrance device 24 is used to receive instructions from CDC11 and control the release of fragrance of the desired fragrance type according to the target fragrance concentration; and a large screen 25 on the center console is used to receive the driver's manually input of the physiological response intensity to the fragrance and the manually selected type, proportion and concentration of the fragrance to be released.

[0016] Optionally, the CDC11, the steering wheel embedded flexible dry electrode 12, the headrest active pressure textile electrode 13, the IMU 14, the EPS 15, the vehicle infrared camera 16, the cockpit camera 17, the steering wheel grip force sensor 18, the carbon dioxide sensor 19, the temperature sensor 20, the humidity sensor 21, the bandpass filter 22, the vehicle audio system 23, the vehicle fragrance device 24, and the large screen 25 of the center console establish a communication connection via wired or wireless network.

[0017] Based on the above Figure 1 The implementation environment shown in this application provides a method for regulating a driver's emotions, such as... Figure 2 As shown, taking the application of this method to CDC as an example, the method includes steps 201-210.

[0018] In step 201, the CDC acquires the driver's purified EEG signal, infrared video, blink frequency, grip force fluctuation coefficient on the vehicle's steering wheel, and data on the vehicle's environment. In one possible implementation, the CDC acquires the driver's purified EEG signal, including: collecting the driver's original EEG signal, the vehicle's longitudinal acceleration, and the steering wheel angle; and correcting the original EEG signal based on the longitudinal acceleration and steering wheel angle to obtain the purified EEG signal. Optionally, the CDC may acquire the driver's original EEG signal in a manner including, but not limited to, using a combination of a flexible dry electrode embedded in the steering wheel and an actively pressurized textile electrode in the headrest.

[0019] For example, the steering wheel embedded flexible dry electrode is embedded in the grip part or spoke area of ​​the steering wheel; the headrest active pressure textile electrode is integrated on the front of the driver's seat headrest, wherein active pressure is to apply pressure to the flexible contact surface of the electrode, so that the headrest produces a displacement towards the back of the driver's head, avoiding poor contact caused by the driver's head movement or other reasons.

[0020] In one possible implementation, the CDC obtains the vehicle's longitudinal acceleration and steering wheel angle in ways including, but not limited to: the CDC obtains the vehicle's longitudinal acceleration through the IMU and obtains the vehicle's steering wheel angle through the EPS.

[0021] Optionally, after acquiring the driver's raw EEG signal, the vehicle's longitudinal acceleration, and the steering wheel angle, the CDC (Chronic Electroencephalogram) modifies the raw EEG signal based on the longitudinal acceleration and steering wheel angle. This includes: the CDC calculates motion compensation parameters based on the longitudinal acceleration and steering wheel angle, and then calculates the driver's raw EEG signal after removing the motion compensation parameters, resulting in a purified EEG signal. The formula for calculating the motion compensation parameters based on the longitudinal acceleration and steering wheel angle can be determined experimentally beforehand.

[0022] For example, the methods by which the CDC acquires infrared video, blink frequency, grip force fluctuation coefficient on the vehicle steering wheel, and data on the vehicle's environment include, but are not limited to: the CDC acquiring infrared video through an onboard infrared camera; acquiring video of the driver's face through a cockpit camera, and identifying the acquired video of the driver's face through an image recognition device to obtain the driver's blink frequency; acquiring the grip force fluctuation coefficient on the vehicle steering wheel through a steering wheel grip force sensor; acquiring the carbon dioxide concentration in the vehicle's environment through a carbon dioxide sensor; acquiring the temperature in the vehicle's environment through a temperature sensor; and acquiring the humidity in the vehicle's environment through a humidity sensor.

[0023] In one possible implementation, the infrared camera and cockpit camera can be mounted at the front of the vehicle facing the driver's face; the steering wheel grip force sensor is integrated inside the steering wheel; and the carbon dioxide sensor, temperature sensor, and humidity sensor are mounted inside the vehicle.

[0024] In step 202, the CDC acquires the power of theta waves and beta waves in the purified EEG signal, with the frequency range of theta waves being lower than that of the beta waves.

[0025] Optionally, after acquiring the purified EEG signal from the driver, the CDC acquires the power of theta waves and beta waves in the purified EEG signal, wherein the frequency range of theta waves is lower than that of the beta waves. For example, theta waves indicate low-frequency EEG signals with a frequency range of 4 Hz to 8 Hz, and beta waves indicate mid-to-high-frequency EEG signals with a frequency range of 12 Hz to 30 Hz.

[0026] For example, the CDC obtains the power of theta waves and beta waves in the purified EEG signal, including: the CDC extracts theta waves and beta waves through a bandpass filter; calculates the sum of squares of the amplitude of theta waves at each sampling point, and divides the calculation result by the number of sampling points to obtain the power of theta waves; then calculates the sum of squares of the amplitude of the beta waves at each sampling point, and divides the calculation result by the number of sampling points to obtain the power of the beta waves.

[0027] In step 203, the CDC acquires the clean PPG waveform of the driver's facial ROI from the infrared video. The ROI indicates the specific area of ​​the driver's face that is best suited for PPG extraction. The PPG waveform is used to reflect the periodic changes in the driver's subcutaneous capillary blood volume with heartbeats.

[0028] In one possible implementation, after acquiring infrared video, the CDC obtains the clean PPG waveform of the driver's facial ROI from the infrared video, where the ROI indicates the specific area of ​​the driver's face most suitable for PPG extraction, and the PPG waveform is used to reflect the periodic changes in the driver's subcutaneous capillary blood volume with heartbeats.

[0029] Optionally, the CDC obtains a clean PPG waveform of the driver's facial ROI from the infrared video, including: determining the driver's facial ROI in the infrared video using an adaptive ROI extraction algorithm; extracting the original signal sequence of the average brightness of pixels within the driver's facial ROI over time from the infrared video; and denoising the original signal sequence to obtain a clean PPG waveform.

[0030] For example, the CDC inputs the infrared video into a pre-trained ROI extraction model to extract the driver's face ROI from the infrared video; then it crops out the ROI in each frame of the infrared video to obtain an ROI video sequence; and calculates the average brightness of each frame of the ROI video sequence to obtain the original signal sequence of the average brightness of the pixels in the driver's face ROI changing over time.

[0031] In one possible implementation, after obtaining the original signal sequence, the CDC performs denoising processing on the original signal sequence, including: performing a discrete wavelet transform on the original signal sequence to decompose the signal into one approximation coefficient and six detail coefficients, where the approximation coefficient and detail coefficients represent signal components in different frequency ranges; then comparing the detail coefficients with a coefficient threshold and setting the detail coefficients below the coefficient threshold to zero to obtain the processed detail coefficients; and using an inverse discrete wavelet transform to reconstruct the approximation coefficients and the processed detail coefficients into a time-domain signal, thus obtaining a clean PPG waveform.

[0032] In step 204, CDC calculates the ratio of the power of the theta wave to the power of the beta wave to obtain the driver's fatigue index.

[0033] Optionally, after obtaining the power of theta waves and beta waves in the purified EEG signal, the CDC divides the power of theta waves by the power of the beta waves, and the result is used as the driver's fatigue index.

[0034] Since the power of theta waves increases when the human brain is fatigued, while the power of beta waves decreases, the ratio of the power of theta waves to the power of beta waves increases as the degree of human brain fatigue increases. Therefore, the driver's fatigue level can be measured by calculating the power of theta waves divided by the power of beta waves.

[0035] In step 205, the CDC calculates the driver's stress index based on the clean PPG waveform of the driver's facial ROI.

[0036] For example, after obtaining the clean PPG waveform, the CDC calculates the driver's stress index based on the clean PPG waveform of the driver's facial ROI, including: calculating the driver's RMSSD (Root Mean Square of Successive Differences) based on the clean PPG waveform of the driver's facial ROI; and calculating the reciprocal of the RMSSD as the driver's stress index.

[0037] In one possible implementation, the driver's RMSSD is calculated based on the clean PPG waveform of the driver's facial ROI, including: determining the position of all R peaks in the clean PPG waveform using a peak detection algorithm; calculating the time interval between every two adjacent R peaks to obtain a series of RR interval values; calculating the absolute difference between adjacent RR interval values; and calculating the root mean square of the absolute difference to obtain the driver's RMSSD. Optionally, after calculating the driver's RMSSD, the reciprocal of the RMSSD is calculated, and the result is used as the driver's stress index.

[0038] Because RMSSD is higher in a state of physiological relaxation and lower in a state of physiological anxiety, using the reciprocal of RMSSD as the driver's stress index allows the stress index data to intuitively reflect the driver's stress level. For example, when the driver's stress index increases, it indicates that the driver's stress has increased; when the driver's stress index decreases, it indicates that the driver's stress has decreased.

[0039] In step 206, the CDC calculates the product of the driver's blink frequency and the grip force fluctuation coefficient on the vehicle's steering wheel to obtain the driver's attention distraction.

[0040] For example, after obtaining the driver's blink frequency and the grip force fluctuation coefficient on the vehicle's steering wheel, the CDC calculates the product of the driver's blink frequency and the grip force fluctuation coefficient on the vehicle's steering wheel to obtain the driver's attention distraction, including: the CDC multiplies the driver's blink frequency by the grip force fluctuation coefficient on the vehicle's steering wheel and uses the calculation result as the driver's attention distraction.

[0041] Because fatigue leads to decreased eyelid muscle tone and increased blinking frequency; and when drivers are distracted, they may not operate the steering wheel for a period of time before making a large operation, resulting in a significant increase in the grip force fluctuation coefficient; therefore, the product of the driver's blinking frequency and the grip force fluctuation coefficient on the vehicle's steering wheel is used to measure the driver's distractibility.

[0042] In step 207, the CDC determines music parameters for regulating the driver's mood based on the driver's fatigue index, driver's stress index, and driver's distractibility.

[0043] In one possible implementation, after acquiring the driver's fatigue index, stress index, and distractibility, the CDC determines music parameters for regulating the driver's mood based on these parameters. These parameters include: comparing the driver's fatigue index to a fatigue threshold, the driver's stress index to a stress threshold, and the driver's distractibility to a distractibility threshold; if the driver's fatigue index is greater than the fatigue threshold, the music parameters for regulating the driver's mood indicate that the music rhythm be increased by a preset number of beats based on a preset baseline; if the driver's stress index is greater than the stress threshold, the music parameters for regulating the driver's mood indicate that the main instrument in the music should be a soothing instrument; and if the driver's distractibility is greater than the distractibility threshold, the music parameters for regulating the driver's mood indicate that guiding beats need to be added to the music.

[0044] Optionally, the fatigue threshold, stress threshold, and attention distraction threshold can be set based on experience. For example, the fatigue threshold can be 0.8, the stress threshold can be 0.7, and the attention distraction threshold can be 0.6. The preset baseline level and preset beat count can be set based on experience. For example, the preset beat count can be 40 BPM (Beats Per Minute). Soothing instruments include, but are not limited to, the cello.

[0045] For example, when a driver's fatigue index is greater than the fatigue threshold, increasing the preset number of beats in the music rhythm can help the driver resist drowsiness; when a driver's stress index is greater than the stress threshold, selecting a soothing instrument as the main instrument can help alleviate anxiety through low-frequency vibrations; when a driver's distractibility is greater than the distractibility threshold, adding guiding beats can help the driver enhance focus.

[0046] In step 208, the CDC combines music parameters, data on the vehicle's environment, and the driver's historical music preferences to generate a music prompt text for regulating the driver's emotions. The music prompt text includes sound effects, instruments used, tempo, frequency, and neuromodulation stimulation signals.

[0047] In one possible implementation, after determining the music parameters used to regulate the driver's emotions, the CDC combines the music parameters, data on the vehicle's environment, and the driver's historical music preferences to generate a music prompt text for regulating the driver's emotions. The music prompt text includes sound effects, instruments used, tempo, frequency, and neuromodulation stimulation signals.

[0048] Optionally, the driver's historical music preferences can be summarized by the types of music the driver has played in the past, or they can be set manually by the driver. Historical music preferences include, but are not limited to, the types of instruments the driver prefers, overlaid sound effects, and sound scenarios.

[0049] For example, the CDC combines music parameters, vehicle environment data, and the driver's historical music preferences to generate music prompt text for regulating the driver's emotions. This includes: the CDC combines music parameters, vehicle environment data, and the driver's historical music preferences, and based on the correspondence between music parameters, vehicle environment data, and the driver's historical music preferences and sound effects, rhythm, instruments used, speed, frequency, and neuromodulation stimulation signals, to build a basic framework with rhythm and instruments used, speed, and frequency, inject neuromodulation stimulation signals, and superimpose sound effects and sound scenarios to generate music prompt text for regulating the driver's emotions.

[0050] In one possible implementation, the correspondence between music parameters, vehicle environment data, and the driver's historical music preferences and sound effects, rhythm, instruments used, speed of sound, frequency, and neural modulation stimulation signals can be set empirically. When determining sound effects, rhythm, instruments used, speed of sound, frequency, and neural modulation stimulation signals, the driver's historical music preferences take precedence over the music parameters and vehicle environment data. Sound effects include, but are not limited to, birdsong, stream sounds, and wind chime sounds. Instruments used include, but are not limited to, cello, violin, piano, harp, harmonica, guzheng, xiao, and flute. Sound scenarios include, but are not limited to, raindrop sounds, wind sounds, and snow sounds.

[0051] In step 209, CDC generates a buffered audio of a certain duration based on the music Prompt text.

[0052] Optionally, after generating the Prompt text, CDC generates a buffered audio of a certain duration based on the music Prompt text, including: generating a Mel spectrum frame sequence based on the music Prompt text; restoring the Mel spectrum frame sequence to waveform audio of a specific sampling frequency for a certain duration, thus obtaining a buffered audio of a certain duration.

[0053] For example, after generating the Prompt text, the CDC inputs the Prompt text into a music language model, and the decoder of the music language model generates a Mel spectrum frame sequence, where the Mel spectrum frame sequence indicates the changes in the spectral content of the music used to regulate the driver's mood over time. In one possible implementation, after generating the Mel spectrum frame sequence, the CDC uses a vocoder to restore the Mel spectrum frame sequence to waveform audio at a specific sampling frequency for a certain duration, i.e., obtains buffered audio of a certain duration.

[0054] Optionally, the decoder is used to convert the Prompt text into a Mel-spectrum frame sequence; the vocoder is used to convert the Mel-spectrum frame sequence into waveform audio that can be heard by the human ear; a certain duration can be set empirically, for example, it can be set to 30 seconds; a specific sampling frequency can also be set empirically, for example, it can be set to a preferred sampling frequency of 44.1 kHz.

[0055] In one possible implementation, after generating a buffered audio of a certain duration, the CDC modulates the buffered audio with amplitude using a sinusoidal carrier signal within a specific frequency range. This specific frequency range corresponds to the frequency range of alpha brain waves experienced by the human brain in a relaxed state. For example, the amplitude modulation formula includes: Modulated buffered audio = Buffered audio × (1 + Gain coefficient × Sine carrier), where all values ​​used in the formula are amplitude values. Optionally, the frequency range of the alpha brain waves can be 8-12 Hz; the gain coefficient can be pre-calibrated.

[0056] By using a sinusoidal carrier signal within the frequency range of alpha brain waves to modulate the amplitude of buffered audio, the relaxing and therapeutic effects of music are enhanced from a physiological perspective.

[0057] In step 210, CDC arranges the buffered audio in the playlist according to the generation order, and plays the music in the playlist in order.

[0058] For example, after generating a buffered audio of a certain duration and completing amplitude modulation, the CDC arranges the buffered audio in the order of generation in the playlist, and plays the music in the playlist in sequence through the car audio system.

[0059] In one possible implementation, after playing music from the playlist in sequence, after a second duration, the power of alpha waves in the purified EEG signal is acquired. The frequency range of alpha waves is greater than that of theta waves but lower than that of beta waves. In response to the failure of alpha wave power to increase to a first degree, the driver's stress index not decreasing to a second degree, or the driver's distractibility not decreasing to a third degree, a detection result indicating that the driver's emotional regulation is not up to standard is obtained. In response to the detection result indicating that the driver's emotional regulation is not up to standard, the music parameters used to regulate the driver's emotions are optimized.

[0060] Optionally, the CDC counts the duration of music played in the playlist, and after a second duration, obtains the power of alpha waves in the purified EEG signal, continuously obtains the driver's stress index and driver's attention distraction; and calculates the degree of change in alpha wave power, the degree of change in the driver's stress index, and the degree of change in the driver's attention distraction; compares the degree of change in alpha wave power with the first degree, the degree of change in the driver's stress index with the second degree, and the degree of change in the driver's attention distraction with the third degree.

[0061] For example, if the power of the alpha wave is not increased to a first degree, the driver's stress index is not reduced to a second degree, or the driver's distraction is not reduced to a third degree, the CDC obtains a detection result indicating that the driver's emotion regulation is not up to standard, and optimizes the music parameters used to regulate the driver's emotion, including: if the power of the alpha wave is not increased to a first degree, increasing the gain coefficient by a preset multiple; if the driver's stress index is not reduced to a second degree, increasing the proportion of natural sound effects in the subsequent music parameters used to regulate the driver's emotion to a fourth degree; if the driver's distraction is not reduced to a third degree, reducing the tempo in the subsequent music parameters used to regulate the driver's emotion to a fifth degree.

[0062] In one possible implementation, the preset multiplier, the fourth level, and the fifth level are set based on experience, and the fifth level needs to be less than the fourth level. For example, the preset multiplier can be 0.2 times, the fourth level can be 30%, and the fifth level can be 10%.

[0063] Optionally, the system can also acquire the user's physiological response intensity to the fragrance, which is manually input by the user; select the fragrance type based on the driver's fatigue index, driver's stress index, and driver's attention distraction; calculate the target fragrance concentration based on the driver's fatigue index, driver's stress index, driver's attention distraction, data on the vehicle's environment, and the user's physiological response intensity to the fragrance; and control the fragrance release of the fragrance type according to the target fragrance concentration.

[0064] In one possible implementation, the user can manually input the intensity of the physiological response to the fragrance via the large screen on the central control panel, where the value of the physiological response intensity to the fragrance ranges from 0.3 to 0.7.

[0065] For example, after obtaining the user's input of the physiological response intensity to the fragrance, the CDC selects the fragrance type based on the driver's fatigue index, stress index, and distractibility, including: if the driver's fatigue index is greater than the fatigue threshold, the stress index is not greater than the stress threshold, and the driver's distractibility is not greater than the distractibility threshold, then the user's dominant emotion is fatigue, and rosemary is selected as the main fragrance and peppermint as the secondary fragrance; if the driver's stress index is greater than the stress threshold, the fatigue index is not greater than the fatigue threshold, and the driver's distractibility is not greater than the distractibility threshold, then the user's dominant emotion is anxiety, and lavender is selected as the main fragrance and bergamot as the secondary fragrance.

[0066] Optionally, after selecting the fragrance type, the CDC calculates the target fragrance concentration based on the driver's fatigue index, driver's stress index, driver's attention distraction, data on the vehicle's environment, and the user's physiological response intensity to the fragrance using the following formula: Target fragrance concentration = Base fragrance concentration × (1 + Physiological response intensity × Emotional intensity) × Environmental factor, where emotional intensity refers to the parameter among fatigue index, stress index, and driver's attention distraction that is greater than the corresponding threshold, and the environmental factor is determined by data on the vehicle's environment.

[0067] For example, the CDC compares the carbon dioxide concentration in the vehicle's environment with a carbon dioxide concentration threshold, the temperature in the vehicle's environment with a temperature threshold, and the humidity in the vehicle's environment with a humidity threshold. Optionally, if the carbon dioxide concentration in the vehicle's environment is greater than the carbon dioxide concentration threshold, but the temperature and humidity are not greater than the temperature threshold and humidity threshold, a first environmental factor is used; if the carbon dioxide concentration in the vehicle's environment is not greater than the carbon dioxide concentration threshold, but the temperature is greater than the temperature threshold and humidity is not greater than the humidity threshold, a second environmental factor is used; if the carbon dioxide concentration in the vehicle's environment is not greater than the carbon dioxide concentration threshold, but the temperature is not greater than the temperature threshold and humidity is greater than the humidity threshold, a third environmental factor is used.

[0068] In one possible implementation, the first environmental factor, the second environmental factor, and the third environmental factor can be set empirically. For example, the first environmental factor can be 0.7, the second environmental factor can be 0.8, and the third environmental factor can be 1.2.

[0069] For example, after calculating the target fragrance concentration, the CDC controls the in-vehicle fragrance device to release fragrance of the desired type according to the target fragrance concentration. Optionally, if fragrance control is required in situations other than those listed above, the driver can manually select the type, proportion, and concentration of fragrance to be released via the large screen on the center console.

[0070] In one possible implementation, the color, brightness, and flashing frequency of the breathing light can be determined based on the driver's fatigue index, driver's stress index, and driver's distraction level; the breathing light can then be controlled to flash according to its color, brightness, and flashing frequency.

[0071] Optionally, the CDC determines the color, brightness, and flashing frequency of the breathing light based on the driver's fatigue index, driver's stress index, and driver's distractibility, including: if the driver's stress index is greater than the stress threshold, the fatigue index is not greater than the fatigue threshold, and the driver's distractibility is not greater than the distractibility threshold, the breathing light is selected as blue light with a wavelength of 470-480 nm, the brightness is 30%, and the flashing frequency is 0.1 Hz; if the driver's stress index is not greater than the stress threshold, the fatigue index is not greater than the fatigue threshold, and the driver's distractibility is greater than the distractibility threshold, the breathing light is selected as green light with a wavelength of 520-530 nm, the brightness is 50%, and the flashing frequency is 0.3 Hz.

[0072] For example, after selecting the color, brightness, and flashing frequency of the breathing light, the CDC controls the vehicle breathing light to flash according to the color, brightness, and flashing frequency of the breathing light.

[0073] In one possible implementation, when a driver's physiological indicators are severely elevated, the CDC can also provide psychological counseling to the driver via in-vehicle intelligent voice control, and, with the driver's authorization, automatically send location information to a pre-set emergency contact. Optionally, the psychological counseling may include, but is not limited to, inquiring about the causes of stress, initiating voice reassurance, guiding cognitive restructuring, and providing breathing guidance. The physiological indicators include the driver's fatigue index, stress index, and attention deficit level.

[0074] This application embodiment acquires the purified EEG signal of the driver, extracting the power of theta waves and beta waves; acquires infrared video of the driver, obtaining the pure PPG waveform of the driver's facial ROI from the infrared video; calculates the driver's attention distraction by acquiring blink frequency and grip force fluctuation coefficient on the vehicle's steering wheel; calculates the driver's fatigue index based on the power of theta waves and beta waves; calculates the driver's stress index based on the pure PPG waveform of the driver's facial ROI; determines music parameters for regulating the driver's emotions based on the driver's fatigue index, stress index, and attention distraction, and generates a music prompt text for regulating the driver's emotions by combining data on the vehicle's environment and the driver's historical music preferences, thereby generating a buffered audio of a certain duration based on the music prompt text and playing it in the generated order. This achieves accurate identification of the driver's emotions and corresponding effective emotion regulation operations, thereby improving driving safety and experience.

[0075] See Figure 3 This application provides a driver's emotion regulation device, which includes: The first acquisition module 301 is used to acquire the driver's purified EEG signal, infrared video, blink frequency, grip force fluctuation coefficient on the vehicle steering wheel, and data of the vehicle's environment. The second acquisition module 302 is used to acquire the power of theta waves and the power of beta waves in the purified EEG signal, wherein the frequency range of theta waves is lower than that of the beta waves. The third acquisition module 303 is used to acquire the pure PPG waveform of the driver's facial ROI from the infrared video. The ROI indicates the specific area of ​​the driver's face that is most suitable for PPG extraction. The PPG waveform is used to reflect the periodic changes in the blood volume of the driver's subcutaneous capillaries with the heartbeat. The first calculation module 304 is used to calculate the ratio of the power of the theta wave to the power of the beta wave to obtain the driver's fatigue index. The second calculation module 305 is used to calculate the driver's stress index based on the pure PPG waveform of the driver's facial ROI. The third calculation module 306 is used to calculate the product of the driver's blinking frequency and the grip force fluctuation coefficient on the vehicle's steering wheel to obtain the driver's attention distraction. The first determining module 307 is used to determine music parameters for regulating the driver's mood based on the driver's fatigue index, driver's stress index and driver's distraction. The first generation module 308 is used to combine music parameters, data of the vehicle's environment, and the driver's historical music preferences to generate a music prompt text for regulating the driver's emotions. The music prompt text includes sound effects, instruments used, speed, frequency, and neuromodulation stimulation signals. The second generation module 309 is used to generate a buffered audio of a certain duration based on the music Prompt text; The playback module 310 is used to arrange the buffered audio in the playlist according to the generation order, and to play the music in the playlist in order.

[0076] In one possible implementation, the second generation module 309 is further configured to amplitude modulate the buffered audio using a sinusoidal carrier signal within a specific frequency range, the specific frequency range being the frequency range of alpha brain waves when the human brain is in a relaxed state.

[0077] In one possible implementation, the playback module 310 is further configured to, after a second duration, acquire the power of alpha waves in the purified EEG signal, wherein the frequency range of alpha waves is greater than the frequency range of theta waves and lower than the frequency range of beta waves; in response to the alpha wave power not increasing to a first degree, the driver's stress index not decreasing to a second degree, or the driver's attention distraction not decreasing to a third degree, acquire a detection result indicating that the driver's emotional regulation is not up to standard; and in response to the acquisition of the detection result indicating that the driver's emotional regulation is not up to standard, optimize the music parameters used to regulate the driver's emotions.

[0078] In one possible implementation, the second generation module 309 is used to generate a Mel spectrum frame sequence based on the music Prompt text, the Mel spectrum frame sequence indicating the change of the spectrum content of the music used to regulate the driver's mood over time; and to restore the Mel spectrum frame sequence to waveform audio of a specific sampling frequency for a certain duration to obtain buffered audio of a certain duration.

[0079] In one possible implementation, the first acquisition module 301 is used to acquire the driver's original EEG signal, the vehicle's longitudinal acceleration, and the steering wheel angle; and to correct the original EEG signal based on the longitudinal acceleration and the steering wheel angle to obtain a purified EEG signal.

[0080] In one possible implementation, the third acquisition module 303 is used to determine the driver's face ROI in the infrared video through an adaptive ROI extraction algorithm; extract the original signal sequence of the average brightness of the pixels within the driver's face ROI over time from the infrared video; and perform noise reduction processing on the original signal sequence to obtain a clean PPG waveform.

[0081] In one possible implementation, the second calculation module 305 is used to calculate the driver's RMSSD based on the clean PPG waveform of the driver's facial ROI; and calculate the reciprocal of the RMSSD as the driver's stress index.

[0082] This device acquires the driver's purified EEG signal, extracting the power of theta waves and beta waves; it acquires the driver's infrared video, obtaining the purified PPG waveform of the driver's facial ROI; it calculates the driver's attention distraction by acquiring blink frequency and the grip force fluctuation coefficient on the vehicle's steering wheel; it calculates the driver's fatigue index based on the power of theta waves and beta waves; and it calculates the driver's stress index based on the purified PPG waveform of the driver's facial ROI. Based on the driver's fatigue index, stress index, and attention distraction, it determines music parameters for regulating the driver's emotions. Combining data from the vehicle's environment and the driver's historical music preferences, it generates a music prompt text for regulating the driver's emotions. A buffered audio of a certain duration is then generated based on the music prompt text and played in the generated order. This allows for accurate identification of the driver's emotions and corresponding effective emotion regulation, thereby improving driving safety and experience.

[0083] It should be noted that the apparatus provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0084] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one computer program that is loaded and executed by a processor of a computer device to enable the computer to implement any of the above-described driver emotion regulation methods.

[0085] In one possible implementation, the aforementioned computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0086] In an exemplary embodiment, a computer program product or computer program is also provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the aforementioned driver emotion regulation methods.

[0087] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the driver's purified EEG signal, infrared video, blink frequency, grip force fluctuation coefficient on the vehicle's steering wheel, and data on the vehicle's environment involved in this application were all obtained with full authorization.

[0088] It should be understood that "multiple" as used in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0089] It should be noted that the terms "first," "second," etc. (if applicable) in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0090] The above description is merely an exemplary embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A method for regulating a driver's emotions, characterized in that, The method includes: Acquire the driver's purified EEG signal, infrared video, blink frequency, grip force fluctuation coefficient on the vehicle's steering wheel, and data on the vehicle's environment; The power of theta waves and the power of beta waves in the purified EEG signal are obtained, wherein the frequency range of theta waves is lower than that of the beta waves; The pure photoplethysmography (PPG) waveform of the region of interest (ROI) on the driver's face is obtained from the infrared video. The ROI indicates the specific area on the driver's face that is most suitable for extracting the PPG. The PPG waveform is used to reflect the periodic changes in the subcutaneous capillary blood volume of the driver with the heartbeat. The driver's fatigue index is obtained by calculating the ratio of the power of the theta wave to the power of the beta wave. The driver's stress index was calculated based on the pure PPG waveform of the driver's facial ROI. The driver's attentional distraction is obtained by multiplying the driver's blink frequency by the grip force fluctuation coefficient on the vehicle's steering wheel. Music parameters for regulating the driver's mood are determined based on the driver's fatigue index, the driver's stress index, and the driver's distractibility. By combining the music parameters, the data of the vehicle's environment, and the driver's historical music preferences, a music prompt text is generated to regulate the driver's emotions. The music prompt text includes sound effects, instruments used, tempo, frequency, and neuromodulation stimulation signals. A buffered audio of a certain duration is generated based on the music Prompt text; The buffered audio is arranged in the order it was generated in the playlist, and the music in the playlist is played in that order.

2. The method according to claim 1, characterized in that, After generating a buffered audio of a certain duration based on the music Prompt text, the method further includes: The buffered audio is amplitude modulated by a sinusoidal carrier signal within a specific frequency range, which is the frequency range of alpha brain waves when the human brain is in a relaxed state.

3. The method according to claim 1, characterized in that, After playing the music in the playlist in sequence, the method further includes: After a second duration, the power of the alpha wave in the purified EEG signal is obtained, wherein the frequency range of the alpha wave is greater than the frequency range of the theta wave and lower than the frequency range of the beta wave. In response to the fact that the power of the alpha wave does not increase to a first degree, the driver's stress index does not decrease to a second degree, or the driver's distraction does not decrease to a third degree, a detection result is obtained indicating that the driver's emotional regulation has not met the standard. In response to the detection result that the driver's emotion regulation did not meet the standard, the music parameters used to regulate the driver's emotion are optimized.

4. The method according to claim 1, characterized in that, The process of generating a buffered audio of a certain duration based on the music Prompt text includes: A Mel spectrum frame sequence is generated based on the music Prompt text, the Mel spectrum frame sequence indicating the changes in the spectral content of the music used to regulate the driver's mood over time; The Mel spectrum frame sequence is restored to a waveform audio with a specific sampling frequency for a certain duration, resulting in a buffered audio of a certain duration.

5. The method according to claim 1, characterized in that, The acquisition of the driver's purified EEG signal includes: The driver's raw electroencephalogram (EEG) signals, the vehicle's longitudinal acceleration, and the steering wheel angle were collected. The original EEG signal is corrected based on the longitudinal acceleration and the steering wheel angle to obtain the purified EEG signal.

6. The method according to claim 1, characterized in that, The step of obtaining the clean PPG waveform of the driver's facial ROI from the infrared video includes: The driver's facial ROI in the infrared video was determined using an adaptive ROI extraction algorithm; Extract the raw signal sequence of the average brightness of pixels within the driver's facial ROI over time from the infrared video; The original signal sequence is denoised to obtain the clean PPG waveform.

7. The method according to claim 1, characterized in that, The calculation of the driver's stress index based on the pure PPG waveform of the driver's facial ROI includes: The root mean square difference (RMSSD) between adjacent RR intervals of the driver is calculated based on the pure PPG waveform of the driver's facial ROI. The reciprocal of the RMSSD is calculated as the driver's stress index.

8. A driver's emotion regulation device, characterized in that, The device includes: The first acquisition module is used to acquire the driver's purified EEG signal, infrared video, blink frequency, grip force fluctuation coefficient on the vehicle's steering wheel, and data on the vehicle's environment. The second acquisition module is used to acquire the power of theta waves and the power of beta waves in the purified EEG signal, wherein the frequency range of theta waves is lower than the frequency range of the beta waves. The third acquisition module is used to acquire the pure PPG waveform of the driver's facial ROI from the infrared video. The ROI indicates the specific area of ​​the driver's face that is most suitable for extracting the PPG. The PPG waveform is used to reflect the periodic changes in the subcutaneous capillary blood volume of the driver with the heartbeat. The first calculation module is used to calculate the ratio of the power of the theta wave to the power of the beta wave to obtain the driver's fatigue index. The second calculation module is used to calculate the driver's stress index based on the pure PPG waveform of the driver's facial ROI; The third calculation module is used to calculate the product of the driver's blinking frequency and the grip force fluctuation coefficient on the vehicle steering wheel to obtain the driver's attention distraction. The first determining module is used to determine music parameters for regulating the driver's emotions based on the driver's fatigue index, the driver's stress index, and the driver's attention distraction. The first generation module is used to combine the music parameters, the data of the vehicle's environment, and the driver's historical music preferences to generate a music prompt text for regulating the driver's emotions. The music prompt text includes sound effects, instruments used, tempo, frequency, and neuromodulation stimulation signals. The second generation module is used to generate a buffered audio of a certain duration based on the music Prompt text; The playback module is used to arrange the buffered audio in the playlist according to the generation order, and to play the music in the playlist in sequence.

9. A computer program product comprising computer instructions that, when executed by a processor, implement the steps of the driver's emotion regulation method as described in any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which is loaded and executed by a processor to implement the driver's emotion regulation method as described in any one of claims 1 to 6.

Images