Driver emotion recognition method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a driver emotion recognition method and device, electronic equipment and a medium. The method comprises the steps that identity information of a driver is acquired, and personalized standard parameters corresponding to the driver are determined based on the identity information; acquiring facial information and physiological features of the driver, and performing data fusion on the facial information and the physiological features to obtain emotion state parameters corresponding to the driver; determining an emotion stability value of the driver based on the emotion state parameters; and determining a preset emotion stability threshold value corresponding to the driver based on the personalized standard parameters, and if the emotion stability value is lower than the preset emotion stability threshold value, determining an emotion adjustment strategy of the driver based on the facial information and the physiological features of the driver. According to the embodiment of the invention, a personalized emotion adjustment strategy can be formulated according to the facial information and physiological characteristics of the driver, so that the pressure of the driver is relieved, the mood is relaxed and the driving experience is improved.

Description

Technical Field

[0001] The present application relates to the technical field of emotion recognition, and in particular to a driver emotion recognition method, device, electronic device, and computer-readable storage medium. Background Art

[0002] Drivers need to concentrate highly when driving to ensure driving safety. However, when drivers have emotional problems, their attention may be distracted. For example, drivers may be distracted by emotions such as anger, happiness, and frustration, resulting in inability to pay attention to the situation ahead. In addition, drivers communicating or even arguing with people in the car can also distract their attention and increase driving risks.

[0003] Existing driver emotion detection systems usually rely on a single data source or shallow machine learning models, which leads to limited detection accuracy and real-time performance when faced with complex emotional states or environmental interference. In addition, the feedback mechanisms of these systems are often too simple to provide effective emotion adjustment strategies, which cannot meet the accuracy and efficiency of driver emotion detection. Summary of the invention

[0004] To solve the above technical problems, embodiments of the present application provide a driver emotion recognition method and device, an electronic device, a computer-readable storage medium, and a computer program product.

[0005] According to one aspect of an embodiment of the present application, a driver emotion recognition method is provided, including: obtaining the driver's identity information, and determining the personalized standard parameters corresponding to the driver based on the identity information; obtaining the driver's facial information and physiological characteristics, and fusing the facial information and the physiological characteristics to obtain the driver's corresponding emotional state parameters; determining the driver's emotional stability value based on the emotional state parameters; determining the driver's corresponding preset emotional stability threshold based on the personalized standard parameters, and if the emotional stability value is lower than the preset emotional stability threshold, determining the driver's emotional adjustment strategy based on the driver's facial information and the physiological characteristics.

[0006] According to one aspect of the embodiment of the present application, the identity information includes: gender, age and driving experience, and the determining of the personalized standard parameters corresponding to the driver based on the identity information includes: determining the personalized standard parameter library corresponding to the driver based on the age, and the personalized standard library includes multiple different personalized standard parameters; determining the target personalized standard parameters that match the driver from the multiple different personalized standard parameters based on the driving experience; determining the preset emotional stability threshold of the driver based on the target personalized standard parameters and the gender. According to one aspect of the embodiment of the present application, the physiological characteristics also include voice characteristics and physiological parameters, and the method also includes: obtaining the voice data output by the driver, and parsing the voice data to obtain the corresponding voice characteristics; obtaining the driver's heart rate and blood oxygen saturation, and determining the driver's physiological parameters based on the heart rate and blood oxygen saturation.

[0007] According to one aspect of an embodiment of the present application, the method also includes: performing multimodal fusion processing on the facial information, the voice features and the physiological parameters to obtain processed emotion data; and determining the emotion state parameters corresponding to the driver based on the processed emotion data and the personalized standard parameters.

[0008] According to one aspect of an embodiment of the present application, the facial information, the voice features and the physiological parameters are subjected to multimodal fusion processing to obtain processed emotion data, including: determining the weights corresponding to the facial information, the voice features and the physiological parameters based on a preset multimodal attention mechanism; calculating the weighted values ​​of the facial information, the voice features and the physiological parameters based on the weights, so as to determine the driver's emotion data based on the weighted values.

[0009] According to one aspect of an embodiment of the present application, the physiological parameters include blood oxygen saturation and heart rate, and the method also includes: if the blood oxygen saturation is lower than a preset saturation threshold and the heart rate is not lower than a preset heart rate threshold, it is determined that the driver has dangerous driving behavior and a corresponding prompt message is generated.

[0010] According to one aspect of an embodiment of the present application, the method also includes: obtaining voice data emitted by the driver within a preset time length; determining the volume fluctuation frequency, breathing fluctuation frequency and emotional fluctuation in the voice data; and determining the voice features based on the volume fluctuation frequency, the breathing fluctuation frequency and the emotional fluctuation.

[0011] According to one aspect of an embodiment of the present application, determining the driver's emotion adjustment strategy based on the driver's facial information and the physiological characteristics includes: obtaining temperature information in the cabin of the vehicle, and adjusting the target temperature in the cabin based on the temperature information; obtaining the driver's sitting posture information, and adjusting the target seat position in the cabin based on the sitting posture information; obtaining the vehicle's operating condition information, and adjusting the vehicle's early warning strategy based on the operating condition information; determining the driver's emotion adjustment strategy based on at least one of the target temperature, target seat position, and the early warning strategy.

[0012] According to one aspect of the embodiments of the present application, a driver emotion recognition device is provided, comprising: an acquisition module for acquiring the identity information of the driver, and determining the personalized standard parameters corresponding to the driver based on the identity information; a data integration module for acquiring the facial information and physiological characteristics of the driver, and fusing the facial information and the physiological characteristics to obtain the emotional state parameters corresponding to the driver; a determination module for determining the emotional stability value of the driver based on the emotional state parameters; an adjustment module for determining the preset emotional stability threshold corresponding to the driver based on the personalized standard parameters, and if the emotional stability value is lower than the preset emotional stability threshold, determining the emotional adjustment strategy of the driver based on the facial information and the physiological characteristics of the driver. According to one aspect of the embodiments of the present application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by the one or more processors, the electronic device implements the driver emotion recognition method as described above.

[0013] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a computer, the computer executes the driver emotion recognition method as described above.

[0014] According to one aspect of an embodiment of the present application, a computer program product is also provided, including a computer program, which implements the steps in the driver emotion recognition method as described above when executed by a processor.

[0015] In the technical solution provided in the embodiments of the present application, by obtaining the driver's identity information, personalized standard parameters can be set for him, which helps to more accurately assess the driver's driving ability and risks, and by integrating the driver's facial information and physiological characteristics, the driver's emotional state can be monitored in real time to reduce the risk of safety accidents caused by emotional problems. When the emotional stability value is lower than the preset emotional stability threshold, a personalized emotional adjustment strategy can be formulated based on the driver's facial information and physiological characteristics to relieve the driver's stress, relax his mood and improve the driving experience.

[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0018] Figure 1 is a schematic diagram of an implementation environment for driver emotion recognition during driving, shown in an exemplary embodiment of the present application;

[0019] Figure 2 is a flow chart of a driver emotion recognition method shown in an exemplary embodiment of the present application;

[0020] Figure 3 is a flowchart of a driver emotion recognition method shown in another exemplary embodiment of the present application;

[0021] Figure 4 is a flowchart of a driver emotion recognition method shown in another exemplary embodiment of the present application;

[0022] Figure 5 is a flowchart of a driver emotion recognition method shown in another exemplary embodiment of the present application;

[0023] Figure 6 is a flowchart of a driver emotion recognition method shown in another exemplary embodiment of the present application;

[0024] Figure 7 is a flowchart of a driver emotion recognition method shown in another exemplary embodiment of the present application;

[0025] Figure 8is a flowchart of a driver emotion recognition method shown in another exemplary embodiment of the present application;

[0026] Fig. 9 is a schematic diagram of a brief process of driver emotion recognition in an exemplary application scenario;

[0027] Fig.10 is a block diagram of a driver emotion recognition device shown in an exemplary embodiment of the present application;

[0028] Fig.11 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0029] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0030] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0031] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0032] The term "multiple" as used in this application refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0033] First of all, it should be noted that the importance of driver emotions in the driving process cannot be ignored. It is directly related to driving safety, driving efficiency and passenger comfort. The following is a detailed analysis of the importance of driver emotions from several aspects. Driving safety: The driver's emotional state directly affects his attention, reaction speed and judgment. For example, negative emotions such as anger, anxiety or fatigue may cause the driver to be distracted, make mistakes in judgment or operate improperly, thereby increasing the risk of traffic accidents. Positive emotions such as calmness and concentration help drivers stay awake and alert, better deal with various situations during driving, and ensure driving safety. Driving efficiency: The driver's emotional fluctuations may affect driving speed, route selection and compliance with traffic rules. Negative emotions may cause drivers to violate traffic rules, such as speeding and running red lights, which not only affects driving efficiency, but may also cause traffic jams and accidents. On the contrary, stable emotions help drivers plan driving routes reasonably, abide by traffic rules and improve driving efficiency. Passenger comfort: The driver's emotional state will also affect the passengers' riding experience. If the driver shows negative emotions such as irritability and anxiety, the passengers may feel uneasy or nervous, which will affect the riding comfort. On the contrary, if the driver is emotionally stable and friendly, the passengers' riding experience will be more pleasant. Mental health: Long-term negative emotions may cause drivers to have psychological problems, such as anxiety and depression, which not only affects the driver's physical and mental health, but also may pose a potential threat to driving safety. Therefore, drivers need to learn to regulate and manage their emotions and maintain mental health. Social responsibility: As a participant in road traffic, the driver's emotional state is not only related to personal safety, but also to the safety of other road users. Therefore, the driver has the responsibility to maintain a stable emotional state to reduce the occurrence of traffic accidents and maintain road traffic order.

[0034] In summary, driver emotions play a vital role in driving. In order to ensure driving safety, improve driving efficiency and passenger comfort, drivers need to learn to identify and manage their emotions and stay calm, focused and friendly.

[0035] Figure 1 FIG. 1 is a schematic diagram of an implementation environment for driver emotion recognition during driving, shown in an exemplary embodiment of the present application. Figure 1As shown, during the driving of the vehicle, the identity information of the driver is collected by the smart terminal 110 and the identity information of the driver is sent to the corresponding server 120. The server 120 determines the personalized standard parameters corresponding to the driver according to the identity information. In addition, the smart terminal 110 also collects the facial information and physiological characteristics of the driver, and transmits the collected facial information and physiological characteristics to the server 120. The server 120 performs data fusion based on the facial information and physiological characteristics of the driver to obtain the driver's emotional state parameters according to the result of data fusion, and then the driver's emotional stability value can be determined according to the emotional state, and the preset emotional stability threshold of the driver can be determined according to the personalized standard corresponding to the driver. If the driver's emotional stability value is lower than the preset emotional stability threshold, the driver's emotional adjustment strategy can be determined based on the driver's facial information and physiological characteristics, thereby realizing the conditions for the driver's emotions and reducing the driving risks caused by unstable emotions.

[0036] in, Figure 1 The smart terminal 110 shown can be any terminal device that supports the collection of facial information and physiological characteristics, such as a smart phone, a car computer, a tablet computer, a laptop computer, or a wearable device, but is not limited thereto. Figure 1 The server 120 shown, for example, can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and basic cloud computing services such as big data and artificial intelligence platforms, which are not limited here. The smart terminal 110 can communicate with the server 120 through wireless networks such as 3G (third generation mobile information technology), 4G (fourth generation mobile information technology), and 5G (fifth generation mobile information technology), which are not limited here.

[0037] Drivers need to concentrate highly when driving to ensure driving safety. However, when drivers have emotional problems, their attention may be distracted. For example, drivers may be distracted by emotions such as anger, happiness, and frustration, resulting in inability to pay attention to the situation ahead. In addition, drivers communicating or even arguing with people in the car can also distract their attention and increase driving risks.

[0038] Existing driver emotion detection systems usually rely on a single data source or shallow machine learning models, which leads to limited detection accuracy and real-time performance when faced with complex emotional states or environmental interference. In addition, the feedback mechanisms of these systems are often too simple to provide effective emotion adjustment strategies, which cannot meet the accuracy and efficiency of driver emotion detection.

[0039] The above-mentioned problems are generally applicable in general travel scenarios. In order to solve these problems, the embodiments of the present application respectively propose a driver emotion recognition method, a driver emotion recognition device, an electronic device, a computer-readable storage medium and a computer program product, which will be described in detail below.

[0040] See also Figure 2 , Figure 2 is a flowchart of a driver emotion recognition method shown in an exemplary embodiment of the present application. The method can be applied to Figure 1 The implementation environment shown is specifically executed by the server 120 in the implementation environment. It should be understood that the method can also be applied to other exemplary implementation environments and be specifically executed by devices in other implementation environments. This embodiment does not limit the implementation environment to which the method is applicable.

[0041] like Figure 2 As shown, in an exemplary embodiment, the driver emotion recognition method includes at least steps S210 to S240, which are described in detail as follows:

[0042] Step S210, obtaining the driver's identity information, and determining the personalized standard parameters corresponding to the driver based on the identity information.

[0043] Specifically, the driver's identity information can be obtained, where the driver's identity information can be information such as a driver's license and an ID card that can be used to identify the driver. Then, the driver's age, driving experience, physical condition, etc. can be determined based on the driver's Schengen information. Then, the driver's corresponding personalized standard parameters can be determined based on the driver's age, driving experience, physical condition, etc.

[0044] For example, the driver can manually input identity information such as name, ID number, driver's license number, etc. through the vehicle's interface. Alternatively, the driver's identity can be automatically identified using biometric technology (such as fingerprint, facial recognition), or the driver's identity information can be synchronized with the vehicle system through the Bluetooth or wireless connection of a smartphone. The driver can also use an RFID card (such as an employee card) to approach the vehicle to automatically read the identity information. Then, the personalized standard parameters are determined based on the driver's identity information, for example, age: physiological factors that affect driving reaction speed, vision, etc. Gender: may affect driving habits and preferences, driving experience: novice drivers and experienced drivers may have significant differences in driving behavior, health status: such as vision and hearing conditions, which may affect driving safety, preference settings: such as seat position, rearview mirror angle, audio settings, etc.

[0045] Step S220, acquiring the facial information and physiological features of the driver, fusing the facial information and the physiological features, and obtaining the corresponding emotional state parameters of the driver.

[0046] Specifically, the driver's facial information and physiological characteristics while the vehicle is driving can be obtained, wherein the physiological characteristics may include the driver's blood oxygen concentration, breathing rate, voice tone, etc., and then the driver's emotional state parameters can be determined based on data fusion of the driver's facial information and physiological characteristics, wherein the emotional state parameters include the driver's current emotional state, voice, facial expressions, etc.

[0047] Exemplarily, a high-definition camera is installed inside the vehicle to capture the driver's facial expression. The facial expression features of the driver, such as the movement state of the eyes, mouth, eyebrows, etc., are analyzed using facial recognition technology and image processing algorithms. The driver's heart rate, skin conductance and other physiological indicators are measured using devices such as heart rate monitors and skin conductance sensors. The driver's body temperature, breathing rate, etc. are measured using technologies such as infrared sensors. Emotion-related features are extracted from facial images, such as the degree of eye opening and closing, the degree of mouth opening, the degree of eyebrow curvature, etc. The emotion-related components of physiological features such as heart rate, skin conductance, body temperature, and breathing rate are extracted. Facial expression features and physiological features are spliced ​​or combined to form a comprehensive feature vector. Emotion recognition is performed on facial expressions and physiological features respectively, and then the recognition results of the two are combined to obtain the final emotional state through voting, weighted averaging, etc. A machine learning algorithm such as support vector machine and neural network is used to train an emotion recognition model, and the fused feature vector is input into the model to output the emotional state parameters. Emotional states are classified into several basic types, such as happiness, sadness, anger, surprise, fear, and disgust, and corresponding probability values ​​are given. Based on the probability values ​​of the emotion classification, a parameter representing the intensity of the emotion is calculated to measure the intensity of the driver's emotional state. Based on the driver's emotional state parameters, the vehicle's music, lighting, seats and other configurations are automatically adjusted to improve the driver's driving experience.

[0048] Step S230: determining the driver's emotional stability value based on the emotional state parameter.

[0049] Specifically, there are basic emotion types such as happiness, sadness, anger, surprise, fear and disgust, as well as the probability value of each emotion. Emotional intensity: a quantitative indicator used to measure the intensity of the driver's emotional state. The emotional stability value is a comprehensive indicator used to measure the stability and fluctuation of the driver's emotional state in a specific time period. It can be calculated and evaluated based on the changes in the emotional state parameters. Based on the emotional state parameter data of a large number of drivers, a benchmark range of emotional stability values ​​is set. This range can be adjusted according to the characteristics of different driving groups (such as novice drivers, skilled drivers, professional drivers, etc.). For each driver, the emotional volatility is calculated based on the changes in his emotional state parameters (especially emotional intensity). The emotional volatility can reflect the degree of fluctuation of the driver's emotional state. Based on the comparison of the emotional volatility with the benchmark value, the driver's emotional stability value is evaluated.

[0050] Step S240, determining a preset emotion stability threshold corresponding to the driver based on the personalized standard parameters, and if the emotion stability value is lower than the preset emotion stability threshold, determining the driver's emotion adjustment strategy based on the driver's facial information and physiological characteristics.

[0051] Specifically, the personalized standard parameters of the driver are reviewed and analyzed, including age, gender, driving experience, health status, etc. According to the personalized standard parameters, a unique emotional stability threshold is set for each driver. The threshold can be determined based on historical data, past experience, or a machine learning model. For example, a young driver may have a higher tolerance for emotional fluctuations, so their threshold may be set relatively high. The driver's facial information and physiological characteristics, such as facial expressions, heart rate, skin conductance, etc., are collected in real time. Emotional state parameters, such as emotion classification and emotion intensity, are calculated using these parameters. The current emotional stability value is compared with the preset emotional stability threshold. If the emotional stability value is lower than the threshold, it is determined that the driver is in an emotionally unstable state. According to the driver's facial information and physiological characteristics, a personalized emotional adjustment strategy is formulated. Among them, the strategy can include changes in the in-vehicle environment such as music playing, lighting adjustment, seat adjustment, ventilation setting, etc. to improve the driver's emotional state. It can also be considered to guide the driver to take deep breaths, relax, and other self-regulations through voice prompts or visual reminders. The emotional adjustment strategy is automatically or manually implemented to ensure that the driver's emotional state is improved. The changes in the emotional stability value are monitored in real time to evaluate the effectiveness of the strategy. As the driver's emotional state changes and personalized parameters are updated, the preset emotional stability threshold is adjusted regularly or in real time.

[0052] In this embodiment, by obtaining the driver's identity information, personalized standard parameters can be set for the driver, which helps to more accurately assess the driver's driving ability and risk, and by integrating the driver's facial information and physiological characteristics, the driver's emotional state can be monitored in real time to reduce the risk of safety accidents caused by emotional problems. When the emotional stability value is lower than the preset emotional stability threshold, a personalized emotional adjustment strategy can be formulated based on the driver's facial information and physiological characteristics to relieve the driver's stress, relax the mood and improve the driving experience. Further, based on the above embodiment, please refer to Figure 3 In one of the exemplary embodiments provided in the present application, the identity information includes: gender, age and driving experience. The specific implementation process of determining the personalized standard parameters corresponding to the driver based on the identity information may also include steps S310 to S330, which are described in detail as follows:

[0053] Step S310, determining a personalized standard parameter library corresponding to the driver based on age, wherein the personalized standard library includes a plurality of different personalized standard parameters;

[0054] Step S320, determining a target personalized standard parameter that matches the driver from a plurality of different personalized standard parameters based on the driving years;

[0055] Step S330, determining a preset emotional stability threshold of the driver based on the target personalized standard parameters and gender.

[0056] Specifically, a database containing multiple personalized standard parameters is pre-established. These personalized standard parameters cover different characteristics that drivers of different age groups may have, such as reaction speed, concentration, and fatigue sensitivity, etc., and then the personalized standard parameter database corresponding to the age group can be screened from the database according to the age of the driver. For example, for drivers aged 30-35, a parameter subset containing the common characteristics of drivers of this age group will be selected.

[0057] Furthermore, after determining the personalized standard parameter library corresponding to the driver's age, the target personalized standard parameters that match the driver can be further screened according to the driver's years of experience. For example, a 32-year-old driver has 5 years of driving experience. The system will select personalized standard parameters that match 5 years of driving experience from the parameter subset of the 30-35 age group. These parameters may reflect the typical behavior and emotional response of a 32-year-old driver with 5 years of driving experience during driving.

[0058] Finally, after obtaining the target personalized standard parameter set, the driver's gender factor also needs to be considered. Gender may affect the driver's emotional stability and response pattern. Assume that the target driver is male. The system will determine the driver's preset emotional stability threshold through a certain algorithm or rule based on the target personalized standard parameter set and the general characteristics of male drivers. This threshold may represent the maximum emotional fluctuation range in which the driver can maintain emotional stability without overreacting or engaging in dangerous behaviors in a specific driving situation. For example, if the analysis shows that the driver is prone to excessive anxiety or excitement in tense or dangerous situations, the preset emotional stability threshold may be set relatively low to remind the driver to stay calm and alert during driving.

[0059] In this embodiment, by comprehensively considering the driver's age, driving years and gender, the individual differences of the driver can be accurately reflected, and the corresponding emotional stability threshold can be customized for the driver, which helps to timely identify and warn the driver of possible emotional fluctuations during driving, and can significantly improve driving safety.

[0060] Further, based on the above embodiments, please refer to Figure 4 In one of the exemplary embodiments provided in the present application, the physiological characteristics further include voice characteristics and physiological parameters, and the specific recognition process of the driver emotion recognition method may further include step S410 and step S420, which are described in detail as follows:

[0061] Step S410, acquiring voice data output by the driver, and analyzing the voice data to obtain corresponding voice features;

[0062] Step S420, obtaining the driver's heart rate and blood oxygen saturation, and determining the driver's physiological parameters based on the heart rate and blood oxygen saturation.

[0063] Specifically, the feature vectors of facial information, voice features, and physiological parameters are concatenated or combined to form a comprehensive feature set.

[0064] This feature set contains complementary information from different modalities, which helps to describe the driver's emotional state more comprehensively. Then, emotion recognition can be performed on facial information, voice features, and physiological parameters respectively. Use machine learning or deep learning algorithms to fuse the emotion recognition results of each modality to obtain the final emotional state parameters. Select a suitable fusion algorithm based on the specific application scenario and data characteristics. Common fusion algorithms include weighted average, voting, Bayesian network, support vector machine, neural network, etc. Collect personalized standard parameters of the driver, such as age, gender, driving experience, health status, etc. These parameters will be used to adjust the determination process of the emotional state parameters to improve accuracy. According to the personalized standard parameters, adjust the weights, thresholds and other parameters in the multimodal fusion processing process.

[0065] The emotional state parameters are calibrated to ensure that they match the actual situation of the driver. Based on the processed emotional data, the driver's emotional state is classified into several basic types, such as happiness, sadness, anger, surprise, fear, and disgust. The intensity of the emotional state is calculated to measure the intensity of the driver's emotional state. The stability of the driver's emotional state is evaluated, that is, the degree of fluctuation of the emotional state.

[0066] In this embodiment, by acquiring the driver's voice data and analyzing it to obtain voice features, and determining physiological parameters based on heart rate and blood oxygen saturation, it has a significant beneficial effect on improving driving safety and driving experience. These measures not only help to timely discover and warn of potential safety risks, but also can provide drivers with personalized driving suggestions to achieve a more intelligent and safe driving experience.

[0067] Further, based on the above embodiments, please refer to Figure 5 In one of the exemplary embodiments provided in the present application, the specific implementation process of the above driver emotion recognition method may further include step S510 and step S520, which are described in detail as follows:

[0068] Step S510, performing multimodal fusion processing on facial information, voice features and physiological parameters to obtain processed emotion data;

[0069] Step S520, determining the driver's corresponding emotional state parameters based on the processed emotional data and the personalized standard parameters.

[0070] Specifically, the driver's facial expressions are captured by cameras and other devices, which often reflect the person's emotional state. The driver's voice is captured by microphones and other devices, and the characteristics such as tone, speech speed, and volume are analyzed, which can also provide clues about the emotional state. The driver's heart rate, blood pressure, skin conductance and other physiological indicators are monitored by sensors and other devices. These physiological reactions are closely related to the emotional state. The fusion of the information of the above three modalities means integrating data from different sources and taking advantage of their complementarity to obtain a more comprehensive and accurate description of emotions. This fusion may involve complex algorithms and technologies, such as machine learning and deep learning, to extract and integrate key information from different modalities. The data obtained after multimodal fusion processing has integrated key emotional information from facial information, voice features and physiological parameters. These parameters are formulated based on the driver's personal characteristics, historical emotional data, etc., and are used to match and calibrate the processed emotional data with the driver's individual emotional characteristics. Based on the processed emotional data and personalized standard parameters, the driver's current emotional state parameters can be determined. These parameters may include emotional type (such as happiness, sadness, anger, etc.), emotional intensity, etc., which are used to quantitatively describe the driver's emotional state.

[0071] In this embodiment, through multimodal fusion processing, the driver's facial expressions, voice intonation and physiological reactions can be captured and recognized more accurately, so as to monitor the driver's emotional state in real time. The facial information, voice features and physiological parameters are multimodally fused, and the driver's corresponding emotional state parameters are determined based on the processed emotional data and personalized standard parameters, making the driver's emotional recognition more accurate and effective.

[0072] Further, based on the above embodiments, please refer to Figure 6 In one of the exemplary embodiments provided in the present application, the above-mentioned multimodal fusion processing of facial information, voice features and physiological parameters to obtain processed emotion data may also include step S610 and step S620, which are described in detail as follows:

[0073] Step S610, determining weights corresponding to facial information, voice features, and physiological parameters based on a preset multimodal attention mechanism;

[0074] Step S620, calculating weighted values ​​of facial information, voice features, and physiological parameters based on the weights, so as to determine the driver's emotional data based on the weighted values.

[0075] Specifically, the multimodal attention mechanism corresponding to the multimodal deep learning model can be used to determine the corresponding weights of the driver's facial information, voice features and physiological parameters in terms of their impact on the driver's emotional stability, and then the weighted values ​​of the facial information, voice features and physiological parameters can be calculated according to the corresponding weights of the driver's facial information, voice features and physiological parameters, and then the driver's emotional data can be determined based on the weighted value.

[0076] Exemplarily, a multimodal attention mechanism is preset. This mechanism first needs to preset a set of initial weights for facial information, voice features and physiological parameters based on experience and data. These weights reflect the relative importance of each modality for emotion recognition in general. The core of the attention mechanism is that it can dynamically adjust these weights based on the current input data (i.e., specific facial information, voice features and physiological parameters). This means that if a modality is particularly useful for emotion recognition in the current context, its weight will increase accordingly. Using the preset multimodal attention mechanism, the weight of each modality in the current context is calculated based on the input facial information, voice features and physiological parameters. These weights are dynamically adjusted to maximize the accuracy of emotion recognition. Once the weights of each modality are determined, their weighted values ​​can be calculated. This is usually achieved by multiplying the feature vector of each modality by its corresponding weight. The weighted value reflects the contribution of each modality to the final emotion judgment after taking into account the weight. Finally, the driver's emotional data will be determined based on the calculated weighted value, combined with possible other algorithms or models (such as machine learning or deep learning models). These emotional data may include different emotional states such as happiness, sadness, anger, tension, etc.

[0077] In this embodiment, different aspects of the driver's emotions are reflected through facial information, voice features and physiological parameters. Through the multimodal attention mechanism, these information can be comprehensively considered to achieve information complementarity, thereby improving the accuracy of emotion recognition. The emotion recognition method based on the multimodal attention mechanism can allocate weights according to the driver's personalized characteristics, thereby achieving more personalized emotion recognition.

[0078] Further, based on the above embodiment, in one of the exemplary embodiments provided in the present application, the above physiological parameters include blood oxygen saturation and heart rate, and the specific implementation process of the above driver emotion recognition method may also include the following steps, which are described in detail as follows:

[0079] If the blood oxygen saturation is lower than the preset saturation threshold and the heart rate is not lower than the preset heart rate threshold, it is determined that the driver has dangerous driving behavior and a corresponding prompt message is generated.

[0080] Specifically, the heart rate can be monitored in real time through relevant equipment, such as a photoelectric heart rate sensor, and when it is detected that the driver's heart rate is not lower than a preset heart rate threshold, and the driver's current blood oxygen saturation is lower than the preset saturation threshold, it can be determined that the driver has engaged in dangerous driving behavior, and corresponding prompt information can be generated for the dangerous driving behavior.

[0081] For example, the driver's blood oxygen saturation is an important indicator that reflects the body's oxygen supply status. If the blood oxygen saturation is lower than the preset threshold, it may mean that the driver is lacking oxygen, which will affect his reaction speed and judgment ability. The driver's heart rate, a high heart rate may indicate that the driver is in a state of tension or excitement, which may also affect the safety of his driving behavior. When both conditions are met at the same time, that is, low blood oxygen saturation and high heart rate, this is usually not a good combination of physiological states. In this case, the driver may find it difficult to make correct driving decisions due to lack of oxygen and excessive tension, which constitutes dangerous driving behavior.

[0082] In this embodiment, when the driver's blood oxygen saturation is lower than the preset threshold, it may mean that the body is not getting enough oxygen and there is a health risk. At this time, if the heart rate is not lower than the preset threshold, it may further indicate that the driver is in a state of stress or tension, increasing the risk of sudden health events (such as heart attack, syncope, etc.). By timely identifying and prompting dangerous driving behaviors, these events can be effectively prevented and driving safety can be ensured.

[0083] Further, based on the above embodiments, please refer to Figure 7 In one of the exemplary embodiments provided in the present application, the specific implementation process of the above-mentioned driver emotion recognition method may also include steps S710 to S730, which are described in detail as follows:

[0084] Step S710, obtaining voice data emitted by the driver within a preset time period;

[0085] Step S720, determining the volume fluctuation frequency, breathing fluctuation frequency and emotional fluctuation in the voice data;

[0086] Step S730, determining the voice features based on the volume fluctuation frequency, the breathing fluctuation frequency and the emotion fluctuation situation.

[0087] Specifically, the voice data emitted by the driver within a preset period of time can be continuously monitored, and then the data can be processed to obtain the key parameters in the voice data, including the volume fluctuation frequency, breathing fluctuation frequency and emotional fluctuation in the voice data, among which the volume fluctuation frequency refers to the rate of change of the volume in the voice data. This can reflect the changes in the volume of the driver when speaking, such as whether it is sometimes high and sometimes low, which may be related to his emotional state. Breathing fluctuation frequency: Although voice data usually mainly records sound information, in some cases, the driver's breathing frequency can be indirectly inferred by analyzing subtle changes in the voice signal (such as breath sounds, pauses, etc.). Breathing frequency is an important indicator for assessing the state of relaxation or tension of the human body. Emotional fluctuation: This is a more complex parameter, which usually needs to be extracted through speech emotion recognition technology. It involves a comprehensive analysis of multiple aspects such as voice intonation, speech speed, sound quality, etc. to determine the driver's emotional state when speaking (such as happiness, anger, anxiety, etc.). Furthermore, the voice characteristics of the driver in the current state can be determined based on the driver's volume fluctuation frequency, breathing fluctuation frequency and emotional fluctuation situation. These characteristics may include the stability of volume changes, the steadyness of breathing, the stability of emotions, etc.

[0088] In this embodiment, by analyzing the volume fluctuation frequency, breathing fluctuation frequency and emotional fluctuation in the voice data, the driver's mental state and physical condition can be indirectly reflected. Through voice feature analysis, the driver's abnormal state can be discovered in advance and measures can be taken to intervene, thereby effectively preventing the occurrence of traffic accidents. In addition, by fusing voice features with other physiological parameters (such as heart rate, blood oxygen saturation, etc.) and vehicle driving status information, comprehensive monitoring and evaluation of the driver's state can be achieved.

[0089] Further, based on the above embodiments, please refer to Figure 8 In one of the exemplary embodiments provided in the present application, the specific implementation process of determining the driver's emotion adjustment strategy based on the driver's facial information and physiological characteristics may also include steps S810 to S840, which are described in detail as follows:

[0090] Step S810, obtaining temperature information in the cabin of the vehicle, and adjusting a target temperature in the cabin based on the temperature information;

[0091] Step S820, obtaining the driver's sitting posture information, and adjusting the target seat posture in the cockpit based on the sitting posture information;

[0092] Step S830, obtaining the working condition information of the vehicle, and adjusting the early warning strategy of the vehicle based on the working condition information;

[0093] Step S840, determining the driver's emotion adjustment strategy based on at least one of the target temperature, the target seat posture, and the warning strategy.

[0094] Specifically, after detecting the driver's emotional state, if the driver's current emotional stability value is lower than the preset emotional stability threshold, multiple personalized feedback measures can be triggered, such as voice prompts: the car computer central control automatically wakes up the voice assistant, issues a voice prompt, reminds the driver of the current emotional state, and recommends appropriate operations. Automatically wake up the voice assistant and proactively ask the driver if he needs help. Environmental adjustment: Automatically adjust the air conditioning temperature in the car according to the actual local temperature and play music to maintain a suitable in-car environment and improve the driver's sobriety. Adjust according to the driver's previously saved seat posture to provide the best driving comfort and support. Driving safety adjustment: The lane keeping assist function will be automatically turned on to ensure that the driver stays in the center of the lane and improve driving safety. Automatically enable the collision warning and automatic braking functions, monitor the road conditions ahead, issue a collision warning in advance, and automatically start braking when necessary to avoid or reduce potential collision risks. Automatically tighten the seat belt to ensure that the driver is best protected in an emergency.

[0095] For example, the temperature information in the cabin of the vehicle at the current moment can be obtained, and the target temperature in the cabin of the garage can be adjusted according to the temperature information. Then, the target seat position in the cabin can be adjusted according to the driver's sitting posture information, thereby improving the driver's riding experience. The driver's emotions can also be soothed through voice, music, etc., and the lane keeping assist function of the vehicle can be turned on according to the vehicle's current location information, the vehicle's speed, the road information and other working conditions to ensure that the driver stays in the center of the lane and improve driving safety. Or the collision warning and automatic braking functions can be automatically enabled, the road conditions ahead are monitored, a collision warning is issued in advance, and the brakes are automatically started when necessary to avoid or reduce potential collision risks, and the seat belt is automatically tightened to ensure that the driver is best protected in an emergency. And the driver's emotional adjustment strategy can be determined by at least one of the target temperature in the vehicle cabin, the target seat position of the driver's seat, and the vehicle warning strategy.

[0096] In this embodiment, based on the vehicle's cabin temperature information, the driver's sitting posture information and the vehicle's operating condition information, the target cabin temperature, target seat posture and the vehicle's early warning strategy are adjusted respectively, and the driver's emotion adjustment strategy is determined accordingly, which can soothe the driver's emotions, improve driving comfort and safety, and optimize the driving experience.

[0097] Fig. 99 is a brief flow chart of driver emotion recognition in an exemplary application scenario. In the application scenario shown in 9, the driver's identity information is obtained, and the personalized standard parameters corresponding to the driver are determined based on the identity information; the driver's facial information and physiological characteristics are obtained, and the physiological characteristics include voice characteristics and physiological parameters; the facial information, voice characteristics and physiological parameters are multimodally fused to obtain processed emotion data; the driver's corresponding emotion state parameters are determined based on the processed emotion data and personalized standard parameters. The driver's emotional stability value is determined based on the emotional state parameters; the driver's preset emotional stability threshold is determined based on the personalized standard parameters. If the emotional stability value is lower than the preset emotional stability threshold, the temperature information in the cabin of the vehicle is obtained, and the target temperature in the cabin is adjusted based on the temperature information; the driver's sitting posture information is obtained, and the target seat posture in the cabin is adjusted based on the sitting posture information; the vehicle's working condition information is obtained, and the vehicle's warning strategy is adjusted based on the working condition information; the driver's emotion adjustment strategy is determined based on at least one of the target temperature, target seat posture and warning strategy. For the detailed implementation process, please refer to the records in the aforementioned embodiments, and this will not be repeated here.

[0098] Fig.10 is a block diagram of a driver emotion recognition device shown in an exemplary embodiment of the present application. The device can be applied to Figure 1 The implementation environment shown in the figure is specifically configured in the server 120. The device can also be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applied.

[0099] like Fig.10 As shown, the exemplary driver emotion recognition device includes: an acquisition module 1010, used to obtain the driver's identity information, and determine the personalized standard parameters corresponding to the driver based on the identity information; a data fusion module 1020, used to obtain the driver's facial information and physiological characteristics, and fuse the facial information and physiological characteristics to obtain the driver's corresponding emotional state parameters; a determination module 1030, used to determine the driver's emotional stability value based on the emotional state parameters; an adjustment module 1040, used to determine the driver's corresponding preset emotional stability threshold based on the personalized standard parameters, if the emotional stability value is lower than the preset emotional stability threshold, the driver's emotional adjustment strategy is determined based on the driver's facial information and physiological characteristics.

[0100] According to one aspect of an embodiment of the present application, the above-mentioned driver emotion recognition device also includes: a personalized determination module, which is used to determine the personalized standard parameter library corresponding to the driver based on age, and the personalized standard library includes multiple different personalized standard parameters; based on driving years, determine the target personalized standard parameters that match the driver from multiple different personalized standard parameters; based on the target personalized standard parameters and gender, determine the driver's preset emotional stability threshold.

[0101] According to one aspect of an embodiment of the present application, the data fusion module 1020 is also used to obtain voice data output by the driver, and analyze the voice data to obtain corresponding voice features; obtain the driver's heart rate and blood oxygen saturation, and determine the driver's physiological parameters based on the heart rate and blood oxygen saturation.

[0102] According to one aspect of an embodiment of the present application, the above-mentioned data fusion module 1020 is also used to perform multimodal fusion processing on facial information, voice features and physiological parameters to obtain processed emotion data; and determine the corresponding emotion state parameters of the driver based on the processed emotion data and personalized standard parameters.

[0103] According to one aspect of an embodiment of the present application, the above-mentioned data fusion module 1020 is also used to determine the weights corresponding to the facial information, voice features and physiological parameters based on a preset multimodal attention mechanism; calculate the weighted values ​​of the facial information, voice features and physiological parameters based on the weights, so as to determine the driver's emotional data based on the weighted values.

[0104] According to one aspect of an embodiment of the present application, the data fusion module 1020 is also used to determine that the driver has dangerous driving behavior and generate corresponding prompt information if the blood oxygen saturation is lower than a preset saturation threshold and the heart rate is not lower than a preset heart rate threshold.

[0105] According to one aspect of an embodiment of the present application, the above-mentioned data fusion module 1020 is also used to obtain voice data emitted by the driver within a preset time period; determine the volume fluctuation frequency, breathing fluctuation frequency and emotional fluctuation in the voice data; and determine the voice features based on the volume fluctuation frequency, breathing fluctuation frequency and emotional fluctuation.

[0106] According to one aspect of an embodiment of the present application, the above-mentioned adjustment module 1040 is also used to obtain temperature information in the cabin of the vehicle, and adjust the target temperature in the cabin based on the temperature information; obtain the driver's sitting posture information, and adjust the target seat position in the cabin based on the sitting posture information; obtain the vehicle's operating condition information, and adjust the vehicle's early warning strategy based on the operating condition information; determine the driver's emotion adjustment strategy based on at least one of the target temperature, target seat position and early warning strategy.

[0107] It should be noted that the driver emotion recognition device provided in the above embodiment and the driver emotion recognition method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs operations has been described in detail in the method embodiment and will not be repeated here. In actual applications, the driver emotion recognition device provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0108] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, when the one or more programs are executed by one or more processors, the electronic device implements the driver emotion recognition method provided in the above-mentioned embodiments.

[0109] Fig.11 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Fig.11 The computer system 1100 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0110] like Fig.11 As shown, the computer system 1100 includes a central processing unit (CPU) 1101, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1102 or the program loaded from the storage part 1108 to the random access memory (RAM) 1103, such as executing the method in the above embodiment. In the RAM 1103, various programs and data required for system operation are also stored. The CPU 1101, the ROM 1102 and the RAM 1103 are connected to each other through the bus 1104. The input / output (I / O) interface 1105 is also connected to the bus 1104.

[0111] The following components are connected to the I / O interface 1105: an input section 1106 including a keyboard, a mouse, etc.; an output section 1107 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the I / O interface 1105 as needed. A removable medium 1111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1110 as needed so that a computer program read therefrom is installed into the storage section 1108 as needed.

[0112] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1109, and / or installed from a removable medium 1111. When the computer program is executed by a central processing unit (CPU) 1101, various functions defined in the system of the present application are executed.

[0113] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0114] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0115] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0116] Another aspect of the present application further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the driver emotion recognition method as described above is implemented. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0117] Another aspect of the present application also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the driver emotion recognition method provided in each of the above embodiments.

[0118] The above content is only a preferred exemplary embodiment of the present application and is not intended to limit the implementation scheme of the present application. A person skilled in the art can easily make corresponding changes or modifications based on the main concept and spirit of the present application. Therefore, the scope of protection of the present application shall be based on the scope of protection required by the claims.

Claims

1. A driver emotion recognition method, characterized in that: include: Acquiring identity information of a driver, and determining personalized standard parameters corresponding to the driver based on the identity information; Acquiring facial information and physiological features of the driver, fusing the facial information and the physiological features, and obtaining emotional state parameters corresponding to the driver; determining an emotional stability value of the driver based on the emotional state parameter; A preset emotion stability threshold corresponding to the driver is determined based on the personalized standard parameter; if the emotion stability value is lower than the preset emotion stability threshold, an emotion adjustment strategy of the driver is determined based on the facial information and the physiological characteristics of the driver.

2. The method according to claim 1, characterized in that The identity information includes: gender, age and driving experience, and the determining of the personalized standard parameters corresponding to the driver based on the identity information includes: Determine a personalized standard parameter library corresponding to the driver based on the age, wherein the personalized standard library includes a plurality of different personalized standard parameters; Determining a target personalized standard parameter matching the driver from the plurality of different personalized standard parameters based on the driving years; A preset emotional stability threshold of the driver is determined based on the target personalized standard parameters and the gender.

3. The method according to claim 1, characterized in that The physiological characteristics also include voice characteristics and physiological parameters, and the method also includes: Acquiring voice data output by the driver, and analyzing the voice data to obtain corresponding voice features; The heart rate and blood oxygen saturation of the driver are obtained, and the physiological parameters of the driver are determined based on the heart rate and the blood oxygen saturation.

4. The method according to claim 3, characterized in that The method further comprises: Performing multimodal fusion processing on the facial information, the voice features and the physiological parameters to obtain processed emotion data; The emotional state parameters corresponding to the driver are determined based on the processed emotional data and the personalized standard parameters.

5. The method according to claim 4, characterized in that The multimodal fusion processing of the facial information, the voice features and the physiological parameters to obtain processed emotion data includes: Determine the weights corresponding to the facial information, the voice features, and the physiological parameters based on a preset multimodal attention mechanism; Weighted values ​​of the facial information, the voice features, and the physiological parameters are calculated based on the weights to determine the driver's emotion data based on the weighted values.

6. The method according to claim 3, characterized in that The method further comprises: Acquiring voice data emitted by the driver within a preset time period; Determining volume fluctuation frequency, breathing fluctuation frequency and emotional fluctuation in the voice data; The voice feature is determined based on the volume fluctuation frequency, the breathing fluctuation frequency and the emotion fluctuation condition.

7. The method according to claim 1, characterized in that The determining the driver's emotion adjustment strategy based on the driver's facial information and the physiological characteristics includes: acquiring temperature information in a cabin of the vehicle, and adjusting a target temperature in the cabin based on the temperature information; Acquiring the driver's sitting posture information, and adjusting a target seat posture in the cockpit based on the sitting posture information; Acquiring operating condition information of the vehicle, and adjusting a warning strategy of the vehicle based on the operating condition information; The driver's emotion adjustment strategy is determined based on at least one of the target temperature, the target seat posture, and the warning strategy.

8. A driver emotion recognition device, characterized in that: The device comprises: An acquisition module, used to acquire the identity information of the driver, and determine the personalized standard parameters corresponding to the driver based on the identity information; A data fusion module, used to obtain the facial information and physiological characteristics of the driver, and fuse the facial information and the physiological characteristics to obtain the emotional state parameters corresponding to the driver; a determination module, configured to determine the driver's emotional stability value based on the emotional state parameter; An adjustment module is used to determine a preset emotion stability threshold corresponding to the driver based on the personalized standard parameters, and if the emotion stability value is lower than the preset emotion stability threshold, determine the driver's emotion adjustment strategy based on the driver's facial information and the physiological characteristics.

9. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, enables the electronic device to implement the driver emotion recognition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and when the computer-readable instructions are executed by a processor of a computer, the computer is caused to execute the driver emotion recognition method according to any one of claims 1 to 7.

Images