Vehicle-mounted voice interaction method, device, equipment, program product and automobile

Abstract

The invention relates to the technical field of vehicle-mounted intelligent cabins, and discloses a vehicle-mounted voice interaction method, device and equipment, a program product and an automobile, and the method comprises the steps: obtaining the emotion behavior information and driving state information of a driver; identifying the emotional state of the driver according to the emotional behavior information, and identifying the behavior state of the driver according to the driving state information; and determining the driving state of the driver according to the emotional state and the behavior state, and performing voice interaction based on the driving state. According to the method, the current driving state of the driver can be determined by combining the emotional behavior and the driving behavior, the driving requirement of the driver is met through voice interaction, traffic accidents are avoided, and the driving comfort and safety are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle-mounted intelligent cockpits, and in particular to a vehicle-mounted voice interaction method, device, equipment, program product and vehicle. Background Art

[0002] With the rapid advancement of vehicle-mounted technology, the field of smart cockpits has shown a diversified development trend, and a variety of ecological application products have been realized in the cockpit. Among these products, the voice module, as one of the core components of the smart cockpit, is of self-evident importance.

[0003] Although in-vehicle voice systems have been widely used in vehicles, some in-vehicle voice systems mostly focus only on executing commands and providing information, paying less attention to the impact of the driver's emotional state and behavior patterns on driving safety, and cannot meet the driver's driving needs. In actual driving, the driver's emotional fluctuations (such as anger, anxiety, fatigue, etc.) and improper driving behaviors (such as sudden acceleration, sharp turns, frequent lane changes, etc.) may increase the risk of traffic accidents and reduce driving comfort and safety. Summary of the invention

[0004] In view of this, the present invention provides an in-vehicle voice interaction method, device, equipment, program product and automobile to solve the problem that in-vehicle voice interaction cannot meet the driving needs of the driver.

[0005] In a first aspect, the present invention provides a vehicle-mounted voice interaction method, the method comprising:

[0006] Obtain the driver's emotional behavior information and driving status information;

[0007] Identify the driver's emotional state based on the emotional behavior information, and identify the driver's behavioral state based on the driving state information;

[0008] The driver's driving status is determined according to the emotional state and behavioral state, and voice interaction is performed based on the driving status.

[0009] The in-vehicle voice interaction method provided by the embodiment of the present invention identifies the driver's emotional state based on the driver's emotional behavior information, identifies the driver's behavior state based on the driver's driving state information, determines the driver's driving state based on the emotional state and the behavior state, and performs voice interaction based on the driving state. The present invention can determine the driver's current driving state by combining emotional behavior and driving behavior, and meet the driver's driving needs through voice interaction, avoid traffic accidents, and improve driving comfort and safety.

[0010] In an optional embodiment, the emotional behavior information includes: facial information and voice information, the driving status information includes: speed information and steering information, the emotional state of the driver is identified based on the emotional behavior information, and the driver's behavioral state is identified based on the driving status information, including: based on a preset emotion recognition algorithm, comprehensively analyzing the facial information and voice information to determine the driver's emotional state, the emotional state includes at least one of the following: emotional state type, emotional intensity, emotional duration, emotional change trend and attention impact degree; based on a preset behavior analysis algorithm, comprehensively analyzing the speed information and steering information to determine the driver's behavioral state, the behavioral state includes at least one of the following: driving safety, driving stability, driving habits, potential accident risks, driving efficiency, traffic impact degree and driving behavior improvement needs.

[0011] The present invention can accurately judge the driver's emotional state and behavioral state by performing multi-dimensional analysis of emotional behavior information and driving status information, thereby providing an accurate basis for in-vehicle voice interaction.

[0012] In an optional embodiment, the driver's driving state is determined according to the emotional state and the behavioral state, and voice interaction is performed based on the driving state, including: determining the driving state according to at least one of the emotional state type, emotional intensity, emotional duration, emotional change trend, or attention impact degree, the driving state including: fatigue driving state, angry driving state, and dangerous driving state; confirming the driving state based on at least one of driving safety, driving stability, driving habits, potential accident risks, driving efficiency, traffic impact degree, or driving behavior improvement needs; obtaining the corresponding preset interactive reply language according to the driving state, and voice playing the preset interactive reply language in a preset tone and preset speed.

[0013] The present invention can reflect the important influence of emotions and behaviors on driving operations by determining the driver's current driving state according to the emotional state and behavioral state, so as to perform voice interaction according to the influence situation, provide comfort or prompts to the driver, avoid the occurrence of dangerous driving behavior, and reduce the probability of traffic accidents.

[0014] In an optional embodiment, when performing voice interaction based on driving status, it also includes: obtaining a corresponding preset interaction interface according to the driving status, and displaying the preset interaction interface; obtaining corresponding warning information according to the driving status, and displaying the warning information.

[0015] The present invention can further provide driving tips and reassurance to the driver through the interactive interface and the display of warning information, thereby avoiding the occurrence of dangerous driving behaviors and reducing the probability of traffic accidents.

[0016] In an optional embodiment, it also includes: when it is determined that the driver is in an angry driving state or a dangerous driving state, executing a preset emergency handling operation, the preset emergency handling operation includes at least one of the following: limiting the vehicle speed, issuing a warning signal and contacting emergency services.

[0017] The present invention can further ensure driving safety through operations such as vehicle control by handling emergency situations in angry or dangerous driving states, thereby avoiding traffic accidents caused by emotional excitement.

[0018] In an optional implementation, it also includes: setting personalized interactive reply speech, personalized interactive interface, personalized warning information and / or personalized emergency handling operation according to driving status.

[0019] Through user personalized settings, the present invention can ensure that the vehicle-computer interaction is more in line with the driver's personal preferences and habits, achieve a perfect match between the vehicle-computer interaction and personal needs, improve driving comfort and safety, and enhance the driver's driving experience.

[0020] In a second aspect, the present invention provides an in-vehicle voice interaction device, the device comprising: an information acquisition module for acquiring the driver's emotional behavior information and driving status information; a state recognition module for identifying the driver's emotional state based on the emotional behavior information, and identifying the driver's behavioral state based on the driving status information; a voice interaction module for determining the driver's driving state based on the emotional state and behavioral state, and performing voice interaction based on the driving state.

[0021] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the in-vehicle voice interaction method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0022] In a fourth aspect, the present invention provides a computer program product, comprising computer instructions for causing a computer to execute the in-vehicle voice interaction method of the first aspect or any corresponding embodiment thereof.

[0023] In a fifth aspect, the present invention provides a car, comprising: a camera for collecting facial information of a driver; a microphone for collecting voice information of the driver; a sensor for collecting driving status information of the driver; and a controller connected to the camera, the microphone and the sensor, for executing the in-vehicle voice interaction method of the first aspect or any corresponding embodiment thereof.

[0024] Beneficial effects of the present invention:

[0025] (1) The present invention can accurately identify the driver's emotional state and behavioral state through a preset algorithm and express them in multiple dimensions, thereby providing an accurate basis for vehicle-machine voice interaction and meeting the driver's driving needs;

[0026] (2) The present invention can create a more intelligent, safe and comfortable driving environment for the driver by determining the driver's driving status and performing voice interaction according to the driving status, so as to comfort and remind the driver in time and ensure driving safety;

[0027] (3) Through driver personalized settings, the present invention can ensure that the vehicle-computer interaction conforms to the driver's personal preferences and habits, achieve a perfect match between the vehicle-computer interaction and personal needs, improve driving comfort and safety, and enhance the driver's driving experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 is a flow chart of a vehicle-mounted voice interaction method according to an embodiment of the present invention;

[0030] Figure 2 is a flow chart of another vehicle-mounted voice interaction system according to an embodiment of the present invention;

[0031] Figure 3 is a schematic diagram of a processing module of another vehicle-mounted voice interaction system according to an embodiment of the present invention;

[0032] Figure 4 is a flow chart of another vehicle-mounted voice interaction method according to an embodiment of the present invention;

[0033] Figure 5 is a structural block diagram of a vehicle-mounted voice interaction device according to an embodiment of the present invention;

[0034] Figure 6 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0036] The embodiment of the present invention is applicable to the scenario of voice interaction according to the current state of the driver while the driver is driving the vehicle. The embodiment of the present invention provides a method for in-vehicle voice interaction, which achieves the effect of satisfying the driver's driving needs through voice interaction by accurately identifying the driver's emotional state and behavioral state.

[0037] According to an embodiment of the present invention, an embodiment of an in-vehicle voice interaction method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0038] In this embodiment, a vehicle-mounted voice interaction method is provided, which can be used in a vehicle controller. Figure 1 is a flow chart of a vehicle voice interaction method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0039] Step S101, obtaining the driver's emotional behavior information and driving status information.

[0040] Specifically, in an embodiment of the present invention, a DMS (Drive Monitor System) camera, an OMS (Occupant Monitor System) camera, a microphone, an accelerometer, a gyroscope and other devices are deployed on the car. The embodiment of the present invention collects the driver's multi-angle facial information through the DMS camera and the OMS camera, such as the shape / position of the eyebrows, the state of the eyes, the shape of the mouth, the posture and movement of the head / body, the changes in skin color and texture, the tension of the muscles, etc.; collects the driver's voice information through the microphone, such as the volume of the voice, the speed of speech, the tone of voice, etc.; monitors the speed information of the vehicle through the acceleration information of the accelerometer to determine the change of the vehicle speed; monitors the steering information of the vehicle through the angle information of the gyroscope to determine the shaking of the vehicle. The above is only an example, but not limited to this. The embodiment of the present invention uses facial information and voice information as the driver's emotional behavior information, and uses speed information and steering information as the driving state information of the vehicle driven by the driver. To a certain extent, the driving state of the vehicle will be affected by emotions.

[0041] Step S102, identifying the driver's emotional state according to the emotional behavior information, and identifying the driver's behavioral state according to the driving state information.

[0042] Specifically, in an embodiment of the present invention, an emotion recognition algorithm is developed based on an artificial intelligence algorithm, and a comprehensive analysis of facial information and sound information is performed, wherein the analysis of facial information involves image processing technology and deep learning algorithms, and facial features are extracted based on eyebrows, mouth, head, etc. in the above facial information, and the driver's emotions can be identified based on facial features. At the same time, if the driver is emotionally excited, he will also express himself verbally, and there will be differences in the volume, speed, tone, etc. of the voice under different emotions. Therefore, the driver's emotions can also be determined by performing audio feature analysis on the sound information. The embodiment of the present invention is based on an emotion recognition algorithm, combined with a comprehensive analysis of facial information and sound information, and can accurately identify the driver's emotional state, and the specific implementation of the emotion recognition algorithm is a conventional technical means in this field, which will not be repeated here.

[0043] In some optional implementations, the vehicle speed information and steering information collected by the embodiments of the present invention are formed based on the driving behaviors of the driver's hands and feet, such as the driver controlling the vehicle to perform rapid acceleration, sharp turns, frequent lane changes, etc., and these abnormal driving behaviors are largely due to the current driving environment affecting the driver's emotions and are made under the control of emotions. Therefore, the present invention is based on a vehicle-integrated behavior analysis algorithm to determine the driver's behavior state based on speed information and steering information, and the specific implementation of the behavior analysis algorithm is a conventional technical means in this field, which will not be repeated here.

[0044] Step S103, determining the driver's driving state according to the emotional state and the behavioral state, and performing voice interaction based on the driving state.

[0045] Specifically, in an embodiment of the present invention, the driver's driving state is divided in advance according to the driver's emotional state and behavioral state, including a fatigue driving state, an angry driving state, and a dangerous driving state, wherein the fatigue driving state corresponds to a scenario where the driver is fatigued due to long-distance driving, the angry driving state corresponds to a scenario where the driver is angry due to traffic congestion, and the dangerous driving state corresponds to a scenario where the driver is extremely angry or nervous, which is for example only and is not limited thereto. In an embodiment of the present invention, when the driver controls the vehicle, the driver's emotional state is identified based on the driver's facial information and voice information, the driver's behavioral state is determined based on the vehicle's speed information and steering information, and a comprehensive analysis is performed based on the emotional state and the behavioral state to determine the driver's current driving state, and different voice interactions are performed in different driving states to comfort the driver or warn the driver, meet the driver's current driving needs, ensure driving safety, and avoid traffic accidents caused by abnormal driver emotions.

[0046] The in-vehicle voice interaction method provided by the embodiment of the present invention identifies the driver's emotional state based on the driver's emotional behavior information, identifies the driver's behavior state based on the driver's driving state information, determines the driver's driving state based on the emotional state and the behavior state, and performs voice interaction based on the driving state. The present invention can determine the driver's current driving state by combining emotional behavior and driving behavior, and meet the driver's driving needs through voice interaction, avoid traffic accidents, and improve driving comfort and safety.

[0047] In this embodiment, a vehicle-mounted voice interaction method is provided, which can be used in a vehicle controller. Figure 2 is a flow chart of a vehicle-mounted voice interaction method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0048] Step S201, obtaining the driver's emotional behavior information and the driving status information of the vehicle being driven. Figure 1 Step S101 of the illustrated embodiment will not be described in detail here.

[0049] Step S202, identifying the driver's emotional state based on the emotional behavior information, and identifying the driver's behavioral state based on the driving state information.

[0050] Specifically, the above step S202 includes:

[0051] Step S2021, based on a preset emotion recognition algorithm, comprehensively analyze the facial information and sound information to determine the driver's emotional state, where the emotional state includes at least one of the following: emotional state type, emotional intensity, emotional duration, emotional change trend, and attention impact degree.

[0052] Specifically, in the embodiment of the present invention, Figure 3 As shown, the emotion recognition module comprehensively analyzes the driver's facial information and voice information from five dimensions to determine the emotional state of the five dimensions, which are shown as follows, only as an example and not limited to this:

[0053] 1. Emotional state type, determined based on different facial and vocal information, including: anger, anxiety, fatigue, happiness, and calmness;

[0054] 2. Emotional intensity, determined by the degree of expression of facial information and voice information, including: mild, moderate or moderate emotional state. The more intense the driver's emotional state, the more obvious his facial expression and the louder his voice will be;

[0055] 3. Emotion duration, determined based on the duration of facial or voice information, including: short-term emotion fluctuations or long-term emotion fluctuations;

[0056] 4. Emotional change trends, determined based on changes in facial or voice information, including: emotional aggravation, emotional relief, and emotional stability;

[0057] 5. The degree of attention impact is determined based on the degree of impact of emotions on the driver’s attention, including: slight distraction, severe distraction and no impact.

[0058] Step S2022, based on a preset behavior analysis algorithm, comprehensively analyze the speed information and steering information to determine the driver's behavior state, which includes at least one of the following: driving safety, driving stability, driving habits, potential accident risks, driving efficiency, traffic impact degree and driving behavior improvement needs.

[0059] Specifically, in the embodiment of the present invention, Figure 3 As shown, the speed information and steering information of the driver are comprehensively analyzed from seven dimensions by the behavior analysis module, so as to determine the behavior states in seven dimensions, which are respectively as follows, which are only used as examples and are not limited to this:

[0060] 1. Driving safety, determined by the size of speed information and steering information, including: high-risk driving behavior, medium-risk driving behavior and low-risk driving behavior;

[0061] 2. Driving stability, determined by the frequency of change of speed information and steering information, including: stable, relatively unstable or unstable;

[0062] 3. Driving habits: analyzing speed information and steering information according to driving rules, including good driving habits, driving habits that need to be improved, and bad driving habits;

[0063] 4. Potential accident risk: risk prediction based on current speed information and steering information, including: high probability of accident, medium probability of accident and low probability of accident;

[0064] 5. Driving efficiency, determined based on the current speed information and steering information, including: efficient driving, normal efficiency driving, and inefficient driving;

[0065] 6. Traffic impact degree, determined according to the impact on road traffic when driving according to the current speed information and steering information, including: causing traffic congestion and not causing traffic congestion, among which, the impact on traffic flow is small or almost no impact is considered not causing traffic congestion;

[0066] 7. Demand for improving driving behavior, such as reducing the number of braking times, increasing the accuracy of using turn signals, etc.

[0067] Step S203, determining the driver's driving state according to the emotional state and the behavioral state, and performing voice interaction based on the driving state.

[0068] Specifically, the above step S203 includes:

[0069] Step S2031, determining the driving state according to at least one of the emotional state type, emotional intensity, emotional duration, emotional change trend, or attention impact degree.

[0070] Specifically, in an embodiment of the present invention, the driver's driving state can be determined according to any dimension of the emotional state. For example, if the driver's emotional state type is identified as fatigue, the driving state is determined to be a fatigue driving state; or a comprehensive analysis is performed according to different dimensions to further determine the driver's driving state. For example, if the emotional state type is anger and the emotional intensity is severe, the driving state is comprehensively determined to be a dangerous driving state. This is only an example and is not limited to this.

[0071] Step S2032, confirming the driving status based on at least one of driving safety, driving stability, driving habits, potential accident risks, driving efficiency, traffic impact, or driving behavior improvement needs.

[0072] Specifically, in an embodiment of the present invention, in order to ensure the accuracy of driving state identification, the initially determined driving state is confirmed based on the driver's behavioral state, because human emotions can cause changes in behavior. For example, the driver is determined to be in a fatigued driving state based on the emotional state. When the driver is fatigued driving, he will involuntarily reduce the force of his foot on the accelerator pedal, and his hands will also reduce the force of his grip on the steering wheel, causing the car to slow down and sway slightly left and right. By monitoring the accelerometer and gyroscope, it can be determined that the driver is in an unstable state, so it can be further confirmed that the driver is indeed in a fatigued driving state. Similarly, the embodiment of the present invention can further confirm the driver's driving state based on any dimension in the behavioral state, or conduct a comprehensive analysis based on different dimensions to confirm the driver's driving state, which is specifically determined based on the connection between the driving state and the behavioral state, and is not limited here.

[0073] Step S2033, obtaining the corresponding preset interactive reply language according to the driving status, and playing the preset interactive reply language in voice according to the preset tone and preset speed.

[0074] Specifically, in the embodiment of the present invention, different interactive reply words are set in advance according to different driving states. When the driver is driving, after determining the current driving state according to the driver's emotions and behaviors, the car directly obtains the corresponding interactive reply words and plays them through the microphone in a preset tone and speed. For example, when the driver is angry and has aggressive driving behavior, the voice response will adopt a calm, peaceful and firm tone to remind the driver to pay attention to emotional control and give advice on safe driving; if the driver shows fatigue, the voice response will be gentle and slow, warmly reminding the driver to rest as soon as possible, and actively provide navigation information of nearby rest areas. The above functions are all executed by the adaptive voice adjustment module, such as Figure 3 shown.

[0075] Step S204, obtaining a corresponding preset interactive interface according to the driving state, and displaying the preset interactive interface; obtaining corresponding warning information according to the driving state, and displaying the warning information.

[0076] Specifically, in an embodiment of the present invention, in order to further meet the driver's driving needs under various driving conditions, various functions of the car are fully called upon to comfort or prompt the driver. For example, an interactive interface and warning information are set according to the driving status, and the interactive interface and warning information are displayed on a display screen. Whether there is a corresponding interactive interface and warning information is confirmed according to the user's pre-settings.

[0077] In some optional embodiments, such as Figure 3As shown, the embodiment of the present invention also provides a personalized setting module for users. Users can set personalized interactive reply messages, personalized interactive interface settings and personalized warning information according to their personal preferences and habits and different driving states. For example, they can choose to hear soothing music and encouraging words when they are anxious, such as "Friend, don't worry, every road is an opportunity for growth, I believe you can cope with it smoothly"; when they are tired, they prefer to be directly navigated to the nearest rest point; or they set that they do not want to be disturbed by voice under certain emotions, but prompt through vibration or indicator light. In addition, according to personal language preferences, the expression methods such as language, accent and intonation style of voice can be selected to meet the specific needs of different users. At the same time, the driver can independently set the reminder method and sensitivity for different driving behaviors to achieve a perfect match with their own needs. And according to the user's settings, a specific background color, picture or animation is displayed on the display screen, such as a peaceful forest picture, to help them relax. Alternatively, the driver can select different voice styles, music types, reply contents, etc. corresponding to different emotions in the system settings. When tired, the driver can choose to play soft classical music; when angry, the driver can choose to play cheerful pop music, etc. Specific reminder methods can also be set, such as voice reminders, vibration reminders or flashing lights. These are examples only and are not limited to this.

[0078] In some optional implementation manners, the embodiments of the present invention are described by taking the scenario where a driver is in a fatigued driving state due to long-distance driving and the scenario where a driver is in an angry driving state due to traffic congestion as examples, but are not limited thereto:

[0079] 1. Long-distance driving causes the driver to be in a fatigued driving state

[0080] ① The on-board camera captures characteristics of the driver such as frequent blinking, dull eyes, and relaxed facial muscles. The sound sensor detects that the driver speaks in a low voice, slow speech speed, and monotonous tone. The emotion recognition algorithm determines that the driver's emotional state type is fatigue, and therefore determines that the driver is in a fatigued driving state.

[0081] ② The accelerometer and gyroscope detected that the vehicle's speed was gradually decreasing and there was a slight left and right shaking. At the same time, the vehicle's electronic system showed that the lane departure warning was frequently triggered. The behavior analysis algorithm determined that the driver was in an unstable state, which may be related to fatigue. Therefore, it was confirmed that the driver was in a fatigued driving state.

[0082] ③ Obtain interactive responses in a fatigued driving state: "Dear driver, from your expression and voice, I can feel that you are a little tired. For your safety and comfort, there is a cozy rest area 10 kilometers ahead. How about we go there to rest for a while?", where the kilometer information is obtained through map navigation information.

[0083] ④Interactive UI (User Interface) display content: A pop-up window with a warm reminder pops up on the car display, showing "Fatigue driving warning, rest recommended", and also showing the distance and route to the rest area ahead. The pop-up window background is light blue and the font is white to highlight it. The pop-up window is also equipped with a smiling cartoon image, giving people a friendly feeling.

[0084] ⑤ If the driver has set in advance that he wants to listen to soft music when he is tired, the system will automatically play soothing music.

[0085] 2. Traffic congestion triggers drivers to drive in anger

[0086] ① The on-board camera captures the driver's frowning, gritting his teeth, and tense facial expression. The sound sensor detects that the driver is complaining loudly, with a rapid tone and high volume. The emotion recognition algorithm determines that the driver's emotional state type is anger, and therefore determines that the driver is in an angry driving state.

[0087] ② The accelerometer and gyroscope detected frequent sudden braking and starting actions of the vehicle, and the vehicle's acceleration and deceleration changed dramatically. Through the behavior analysis algorithm, it was determined that the driver was in an unstable state, his behavior was relatively aggressive, and he was greatly affected by anger. Therefore, it was confirmed that the driver was in an angry driving state.

[0088] ③ Get interactive responses in the fatigue driving state: "Dear friend, traffic jams are indeed annoying, but getting angry won't make the road smooth. Let's be patient and everything will be fine soon."

[0089] ④Interactive UI display content: A pop-up window with green as the main color appears on the in-vehicle display screen, which reads "Please stay calm, traffic will improve", and displays the length of the current congested road section and the estimated time to pass. The pop-up window also has a cute little animal smiling, trying to ease the driver's tension.

[0090] ⑤ If the driver has set in advance that he wants to hear humorous words to relieve his emotions when he is angry, the system will say some humorous witticisms, such as: "Don't be anxious, maybe the car in front is enjoying the scenery, let's wait and take a rest." This is just an example and is not limited to this.

[0091] The in-vehicle voice interaction method provided by the embodiment of the present invention identifies the driver's emotional state based on the driver's emotional behavior information, identifies the driver's behavior state based on the driver's driving state information, determines the driver's driving state based on the emotional state and the behavior state, and performs voice interaction based on the driving state. The present invention can determine the driver's current driving state by combining emotional behavior and driving behavior, and meet the driver's driving needs through voice interaction, avoid traffic accidents, and improve driving comfort and safety.

[0092] In this embodiment, a vehicle-mounted voice interaction method is provided, which can be used in a vehicle controller. Figure 4 is a flow chart of a vehicle voice interaction method according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0093] Step S401, obtaining the driver's emotional behavior information and driving status information. Figure 2 Step S201 of the illustrated embodiment will not be described in detail here.

[0094] Step S402: Identify the driver's emotional state based on the emotional behavior information, and identify the driver's behavioral state based on the driving state information. Figure 2 Step S202 of the illustrated embodiment will not be described in detail here.

[0095] Step S403: Determine the driver's driving state according to the emotional state and behavioral state, and perform voice interaction based on the driving state. Figure 2 Step S203 of the illustrated embodiment will not be described in detail here.

[0096] Step S404, when it is determined that the driver is in an angry driving state or a dangerous driving state, a preset emergency handling operation is executed, and the preset emergency handling operation includes at least one of the following: limiting the vehicle speed, issuing a warning signal, and contacting emergency services.

[0097] Specifically, in the embodiment of the present invention, Figure 3As shown, an emergency handling module is provided, and this module is immediately activated when it is detected that the driver has strong emotions such as extreme anger, panic, or serious dangerous driving behaviors, such as speeding out of control, continuous sudden lane changes, and the vehicle is about to lose control. First, the car will quickly limit the speed, reduce the engine power or start the braking system through the vehicle's electronic control unit. At the same time, a strong and eye-catching sound and light warning signal is issued to attract the driver's high attention. In critical situations, emergency services are automatically contacted, such as calling the police and providing the vehicle's accurate location and related situation information to ensure that external rescue support can be obtained in a timely manner. The embodiment of the present invention is described using the scenario in which the driver is in a dangerous driving state due to extreme anger or tension as an example, but is not limited to this:

[0098] ① The on-board camera may capture the driver's expression as tense, focused but slightly out of control. The sound sensor may detect that the driver may be breathing rapidly or making a brief exclamation. The emotion recognition algorithm determines that the driver's emotional state is tense or in a state of danger perception, and therefore determines that the driver is in a dangerous driving state.

[0099] ② The accelerometer and gyroscope detected that the vehicle accelerated and turned sharply multiple times in a short period of time, and the speed fluctuated greatly. The behavior analysis algorithm determined that the driver was in an extremely unstable state, and therefore confirmed that the driver was in a dangerous driving state.

[0100] ③ Get interactive responses in dangerous driving situations: "Danger! Please drive smoothly and pay attention to safety!"

[0101] ④Interactive UI display content: A red emergency pop-up window pops up on the car display screen, which reads "Dangerous driving! Please slow down", accompanied by a strong warning sound effect. The pop-up window also displays the current speed of the vehicle and the number of dangerous behaviors.

[0102] ⑤ Immediately activate the emergency handling module, limit the vehicle speed, and reduce the engine power through the vehicle's electronic control unit; issue a loud and continuous warning sound: "Beep, beep, beep! Danger! Please correct it immediately!"; at the same time, the system automatically contacts emergency services, such as calling the police and providing the vehicle's exact location and relevant situation information.

[0103] ⑥ If the driver sets the system in advance to force the car to stop directly when danger occurs, the car will perform this operation when necessary.

[0104] The in-vehicle voice interaction method provided by the embodiment of the present invention identifies the driver's emotional state based on the driver's emotional behavior information, identifies the driver's behavior state based on the driver's driving state information, determines the driver's driving state based on the emotional state and the behavior state, and performs voice interaction based on the driving state. The present invention can determine the driver's current driving state by combining emotional behavior and driving behavior, and meet the driver's driving needs through voice interaction, avoid traffic accidents, and improve driving comfort and safety.

[0105] In this embodiment, a vehicle-mounted voice interaction device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0106] This embodiment provides a vehicle-mounted voice interaction device, which is applied to a vehicle zone controller, such as Figure 5 As shown, including:

[0107] The information acquisition module 501 is used to acquire the driver's emotional behavior information and driving status information.

[0108] The state recognition module 502 is used to recognize the driver's emotional state according to the emotional behavior information, and to recognize the driver's behavioral state according to the driving state information.

[0109] The voice interaction module 503 is used to determine the driving state of the driver according to the emotional state and the behavioral state, and perform voice interaction based on the driving state.

[0110] In some optional implementations, the state identification module 502 includes:

[0111] The emotional state recognition unit is used to perform a comprehensive analysis of facial information and sound information based on a preset emotion recognition algorithm to determine the driver's emotional state, where the emotional state includes at least one of the following: emotional state type, emotion intensity, emotion duration, emotion change trend, and attention impact degree.

[0112] The behavior state recognition unit is used to perform a comprehensive analysis of speed information and steering information based on a preset behavior analysis algorithm to determine the driver's behavior state, which includes at least one of the following: driving safety, driving stability, driving habits, potential accident risks, driving efficiency, traffic impact degree and driving behavior improvement needs.

[0113] In some optional implementations, the voice interaction module 503 includes:

[0114] The driving state determination unit is used to determine the driving state according to at least one of the emotional state type, emotional intensity, emotional duration, emotional change trend or attention impact degree, and the driving state includes: fatigue driving state, angry driving state and dangerous driving state.

[0115] A driving status confirmation unit is used to confirm the driving status based on at least one of driving safety, driving stability, driving habits, potential accident risks, driving efficiency, traffic impact or driving behavior improvement needs.

[0116] The voice interaction unit is used to obtain the corresponding preset interactive reply language according to the driving state, and to play the preset interactive reply language in a preset tone and preset speed.

[0117] In some optional embodiments, the device also includes: a display interaction module, which is used to obtain a corresponding preset interaction interface according to the driving state and display the preset interaction interface; obtain corresponding warning information according to the driving state and display the warning information.

[0118] In some optional embodiments, the device also includes: an emergency handling module, which is used to execute preset emergency handling operations when it is determined that the driver is in an angry driving state or a dangerous driving state. The preset emergency handling operations include at least one of the following: limiting the vehicle speed, issuing a warning signal, and contacting emergency services.

[0119] In some optional embodiments, the device also includes: a personalized setting module, which is used to perform personalized interactive reply settings, personalized interactive interface settings, personalized warning information settings and / or personalized emergency handling operation settings according to the driving status.

[0120] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0121] The in-vehicle voice interaction device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0122] The embodiment of the present invention also provides a computer device having the above Figure 5 The in-vehicle voice interaction device shown.

[0123] See also Figure 6 , Figure 6is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 6 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0124] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0125] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0126] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0127] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0128] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0129] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0130] The embodiment of the present invention also provides a car, comprising: a camera for collecting facial information of a driver; a microphone for collecting voice information of the driver; a sensor for collecting driving status information of the driver; a controller connected to the camera, the microphone and the sensor for executing the above Figure 1 , Figure 2 or Figure 4 The in-vehicle voice interaction method is shown.

[0131] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A vehicle-mounted voice interaction method, characterized in that: The method comprises: Obtain the driver's emotional behavior information and the driving status information of the vehicle being driven; identifying the driver's emotional state according to the emotional behavior information, and identifying the driver's behavioral state according to the driving state information; The driving state of the driver is determined according to the emotional state and the behavioral state, and voice interaction is performed based on the driving state.

2. The method according to claim 1, characterized in that The emotional behavior information includes: facial information and voice information, the driving state information includes: speed information and steering information, and identifying the emotional state of the driver according to the emotional behavior information and identifying the behavioral state of the driver according to the driving state information includes: Based on a preset emotion recognition algorithm, comprehensively analyzing the facial information and the sound information to determine the driver's emotional state, wherein the emotional state includes at least one of the following: emotional state type, emotional intensity, emotional duration, emotional change trend, and attention impact degree; Based on a preset behavior analysis algorithm, the speed information and the steering information are comprehensively analyzed to determine the driver's behavior state, which includes at least one of the following: driving safety, driving stability, driving habits, potential accident risks, driving efficiency, traffic impact level and driving behavior improvement needs.

3. The method according to claim 2, characterized in that The determining the driving state of the driver according to the emotional state and the behavioral state, and performing voice interaction based on the driving state, includes: Determine the driving state according to at least one of the emotional state type, the emotional intensity, the emotional duration, the emotional change trend or the attention influence degree, the driving state including: fatigue driving state, angry driving state and dangerous driving state; confirming the driving state based on at least one of the driving safety, the driving stability, the driving habits, the potential accident risk, the driving efficiency, the degree of traffic impact, or the driving behavior improvement needs; A corresponding preset interactive reply language is obtained according to the driving state, and the preset interactive reply language is voice-played according to a preset tone and a preset speech speed.

4. The method according to claim 1, characterized in that: When performing voice interaction based on the driving state, it also includes: Acquire a corresponding preset interactive interface according to the driving state, and display the preset interactive interface; Corresponding warning information is obtained according to the driving status, and the warning information is displayed.

5. The method according to claim 3, characterized in that: Also includes: When it is determined that the driver is in the angry driving state or the dangerous driving state, a preset emergency handling operation is performed, and the preset emergency handling operation includes at least one of the following: limiting the vehicle speed, issuing a warning signal, and contacting emergency services.

6. The method according to any one of claims 1 to 5, characterized in that Also includes: According to the driving status, personalized interactive reply speech setting, personalized interactive interface setting, personalized warning information setting and / or personalized emergency handling operation setting are performed.

7. A vehicle-mounted voice interaction device, characterized in that: The device comprises: An information acquisition module, used to obtain the driver's emotional behavior information and driving status information; a state recognition module, used to recognize the emotional state of the driver according to the emotional behavior information, and to recognize the behavioral state of the driver according to the driving state information; A voice interaction module is used to determine the driving state of the driver according to the emotional state and the behavioral state, and perform voice interaction based on the driving state.

8. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the in-vehicle voice interaction method according to any one of claims 1 to 6 by executing the computer instructions.

9. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the in-vehicle voice interaction method according to any one of claims 1 to 6.

10. An automobile, characterized in that: include: A camera for collecting the driver’s facial information; A microphone, used to collect voice information of the driver; A sensor, used to collect driving status information of the driver; A controller is connected to the camera, the microphone and the sensor, and is used to execute the vehicle-mounted voice interaction method according to any one of claims 1 to 6.

Images