Voice interaction method and device for intelligent driving, vehicle and program product

Abstract

The invention provides a voice interaction method for intelligent driving, electronic equipment, a vehicle and a computer program product. The voice interaction method comprises the steps of determining that a vehicle passenger is in a frightened state based on a vehicle passenger behavior; acquiring driving state information of the vehicle; judging whether the vehicle is in an abnormal state based on the driving state information; in response to the abnormal state of the vehicle, determining an abnormal type and an abnormal reason of the abnormal state; obtaining voice information based on the abnormity type and the abnormity reason; and providing the voice information to the vehicle occupant. According to the technical scheme, the experience of intelligent driving of the vehicle is improved.

Description

Technical Field

[0001] The present disclosure relates to a voice interaction method, electronic equipment, vehicle and computer program product for intelligent driving. Background Art

[0002] With the development of intelligent driving, more and more users are choosing intelligent driving. In the process of intelligent driving, as the driving environment changes, the vehicle will automatically take emergency avoidance or emergency braking actions. After the vehicle takes such actions, it will bring discomfort or fright to the passengers. This will reduce the intelligent driving experience. Summary of the invention

[0003] The present disclosure provides a voice interaction method, electronic device, vehicle and computer program product for intelligent driving.

[0004] According to one aspect of the present disclosure, a voice interaction method for intelligent driving is provided, including: determining that a vehicle occupant is in a frightened state based on the vehicle occupant's behavior; obtaining the vehicle's driving state information; judging whether the vehicle is in an abnormal state based on the driving state information; in response to the vehicle being in an abnormal state, determining the abnormal type and abnormal cause of the abnormal state; obtaining voice information based on the abnormal type and abnormal cause; and providing the voice information to the vehicle occupant.

[0005] In one aspect of the technical solution, during the intelligent driving process, when a vehicle occupant is frightened, the vehicle's operating conditions are identified to determine whether the vehicle occupant's fright is caused by an abnormal state of the vehicle. If so, the source of the occupant's nervousness is evaluated and the occupant is interacted with, thereby improving the vehicle's intelligent driving experience.

[0006] According to at least one embodiment of the present disclosure, determining the abnormal type and abnormal cause of the abnormal state includes: acquiring driving state information of a first time length, wherein the first time length is a period of time before a time point when the vehicle occupant is determined to be in a frightened state; and determining the abnormal type and abnormal cause of the abnormal state based on the driving state information of the first time length.

[0007] According to at least one embodiment of the present disclosure, determining that a vehicle occupant is in a frightened state based on the vehicle occupant's behavior includes: monitoring the vehicle occupant's facial expression images, outputting speech or body movements; determining that the vehicle occupant is in a frightened state based on the vehicle occupant's facial expression images, outputting speech or body movements.

[0008] According to at least one embodiment of the present disclosure, obtaining driving state information of a vehicle includes: obtaining motion parameters of the vehicle and driving environment information around the vehicle.

[0009] According to at least one embodiment of the present disclosure, determining the abnormal type and abnormal cause of the abnormal state includes: determining the abnormal type of the abnormal state according to the motion parameter; and determining the abnormal cause of the abnormal state according to the driving environment information.

[0010] According to at least one embodiment of the present disclosure, obtaining voice information based on the abnormality type and the abnormality cause includes: acquiring the abnormality type and the abnormality cause; and generating the voice information through the abnormality type, the abnormality cause, and soothing language.

[0011] According to at least one embodiment of the present disclosure, based on the exception type and the exception cause, voice information is obtained, including: obtaining the exception type and the exception cause; sending the exception type and the exception cause to a server; receiving voice information from the server, wherein the voice information is generated by the server according to the exception type and the exception cause.

[0012] According to another aspect of the present disclosure, an electronic device is provided, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, so that the processor executes the voice interaction method of any embodiment of the present disclosure.

[0013] According to another aspect of the present disclosure, a vehicle is provided, the vehicle including the electronic device as described above.

[0014] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the voice interaction method of any embodiment of the present disclosure is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0016] Figure 1 It is a schematic application scenario diagram according to an embodiment of the present disclosure.

[0017] Figure 2 It is a schematic flowchart of a voice interaction method for intelligent driving according to an embodiment of the present disclosure.

[0018] Figure 3 It is a schematic interaction diagram of a voice interaction method according to an embodiment of the present disclosure.

[0019] Figure 4 It is a schematic flowchart of a voice interaction method for intelligent driving according to another embodiment of the present disclosure.

[0020] Figure 5 is a schematic structural block diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] The present disclosure is further described in detail below in conjunction with the accompanying drawings and examples. It is understood that the specific examples described herein are only used to explain the relevant content, rather than to limit the present disclosure. It should also be noted that, for ease of description, only the parts related to the present disclosure are shown in the accompanying drawings.

[0022] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0023] Intelligent driving technology uses sensors, controllers, actuators and other equipment built into the vehicle to achieve automatic control during vehicle driving. During intelligent driving, if an emergency occurs (the vehicle is in an abnormal state), the vehicle can handle it on its own, such as braking or avoiding. Due to the vehicle's autonomous movement, the occupants may feel uncomfortable or even frightened. After careful research, the inventor of the present disclosure has proposed the following technical solution to reduce the discomfort of vehicle occupants during intelligent driving or to appease customers, so as to improve the vehicle's intelligent driving experience.

[0024] To this end, the present disclosure proposes a technical solution for use in intelligent driving. After detecting that a vehicle occupant is in a frightened state, the reason why the vehicle occupant is frightened is fully evaluated based on the vehicle's driving state information, and voice interaction is performed with the vehicle occupant in combination with actual scenarios, thereby enhancing the intelligent experience of vehicle driving.

[0025] For the convenience of description and to make the technical solution of the present disclosure easier to understand, the terms of the present disclosure are first explained before describing the technical solution of the present disclosure.

[0026] Intelligent driving: Also known as autonomous driving or adaptive driving, it can include assisted driving and unmanned driving. Intelligent driving technology uses advanced sensors (such as radar, lidar, cameras, etc.), computer systems and artificial intelligence algorithms to enable vehicles to complete driving tasks autonomously without or with only a small amount of human intervention. According to the standards of the Society of Automotive Engineers (SAE International), intelligent driving is divided into six levels from Level 0 to Level 5, where Level refers to the automation level of autonomous driving technology. The higher the level, the less human intervention.

[0027] The technical solution disclosed in the present invention is applied to vehicles and can be implemented through electronic devices on board the vehicle. The electronic devices can obtain features such as facial images, output speech, body movements, etc. of vehicle occupants, and judge the status of vehicle occupants based on these features. In addition, the electronic devices can also receive information from vehicle sensing devices (radar, camera) to judge the driving environment around the vehicle. Figure 1 The vehicle control unit, motion recognition system, expression recognition system, voice recognition system, assisted driving system and voice system of the vehicle 100 are shown in the figure. The systems mentioned here can be various parts of the electronic equipment, and the functions of each system can also be realized by executing the corresponding software program through the vehicle control unit. The motion recognition system is used to recognize the body movements of the vehicle occupants, the expression recognition system is used to recognize the expressions of the vehicle occupants, and the voice recognition system is used to recognize the input speech of the vehicle occupants. The assisted driving system is used to receive information from the vehicle's perception system, control the vehicle, and realize functions such as assisted driving. The voice system can be the vehicle's computer, which can interact with the vehicle occupants by voice, etc. The vehicle computer is an in-vehicle infotainment device.

[0028] The technical solution disclosed in the present invention can be implemented only on the vehicle, or can be implemented through the interaction between the vehicle and the cloud server. In the case of interaction between the vehicle 100 and the cloud server 200, the vehicle can communicate with the cloud server through the communication interface of the vehicle to transmit data, etc. As an example, before providing voice information to the vehicle occupants, the vehicle 100 can provide some information to the cloud server through the vehicle's communication structure, and the large model deployed in the cloud server 200 generates the corresponding voice information, and then the cloud server 200 provides the generated voice information to the vehicle, and then broadcasts it to the vehicle occupants.

[0029] The technical solution of the present invention is described in detail below. It should be emphasized that the technical solution of the present invention is a technical solution for the intelligent driving process.

[0030] According to an embodiment of the present disclosure, a voice interaction method for intelligent driving is provided. The voice interaction method can be used in the intelligent driving process. When the vehicle is in the intelligent driving state and the vehicle occupants are frightened, the technical solution of the present disclosure is implemented.

[0031] Figure 2 FIG. 4 is a schematic flow chart of a method for dealing with startle of a vehicle occupant according to an embodiment of the present disclosure. Figure 2 As shown, the startle handling method may include steps S210 to S260.

[0032] In step S210, it is determined that the vehicle occupant is in a frightened state based on the behavior of the vehicle occupant. The frightened state of the vehicle occupant is caused by the abnormal state of the vehicle. For example, when the vehicle is in emergency braking or emergency avoidance, the vehicle occupant will have a frightened reaction. The frightened reaction of the vehicle occupant can be reflected through facial expressions, language, and actions. In addition, when the vehicle is in an accident, the vehicle occupant will also have a frightened reaction. Therefore, in the case of a vehicle accident, the technical solution of the present disclosure is also used.

[0033] According to the technical solution disclosed in the present invention, the behavior of the vehicle occupant can be the facial expression behavior, body behavior, language behavior, etc. of the vehicle occupant. The facial expression change or body change of the occupant can be monitored by the camera in the car. Through the facial expression change or body change, it is determined whether the vehicle occupant is in a frightened state. For example, when it is monitored that the facial expression of the vehicle occupant changes from a relaxed expression to a nervous expression, it can be determined that the vehicle occupant is in a frightened state. The output speech of the vehicle occupant can also be monitored by the microphone of the voice recognition system in the car. When the microphone monitors the traffic hazard words issued by the vehicle occupant, it can be determined that the vehicle occupant is in a frightened state. When the microphone monitors the vehicle occupant to issue traffic hazard words such as "too dangerous" and "almost hit", it is determined that the vehicle occupant is in a frightened state. The description here is only an example, and monitoring can also be carried out in other ways. For example, when the occupant's screaming, rapid breathing, and increased sweating are monitored, they can all be used as conditions for judging that the vehicle occupant is in a frightened state.

[0034] Furthermore, the state of the vehicle occupant can be judged based on at least two of facial expressions, body movements, and language behaviors. When judging the state based on two behaviors, if the vehicle occupant is judged to be in a frightened state based on both behaviors, it is determined that the vehicle occupant is in a frightened state. Preferably, the judgment is made based on the facial expressions and language behaviors of the vehicle occupant.

[0035] In step S220, the driving state information of the vehicle is obtained. According to the technical solution of the present disclosure, after determining that the vehicle occupant is in a frightened state, the driving state information of the vehicle can be obtained.

[0036] The driving state information obtained may be driving state information of a first duration, wherein the first duration is a period of time before the time point when the vehicle occupant is determined to be in a frightened state. For example, if the vehicle occupant is determined to be in a frightened state at the first moment, driving state information of a period of time before the first moment is obtained, such as driving state information of the first 10 seconds.

[0037] Driving status information may include motion parameters of the vehicle and driving environment information around the vehicle. Examples of motion parameters may include acceleration, speed, steering angle, etc. The acceleration, steering angle, etc. of the vehicle can be obtained based on vehicle motion sensors such as acceleration sensors and steering angle sensors of the chassis control system. The electronic device receives motion parameter signals from the chassis control domain. The detection of motion parameters can also be obtained through vehicle-mounted perception systems such as cameras and radars. For example, when obtaining the vehicle speed and acceleration through a camera, the position changes of feature points in continuous frame images in different frames can be analyzed, the displacement of the feature points can be calculated, and then the time interval can be calculated in combination with the timestamp of each frame image, and the speed and acceleration of the vehicle can be calculated by the location and time interval.

[0038] The driving environment information around the vehicle can be obtained through the vehicle-mounted sensing device. As an example, the vehicle-mounted sensing device may include a camera, a radar, an ultrasonic sensor, etc. Taking the camera as an example, during the driving process of the vehicle, the camera will collect image data in front of the vehicle in real time and store these image data. Based on these image data, the driving environment information, such as vehicles, pedestrians, objects, etc., can be obtained through deep learning models.

[0039] In step S230, based on the driving state information, it is determined whether the vehicle is in an abnormal state. When it is determined that the vehicle is not in an abnormal state, the execution of this method ends, otherwise step S240 is executed. When determining that the vehicle is in an abnormal state, the determination can be made based on the vehicle's motion parameters and the driving environment information around the vehicle.

[0040] The abnormal state of the vehicle can be judged based on the vehicle's motion parameters. For example, when the lateral acceleration of the vehicle is greater than 0.2g, it can be judged that the vehicle is in an abnormal state of emergency avoidance. Because in a non-emergency avoidance state, the lateral acceleration of the vehicle is usually not greater than 0.1g. It is also possible to judge whether the vehicle is in an emergency avoidance state based on the driving environment information around the vehicle, such as collecting an image in front of the vehicle, and extracting corresponding features from the image to calculate the lateral acceleration change of the vehicle.

[0041] In the technical solution of the present disclosure, after determining that the vehicle is in an abnormal state, in step S240, the abnormal type and abnormal cause of the abnormal state can be determined. Among them, the abnormal type can be determined according to the motion parameters of the vehicle. For example, when the deceleration of the vehicle is greater than the deceleration threshold, it is determined that the vehicle is abnormal and the abnormal type is emergency braking. For example, when the lateral acceleration of the vehicle is greater than the lateral acceleration threshold, it can be determined that the vehicle is abnormal and the abnormal type is emergency avoidance. As described above, the driving environment information around the vehicle can be determined by a vehicle-mounted sensing device such as a camera. The abnormal cause of the abnormal state can be determined by the driving environment information. Take the camera as an example for illustration. For example, when it is determined that the abnormal type of the vehicle is emergency braking, the cause of the emergency braking can be confirmed in combination with the image captured by the camera. In addition, multi-sensor data fusion technology can be used for determination, such as a camera combined with a radar for confirmation. Multi-sensor data fusion technology can refer to existing technologies and will not be described too much here. Generally speaking, the abnormal type corresponds to a variety of abnormal causes, so the abnormal causes corresponding to the abnormal type can be stored in advance. When determining the cause of the abnormality based on the driving environment information, the information obtained by the on-board sensing device can be compared with the stored abnormality information.

[0042] Taking emergency braking as an example, the on-board sensing device can determine that the cause of the emergency braking is: sudden deceleration of the vehicle in front, pedestrians crossing the road, unidentified objects, etc.

[0043] In step S250, based on the abnormal type and abnormal cause, voice information is obtained. In the case of an abnormal state of the vehicle, it is necessary to provide voice information to the vehicle occupants to relieve tension. The voice information can be generated by the on-board electronic device, or by the cloud server and then provided to the on-board electronic device. Here, the on-board electronic device generates voice information as an example for explanation. In one embodiment, the vehicle control unit can obtain the abnormal type and the abnormal cause, and generate voice information according to the abnormal type and the abnormal cause. The generated voice information includes the abnormal type and the abnormal cause. For example, the generated voice information is: because of the "abnormal cause", the "abnormal type" was performed. As an example, in the case where the abnormal cause is the sudden deceleration of the front vehicle, the voice information can be: because the front vehicle suddenly decelerated, emergency braking was performed. According to the technical solution of the present disclosure, the generated voice information is used to voice comfort the occupants in a frightened state, and the voice information can be further combined with other information to generate the voice information, and the other information can include soothing language, time, etc. For example, combined with the time of occurrence of the abnormal state, the voice information generated in this way can be: X seconds ago, because of the "abnormal cause", the "abnormal type" was performed.

[0044] In some cases, after emergency braking is determined by motion parameters, the reason for the emergency braking may not be found through driving environment information. For example, although emergency braking is performed, the vehicle perception system does not detect abnormal conditions on the road surface. In this case, the abnormal cause can be classified as unknown. The generated voice information can be: Emergency braking was performed due to unknown reasons.

[0045] Furthermore, the corresponding voice information can be generated by an artificial intelligence language model. The model can be deployed in the vehicle or on a cloud server. As a specific example, the cause of the abnormality and the type of abnormality can be input into the model, and the model generates corresponding voice information based on the cause of the abnormality and the type of abnormality.

[0046] In the present disclosure, a large language model is preferably used to generate voice information, so as to achieve a better interactive effect for vehicle occupants. For example, in the case of emergency avoidance, the abnormality type is emergency avoidance, and the abnormality cause is the dropping of an unknown object. After the model receives the abnormality type and the abnormality cause, the following voice information can be generated: "Dear young master, 5 seconds ago, an unknown object was dropped by the vehicle in front. It is huge in size, but its density is unknown. Therefore, in order to protect your safety, I made an emergency avoidance without authorization. Please don't be afraid, young master." In this voice information, emergency avoidance is the abnormality type, and the dropping of an unknown object is the abnormality cause.

[0047] After the voice information is obtained, in step S260, the voice information is provided to the vehicle occupants. In the process of providing the voice information, a speaker of the vehicle, such as a speaker of a vehicle computer, is automatically started.

[0048] Since there is no or little human intervention during intelligent driving, the vehicle occupants may not pay too much attention to the road conditions. Therefore, if the vehicle is suddenly in an abnormal state, it will scare the vehicle occupants. If the vehicle occupants are not aware of the abnormal state, the vehicle occupants will continue to be nervous. Especially in the case of unmanned driving, the vehicle occupants do not need to pay attention to the road conditions at all, and sudden abnormalities will make the vehicle occupants extremely uneasy.

[0049] The vehicle occupant startle handling method disclosed in the present invention can be executed by a vehicle control unit of the vehicle. The vehicle control unit can be a VCU (Vehicle Control Unit) of the vehicle, which can serve as the core of the entire control system of the vehicle. The vehicle control unit can communicate with other components of the vehicle, for example, can communicate via a bus such as a CAN bus.

[0050] Figure 3The interactive schematic diagram of the voice interaction method for intelligent driving according to an example of the present disclosure is shown. The changes in facial expressions can be identified through the image of the in-vehicle camera, and the voice of the vehicle occupant can be identified through the in-vehicle microphone. In step 301, the in-vehicle camera acquires the facial image of the vehicle occupant in real time, and the in-vehicle microphone acquires the output speech of the in-vehicle occupant in real time. In step 302, the facial image and the output speech are sent to the whole vehicle control unit. Expression recognition, speech recognition, and action recognition can be performed in the whole vehicle control unit. After the whole vehicle control unit receives the facial image and the output speech, in step 303, expression recognition processing and speech recognition are performed. In another embodiment of the present disclosure, it is also possible to determine whether the vehicle occupant is in a frightened state by identifying the body movements of the vehicle occupant through the body image of the vehicle occupant taken by the in-vehicle camera. Facial expressions, output speech and body movements can be used in combination, for example, at least two of them are used. In step 304, the whole vehicle control unit determines that the vehicle occupant is in a frightened state according to the recognition result.

[0051] After determining that the vehicle occupant is in a frightened state, the vehicle control unit may request the driving assistance system for driving state information in step 305. After receiving the driving state information request, the driving assistance system may send the driving state information to the vehicle control unit in step 306. The driving state information sent is information for a period of time before it is determined that the vehicle occupant is in a frightened state. As another example, the driving assistance system may directly determine whether the vehicle is in an abnormal state.

[0052] In the present disclosure, the judgment of the whole vehicle control unit is taken as an example for explanation. After the whole vehicle control unit receives the driving state information of the vehicle, it judges whether the vehicle is in an abnormal state. In step 307, the whole vehicle control unit judges that the vehicle is in an abnormal state. In step 308, the whole vehicle control unit sends a wake-up signal to the voice system. The whole vehicle control unit can be the vehicle's car machine, which can interact with the occupants through the screen and voice. After the voice system receives the wake-up signal, it is awakened and can send an information acquisition request to the whole vehicle control unit in step 309, requesting to obtain information such as the abnormal type and abnormal cause from the whole vehicle control unit. After receiving the information acquisition request from the voice system, in step 310, the whole vehicle control unit can send information such as the abnormal type and abnormal cause to the voice system, or generate voice information based on the abnormal type and abnormal cause and send it to the voice system, and control the voice system to provide voice information to the vehicle occupants. The generation of voice information can refer to the above description. In step 311, the voice system can broadcast voice information to comfort the occupants in a frightened state.

[0053] According to the voice interaction method disclosed in the present invention, it is run when the vehicle is in an intelligent driving state. Intelligent driving may include unmanned driving and assisted driving. In the case of unmanned driving, the voice method disclosed in the present invention can be directly implemented. Because when unmanned driving, no manual intervention is required to perform driving behavior. Therefore, vehicle occupants usually do not pay attention to road conditions. If the vehicle is in an abnormal state, it will have a greater impact on the occupants. In the case of assisted driving, it may include a step of determining whether the vehicle is in an assisted driving state, and when it is determined that it is in an assisted driving state, the voice interaction method disclosed in the present invention is run. During the assisted driving process, sometimes manual intervention in driving behavior is required. In the case of manual intervention, the driver will pay attention to the road conditions, and voice interaction is not required at this time; in the case of no manual intervention, voice interaction is performed.

[0054] exist Figure 4 A schematic flow chart of a voice interaction method according to a further embodiment of the present disclosure is shown in FIG. Figure 4 Examples and Figure 2 The difference between the embodiments is that: Figure 4 Step S420 is added in. For the description of steps S410, S430-470, please refer to Figure 2 The relevant contents of the embodiment will not be repeated here.

[0055] In step S420, it is determined whether the vehicle is in an intelligent driving state (the intelligent driving here may be assisted driving). If it is determined that the vehicle is not in an intelligent driving state, the execution of the method ends. If it is determined that the vehicle is in an intelligent driving state, step S430 is executed. It should be noted that step S420 may also be executed before step S410, or may be executed after step S430.

[0056] According to a further embodiment of the present application, an electronic device is also provided. The electronic device is a vehicle-mounted electronic device. The electronic device may include the vehicle control unit, action recognition system, expression recognition system, voice recognition system, auxiliary driving system and voice system as described above. The electronic device may also be a vehicle controller of a vehicle. Figure 5 A schematic block diagram of an electronic device according to an embodiment is shown in FIG.

[0057] like Figure 5As shown, the electronic device 500 may include a processor 510, a memory 520, and a communication bus 530. The number of processors 510 may be one or more, and the number of memories 520 may be one or more. The memory 520 stores a computer-readable program. When the electronic device 400 is running, the processor 510 communicates with the memory 520 via the communication bus 530, and the computer-readable program is called and executed by the processor 510, thereby realizing the voice interaction method of the present application. The communication bus 530 is used to realize communication between various hardware in the electronic device.

[0058] In addition, the electronic device may include more components or adopt Figure 5 Different settings. For example, all or part of the steps of the method of the present application may be implemented by hardware, and the hardware may communicate with the processor 510 or the like via the communication bus 530 .

[0059] The processor may include a general-purpose processor, such as a central processing unit, a microcontroller unit, a network processor or other conventional processors; it may also include a special-purpose processor, including a neural network processor, a graphics processor, a digital signal processor, etc. In addition, some of the multiple processors may be general-purpose processors, and the other part may be special-purpose processors. In the technical solution of the present disclosure, the processor may be a processor of a vehicle controller, and the processor may also include multiple processors, such as a motion recognition system, an expression recognition system, a speech recognition system, an assisted driving system, and a speech system.

[0060] The memory includes, but is not limited to, random access memory, read-only memory, programmable read-only memory, erasable programmable read-only memory, electrically erasable programmable read-only memory, etc. The number of memories may be multiple. If the processor only includes the processor of the vehicle controller, the memory is the memory of the vehicle controller. If there are multiple memories, the memory may be the memory of each system, which is used to store the computer program called and executed by the processor of each system.

[0061] As Figure 5 For example, the processor 510 may call the computer program in the memory 520 to execute the voice interaction method of each embodiment of the present disclosure. For example, after calling the computer program, the processor may execute the following process: determine that the vehicle occupant is in a frightened state based on the vehicle occupant's behavior; obtain the vehicle's driving state information; determine whether the vehicle is in an abnormal state based on the driving state information; in response to the vehicle being in an abnormal state, determine the abnormal type and abnormal cause of the abnormal state; obtain voice information based on the abnormal type and abnormal cause; and provide voice information to the vehicle occupant.

[0062] According to an embodiment of the present application, a vehicle is also provided, which includes the above-mentioned electronic device.

[0063] The present disclosure also provides a readable storage medium, in which a computer program is stored, and the computer program is used to implement the above method when executed by a processor. "Readable storage medium" can be any device that can contain, store, communicate, propagate or transmit a program for use in an instruction execution system, device or equipment or in combination with these instruction execution systems, devices or equipment. More specific examples of readable storage media include the following: an electrical connection portion (electronic device) with one or more wirings, a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and editable read-only memory (EPROM or flash memory), an optical fiber device, and a portable read-only memory (CDROM), etc.

[0064] The present disclosure also provides a computer program product. The method of the present disclosure can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed, the process or function of the present disclosure is executed in whole or in part.

[0065] The computer program or instructions may be stored in a readable storage medium or transmitted from one readable storage medium to another readable storage medium, for example, the computer program or instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired or wireless means. The readable storage medium may be any available medium that can be accessed or a data storage device such as a server, data center, etc. that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; it may also be an optical medium, such as a digital video disk; it may also be a semiconductor medium, such as a solid state drive. The computer readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.

[0066] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0067] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0068] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0069] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0070] In the description of this specification, the description with reference to the terms "one embodiment / method", "some embodiments / methods", "example", "specific example", or "some examples" etc. means that the specific features, structures, or characteristics described in conjunction with the embodiment / method or example are included in at least one embodiment / method or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment / method or example. Moreover, the specific features, structures, or characteristics described may be combined in any one or more embodiments / methods or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments / methods or examples described in this specification and features of different embodiments / methods or examples, unless they are contradictory.

[0071] Those skilled in the art should understand that the above embodiments are only for the purpose of clearly illustrating the present disclosure, and are not intended to limit the scope of the present disclosure. For those skilled in the art, other changes or modifications may be made based on the above disclosure, and these changes or modifications are still within the scope of the present disclosure.

Claims

1. A voice interaction method for intelligent driving, characterized in that: include: determining that the vehicle occupant is in a state of fright based on the vehicle occupant's behavior; Obtain vehicle driving status information; Based on the driving state information, determining whether the vehicle is in an abnormal state; In response to the vehicle being in an abnormal state, determining an abnormal type and an abnormal cause of the abnormal state; Based on the abnormal type and abnormal cause, obtaining voice information; as well as The voice information is provided to a vehicle occupant.

2. The voice interaction method according to claim 1, characterized in that: Determining the abnormal type and abnormal cause of the abnormal state includes: Acquiring driving state information of a first time period, wherein the first time period is a period of time before a time point when the vehicle occupant is determined to be in a frightened state; and Based on the driving state information of the first duration, an abnormal type and an abnormal cause of the abnormal state are determined.

3. The voice interaction method according to claim 1, characterized in that: Determining that a vehicle occupant is in a frightened state based on the vehicle occupant's behavior includes: Monitor the facial expressions of vehicle occupants and output speech or body movements; It is determined that the vehicle occupant is in a frightened state based on the vehicle occupant's facial expression image, output speech or body movement.

4. The voice interaction method according to claim 1, characterized in that: Get the vehicle's driving status information, including: Obtain the vehicle's motion parameters and the driving environment information around the vehicle.

5. The voice interaction method according to claim 4, characterized in that: Determining the abnormal type and abnormal cause of the abnormal state includes: The abnormal type of the abnormal state is determined according to the motion parameter; and the abnormal cause of the abnormal state is determined according to the driving environment information.

6. The voice interaction method according to claim 4, characterized in that: Based on the abnormal type and abnormal cause, voice information is obtained, including: Obtain the exception type and exception cause; The voice information is generated by the abnormality type, abnormality cause, and soothing language.

7. The voice interaction method according to claim 4, characterized in that: Based on the abnormal type and abnormal cause, voice information is obtained, including: Obtain the exception type and exception cause; Send the exception type and exception reason to the server; Receive voice information from the server, wherein the voice information is generated by the server according to the abnormal type and the abnormal cause.

8. An electronic device, characterized in that: include: A memory storing execution instructions; as well as A processor, wherein the processor executes the execution instructions stored in the memory so that the processor executes the voice interaction method according to any one of claims 1 to 7.

9. A vehicle, characterized in that: Comprising the electronic device as claimed in claim 8.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the voice interaction method according to any one of claims 1 to 7 is implemented.

Images