Interaction method and apparatus, and vehicle

Abstract

The present application provides an interaction method and apparatus, and a vehicle. The interaction method may be applied to the field of intelligent cockpits. The method comprises: acquiring a first intent of a user in a vehicle cockpit, wherein the first intent is associated with another vehicle, and the first intent is associated with a negative emotion of the user; performing de-emotion processing on the first intent to obtain a second intent; and sending the second intent to the another vehicle. The present application can be applied to electric vehicles or intelligent vehicles, to prevent drivers from experiencing anger and taking retaliatory action during interactions between vehicles, thereby avoiding traffic accidents, and facilitating improvement of the driving experience of users.

Description

Interaction method, device and vehicle

[0001] The present application claims priority to the Chinese Patent Application No. 202410869842.4, filed on June 28, 2024, entitled "Interaction method, device and vehicle", and the Chinese Patent Application No. 202411993129.7, filed on December 30, 2024, entitled "Interaction method, device and vehicle", the contents of which are incorporated herein by reference in their entirety. TECHNICAL FIELD

[0002] The present application relates to the field of intelligent cockpit, and more particularly, to an interaction method, device and vehicle. BACKGROUND

[0003] When a vehicle is driving on a road, it will have an interaction relationship with a nearby driving vehicle. For example, a series of interactions occur, such as overtaking, following, lane changing, cutting in, sudden braking, or queuing, etc. When these interactions occur between vehicles, the drivers cannot perceive each other's emotions, which is easy to cause misunderstanding, and further leads to the driver's angry emotion. In a more serious case, it can lead to the driver's retaliatory behavior (e.g., cutting in, stopping, etc.), resulting in a traffic accident. SUMMARY

[0004] The present application provides an interaction method, device and vehicle, which can avoid the driver's angry emotion and retaliatory behavior when the interaction occurs between vehicles, and further avoid the occurrence of traffic accidents, which helps to improve the user's driving experience.

[0005] In a first aspect, an interaction method is provided, the method comprising: obtaining a first intention of a user in a vehicle cabin, the first intention being associated with another vehicle and the first intention being associated with a negative emotion of the user; performing de-emotional processing on the first intention to obtain a second intention; and sending the second intention to the other vehicle.

[0006] In combination with the first aspect, in some implementations of the first aspect, the performing de-emotional processing on the first intention to obtain a second intention comprises: performing de-emotional processing on the first intention to obtain a de-emotional processed intention; and adding polite language in the de-emotional processed intention to obtain the second intention.

[0007] In some implementations of the first aspect, before the first intention of the user in the vehicle cabin is acquired, the method further includes: acquiring a first voice instruction of the user; and determining the first intention according to the first voice instruction; and before the second intention is sent to the other vehicle, the method further includes: when the first voice instruction does not include slot information corresponding to the first intention, determining information of the other vehicle according to the first intention and data collected by a sensor outside the vehicle cabin; or when the first voice instruction includes the slot information corresponding to the first intention, determining the information of the other vehicle according to the first slot information and the data collected by the sensor.

[0008] In some implementations of the first aspect, before the first intention of the user in the vehicle cabin is acquired, the method further includes: acquiring a second voice instruction of the user, the second voice instruction including the first intention; and the de-emotional processing of the first intention to obtain the second intention includes: inputting the second voice instruction, information of the vehicle, and surrounding environment information of the vehicle into an inference model to obtain the second intention and information of the other vehicle.

[0009] In some implementations of the first aspect, the first intention of the user in the vehicle cabin is acquired by: acquiring driving behavior of the user and driving records of the other vehicle; and determining the first intention according to the driving behavior of the user and the driving records of the other vehicle; and before the second intention is sent to the other vehicle, the method further includes: determining information of the other vehicle according to the first intention and data collected by a sensor outside the vehicle cabin.

[0010] In some implementations of the first aspect, before the second intention is sent to the other vehicle, the method further includes: determining that the second intention satisfies a road regulation.

[0011] In some implementations of the first aspect, the second intention is sent to the other vehicle by: sending the second intention to the other vehicle according to signal strength and / or signal quality of an environment in which the vehicle is located.

[0012] In some implementations of the first aspect, the second intention is sent to the other vehicle according to signal strength and / or signal quality of an environment in which the vehicle is located by: when the signal strength is greater than or equal to a preset signal strength, and / or the signal quality is greater than or equal to a preset signal quality, sending information of the other vehicle and the second intention to a cloud server, so that the cloud server sends the second intention to the other vehicle according to the information of the other vehicle.

[0013] With reference to the first aspect, in some implementations of the first aspect, the sending the second intention to the other vehicle comprises: sending the second intention to the other vehicle through near field communication when the signal strength is less than a preset signal strength and / or the signal quality is less than a preset signal quality.

[0014] The second aspect provides an interaction method, comprising: receiving a second intention from a vehicle, the second intention being an intention obtained by de-emotionalizing a first intention of a user in a cabin of the vehicle; and controlling a prompt device to prompt the user with the second intention.

[0015] With reference to the second aspect, in some implementations of the second aspect, the controlling the prompt device to prompt the user with the second intention comprises: determining a first driving opinion according to the second intention; and controlling the prompt device to prompt the user with the second intention and the first driving opinion.

[0016] With reference to the second aspect, in some implementations of the second aspect, the controlling the prompt device to prompt the user with the second intention comprises: controlling the prompt device to prompt the user with the second intention according to a state of the user in a main driving area.

[0017] With reference to the second aspect, in some implementations of the second aspect, the second intention indicates an abnormal driving behavior, and the method further comprises: controlling a vehicle light and / or a vehicle exterior projection information to display first information, the first information being used to apologize and / or thank the user in the vehicle.

[0018] The third aspect provides an interaction method, comprising: obtaining a first input of a user in a cabin of a vehicle, information of the vehicle, and environmental information around the vehicle; determining a first output according to the first input, the information of the vehicle, and the environmental information; and sending the first output to another vehicle.

[0019] With reference to the third aspect, in some implementations of the third aspect, the determining the first output according to the first input, the information of the vehicle, and the environmental information comprises: inputting the first input, the information of the vehicle, and the environmental information into an inference model to obtain the first output.

[0020] With reference to the third aspect, in some implementations of the third aspect, the information of the vehicle comprises at least one of a speed of the vehicle, a type of a road where the vehicle is located, and a type of a lane where the vehicle is located.

[0021] With reference to the third aspect, in some implementations of the third aspect, the environmental information comprises data collected by a sensor of the vehicle.

[0022] In a fourth aspect, an interaction apparatus is provided, which comprises units or modules for performing the method of any possible implementation of the first aspect to the third aspect.

[0023] In a fifth aspect, an interaction apparatus is provided, which comprises a processing unit and a storage unit, wherein the storage unit is configured to store instructions, and the processing unit is configured to execute the instructions stored in the storage unit, so that the interaction apparatus performs any possible method of the first aspect to the third aspect.

[0024] In a sixth aspect, an interaction system is provided, which comprises a perception system and a computing platform, wherein the computing platform comprises any possible apparatus of the fourth aspect or the fifth aspect.

[0025] In a seventh aspect, a vehicle is provided, which comprises the apparatus of the fourth aspect or the fifth aspect, or the system of the sixth aspect.

[0026] In an eighth aspect, a computer program product is provided, which comprises computer program code, when the computer program code is run on a computer, so that the computer performs any possible method of the first aspect to the third aspect.

[0027] It should be noted that the above computer program code can be stored in the first storage medium in whole or in part, wherein the first storage medium can be packaged together with the processor, or packaged separately from the processor, and the embodiments of the present application do not make a specific limitation in this regard.

[0028] In a ninth aspect, a computer readable medium is provided, which stores program code, when the computer program code is run on a computer, so that the computer performs any possible method of the first aspect to the third aspect.

[0029] In a tenth aspect, a chip system is provided, which comprises a processor, configured to invoke computer programs or computer instructions stored in a memory, so that the processor performs any possible method of the first aspect to the third aspect.

[0030] In combination with the tenth aspect, in a possible implementation, the processor is coupled with the memory through an interface.

[0031] In combination with the tenth aspect, in a possible implementation, the chip system further comprises the memory, and the memory stores the computer programs or computer instructions.

[0032] In a eleventh aspect, the present application provides an interaction method, comprising: obtaining a first input of a user in a vehicle cabin and environment information of a surrounding of the vehicle, the environment information comprising information of another vehicle; determining a first output according to the first input and the environment information; and sending the first output to the another vehicle or a terminal device associated with the another vehicle.

[0033] Based on the above technical solution, the first output can be determined through the input of the user in the vehicle cabin and the environment information, so that the first output can be sent to the another vehicle or the terminal device. In this way, the information isolation between drivers of two vehicles can be broken, and the interaction between vehicles can be realized.

[0034] In some possible implementation manners, the first input can comprise a voice input of the user.

[0035] In some possible implementation manners, the first input can comprise an input of the user to one or more components in the vehicle. For example, the one or more components can comprise, but are not limited to, a steering wheel, a light, a horn, an accelerator pedal or a brake pedal.

[0036] In some possible implementation manners, the first input can comprise physiological feature information of the user. For example, the physiological feature information can comprise blood pressure, heart rate, facial expression, etc. of the user.

[0037] In combination with the eleventh aspect, in some implementation manners of the eleventh aspect, the first input is associated with a negative emotion of the user, and the first output comprises a de-emotional output.

[0038] Based on the above technical solution, when the input of the user is associated with a negative emotion of the user, the information that needs to be interacted with the another vehicle can be processed again based on the input of the user and the environment information, so that the de-emotional processing is realized. In this way, the purpose of effective and friendly communication between vehicles can be achieved, the purpose of transmitting effective information without transmitting negative emotions is achieved, and the probability of conflict between drivers is reduced.

[0039] In some possible manners, the first input comprises impolite language, and the first output further comprises polite language converted from the impolite language.

[0040] Based on the above technical solution, while the de-emotional processing is performed, the vehicle can also convert the impolite language of the user, so that the converted polite language is sent to the another vehicle. In this way, the purpose of effective and friendly communication between vehicles can be further achieved, the purpose of transmitting effective information without transmitting negative emotions is achieved, and the probability of conflict between drivers is reduced.

[0041] In some possible implementation manners, the method further includes: controlling the prompting device to output the third output, the third output including an output result for soothing the user in the cabin.

[0042] Based on the technical solution above, when the negative emotion of the user associated with the first input is determined, the output result for soothing the user in the cabin can be output based on the first input and the environmental information. In this way, while realizing friendly interaction between the vehicle and another vehicle, the user in the cabin can also be soothed, which helps to relieve the negative emotion of the user and improve the driving experience of the user.

[0043] For example, the third output can include soothing language.

[0044] For example, the third output can include an execution instruction for one or more components in the cabin.

[0045] With reference to the eleventh aspect, in some implementation manners of the eleventh aspect, determining the first output according to the first input and the environmental information includes: inputting the first input and the environmental information into a first inference model to obtain the first output.

[0046] Based on the technical solution above, the first input and the environmental information can be input into the first inference model, so that the first output can be obtained. In this way, end-to-end input and output can be realized through the first inference model.

[0047] For example, the first inference model can be a multi-modal model.

[0048] With reference to the eleventh aspect, in some implementation manners of the eleventh aspect, the method further includes: obtaining information of the vehicle; and wherein determining the first output according to the first input and the environmental information includes: determining the first output according to the first input, the information of the vehicle, and the environmental information.

[0049] Based on the technical solution above, through the input of the user in the cabin of the vehicle, the information of the vehicle, and the environmental information, the first output can be determined, so that the first output can be sent to another vehicle or a terminal device. In this way, information isolation between drivers in two vehicles can be broken, and interaction between vehicles can be realized. Meanwhile, by combining the information of the vehicle, the accuracy of the first output result can be further improved.

[0050] In some possible implementation manners, the information of the vehicle includes historical driving records of the vehicle.

[0051] In some possible implementation manners, the information of the vehicle includes one or more of a speed, an acceleration, and a position of the vehicle.

[0052] In some possible implementation manners of the eleventh aspect, the determining the first output according to the first input, the information of the vehicle, and the environmental information comprises: inputting the first input, the information of the vehicle, and the environmental information into a second inference model to obtain the first output.

[0053] According to the technical solution, the first input, the information of the vehicle, and the environmental information can be input into the first inference model, and thus the first output can be obtained. In this way, the end-to-end input and output can be implemented by using the second inference model.

[0054] In some possible implementation manners, the first inference model and the second inference model can be the same model.

[0055] In some possible implementation manners of the eleventh aspect, the method further comprises: determining information of another vehicle according to the first input and the environmental information.

[0056] According to the technical solution, the information of another vehicle can be determined according to the input of the user in the vehicle cabin and the environmental information, and thus the first output can be sent to the another vehicle or the terminal device. In this way, the information of the vehicle to be interacted can be determined in combination with the input of the user and the environmental information, and the accuracy of the determined vehicle to be interacted can be ensured.

[0057] In some possible implementation manners of the eleventh aspect, the determining the information of another vehicle according to the first input and the environmental information comprises: inputting the first input and the environmental information into a third inference model to obtain the information of another vehicle.

[0058] In some possible implementation manners, the first inference model, the second inference model, and the third inference model can be the same model.

[0059] In some possible implementation manners of the eleventh aspect, the information of another vehicle comprises information of a license plate of the another vehicle; and the sending the first output to the another vehicle or the terminal device comprises: sending the first output and the information of the license plate of the another vehicle to a cloud server, so that the cloud server sends the first output to the another vehicle or the terminal device based on the information of the license plate of the another vehicle.

[0060] According to the technical solution, the vehicle can send the first output and the information of the license plate of the another vehicle to the cloud server, so that the cloud server can send the first output to the another vehicle by using the information of the license plate of the another vehicle. By forwarding the information by using the cloud server, the information isolation between drivers of the two vehicles can be broken, and the interaction between the vehicles can be implemented.

[0061] In some possible implementations, the cloud server stores an association between a license plate of the vehicle and identification information of the terminal device (e.g., a mobile phone or the vehicle).

[0062] With reference to the eleventh aspect, in some implementations of the eleventh aspect, sending the first output to the other vehicle or the terminal device includes sending the first output to the other vehicle through a short-distance communication technology.

[0063] With reference to the eleventh aspect, in some implementations of the eleventh aspect, the method further includes determining a first control instruction according to the first input and the environmental information, the first control instruction being associated with one or more actuators, and controlling the one or more actuators to execute the first control instruction.

[0064] Based on the technical solutions described above, the control instruction can also be obtained according to the first input and the environmental information. In this way, by executing the control instruction on the one or more actuators, the user can be helped to perform a corresponding operation, such as editing a light language, expressing gratitude, expressing apologies, and the like.

[0065] The twelfth aspect provides an interaction method, which includes obtaining a first output of another vehicle and environmental information around the vehicle, determining a second output according to the first output and the environmental information, and controlling a prompt device to output the second output.

[0066] Based on the technical solutions described above, the vehicle can obtain the second output according to the output of the other vehicle and the environmental information around the vehicle, and control the prompt device to output the second output. In this way, the information isolation between drivers of the two vehicles can be broken, and the interaction between the vehicles can be implemented.

[0067] With reference to the twelfth aspect, in some implementations of the twelfth aspect, determining the second output according to the first output and the environmental information includes determining the second output according to data collected by a sensor in a cabin of the vehicle, the first output, and the environmental information.

[0068] Based on the technical solutions described above, the data collected by the sensor in the cabin of the vehicle can be combined when the second output is determined. In this way, the second output can be more easily accepted by a user in the current cabin, and the driving experience of the user can be improved.

[0069] With reference to the twelfth aspect, in some implementations of the twelfth aspect, determining the second output according to the first output and the environmental information includes determining the second output according to first information of a driver in the vehicle, the first output, and the environmental information, the first information including one or more of driving proficiency, driving habits, or physiological characteristic information.

[0070] Based on the above technical solution, driver information can be incorporated when determining the second output. This allows the second output to better match the driver's profile, resulting in different outputs for different driver profiles. This helps improve the vehicle's intelligence and enhances the user's driving experience.

[0071] In conjunction with the twelfth aspect, in some implementations of the twelfth aspect, a second output is determined based on the first output and environmental information, including: determining the second output based on the historical driving records of other vehicles around the vehicle, the first output, and environmental information.

[0072] Based on the above technical solution, the historical driving records of other vehicles around the vehicle can be considered when determining the second output. This makes the second output more accurate and more acceptable to users in the cabin.

[0073] In conjunction with the twelfth aspect, in some implementations of the twelfth aspect, the second output is determined based on the first output and environmental information, including: inputting the first output and environmental information into the inference model to obtain the second output.

[0074] In a thirteenth aspect, this application provides an interactive device, comprising: an acquisition unit for acquiring a first input from a user in a vehicle cabin and environmental information surrounding the vehicle, the environmental information including information about another vehicle; a determination unit for determining a first output based on the first input and the environmental information; and a sending unit for sending the first output to another vehicle or a terminal device, the terminal device being associated with the other vehicle.

[0075] In conjunction with aspect thirteen, in some implementations of aspect thirteen, the first input is associated with the user's negative emotions, and the first output includes de-emotionalized output.

[0076] In conjunction with aspect thirteen, in some implementations of aspect thirteen, a unit is defined for: inputting a first input and environmental information into a first inference model to obtain a first output.

[0077] In conjunction with aspect thirteen, in some implementations of aspect thirteen, the acquisition unit is also used to acquire vehicle information; the determination unit is used to: determine the first output based on the first input, the vehicle information, and the environmental information.

[0078] In conjunction with aspect thirteen, in some implementations of aspect thirteen, a unit is defined for: inputting the first input, vehicle information, and environmental information into the second inference model to obtain the first output.

[0079] In conjunction with aspect thirteen, in some implementations of aspect thirteen, the determining unit is also used to: determine information about another vehicle based on the first input and environmental information.

[0080] With reference to the thirteenth aspect, in some implementations of the thirteenth aspect, the determining unit is configured to: input the first input and the environment information into a third inference model to obtain the information of the other vehicle.

[0081] With reference to the thirteenth aspect, in some implementations of the thirteenth aspect, the information of the other vehicle includes information of a license plate of the other vehicle; and the sending unit is configured to: send the first output and the information of the license plate of the other vehicle to a cloud server, so that the cloud server sends the first output to the other vehicle or the terminal device based on the information of the license plate of the other vehicle.

[0082] With reference to the thirteenth aspect, in some implementations of the thirteenth aspect, the apparatus further includes a control unit, and the determining unit is further configured to determine a first control instruction according to the first input and the environment information, the first control instruction being associated with one or more actuators; and the control unit is configured to control the one or more actuators to execute the first control instruction.

[0083] The fourteenth aspect provides an interaction apparatus, which includes: an obtaining unit configured to obtain a first output of an other vehicle and environment information around the vehicle; a determining unit configured to determine a second output according to the first output and the environment information; and a control unit configured to control a prompting apparatus to output the second output.

[0084] With reference to the fourteenth aspect, in some implementations of the fourteenth aspect, the determining unit is configured to determine the second output according to data collected by a sensor in a cabin of the vehicle, the first output and the environment information.

[0085] With reference to the fourteenth aspect, in some implementations of the fourteenth aspect, the determining unit is configured to determine the second output according to first information of a driver in the vehicle, the first output and the environment information, the first information including one or more of driving proficiency, driving habit or physiological characteristic information.

[0086] With reference to the fourteenth aspect, in some implementations of the fourteenth aspect, the determining unit is configured to determine the second output according to historical driving records of other vehicles around the vehicle, the first output and the environment information.

[0087] With reference to the fourteenth aspect, in some implementations of the fourteenth aspect, the determining unit is configured to input the first output and the environment information into an inference model to obtain the second output.

[0088] The fifteenth aspect provides an interaction apparatus, which includes a processing unit and a storage unit, wherein the storage unit is configured to store instructions, and the processing unit executes the instructions stored in the storage unit, so that the interaction apparatus executes any possible method in the eleventh aspect or the twelfth aspect.

[0089] In a sixteenth aspect, an interaction system is provided, the system comprising a perception system and a computing platform, wherein the computing platform comprises any possible apparatus of the thirteenth aspect to the fifteenth aspect.

[0090] In a seventeenth aspect, a vehicle is provided, the vehicle comprising any possible apparatus of the thirteenth aspect to the fifteenth aspect, or comprising the system of the sixteenth aspect.

[0091] In an eighteenth aspect, a computer program product is provided, the computer program product comprising: computer program code which, when run on a computer, causes the computer to perform any possible method of the eleventh aspect or the twelfth aspect.

[0092] It should be noted that the computer program code can be stored in whole or in part on a first storage medium, wherein the first storage medium can be packaged together with the processor or packaged separately from the processor, and the embodiments of the present application do not make a specific limitation in this regard.

[0093] In a nineteenth aspect, a computer readable medium is provided, the computer readable medium storing program code which, when run on a computer, causes the computer to perform any possible method of the above eleventh aspect or the twelfth aspect.

[0094] In a twentieth aspect, the embodiments of the present application provide a chip system, the chip system comprising a processor, configured to invoke a computer program or computer instructions stored in a memory, so as to cause the processor to perform any possible method of the above eleventh aspect or the twelfth aspect.

[0095] In combination with the twentieth aspect, in a possible implementation manner, the processor is coupled with the memory through an interface.

[0096] In combination with the twentieth aspect, in a possible implementation manner, the chip system further comprises the memory, and the memory stores the computer program or the computer instructions. BRIEF DESCRIPTION OF DRAWINGS

[0097] FIG. 1 is a functional block diagram of a vehicle according to an embodiment of the present application.

[0098] FIGS. 2A-2C are schematic diagrams of an interaction scenario according to an embodiment of the present application.

[0099] FIG. 3 is another schematic diagram of an interaction scenario according to an embodiment of the present application.

[0100] FIG. 4 is another schematic diagram of an interaction scenario according to an embodiment of the present application.

[0101] FIG. 5 is another schematic diagram of an interaction scenario according to an embodiment of the present application.

[0102] FIGS. 6A-6B are another schematic diagram of an interaction scenario according to an embodiment of the present application.

[0103] FIG. 7 is another schematic diagram of an interaction scenario according to an embodiment of the present application.

[0104] FIGS. 8A-8C are another schematic diagram of an interaction scenario according to an embodiment of the present application.

[0105] FIG. 9 is a schematic flowchart of an interaction method according to an embodiment of the present application.

[0106] FIG. 10 is another schematic flowchart of an interaction method according to an embodiment of the present application.

[0107] FIG. 11 is another schematic flowchart of an interaction method according to an embodiment of the present application.

[0108] FIG. 12 is a system architecture diagram according to an embodiment of the present application.

[0109] FIG. 13 is another system architecture diagram according to an embodiment of the present application.

[0110] FIG. 14 is a schematic flowchart of an interaction method according to an embodiment of the present application.

[0111] FIG. 15 is a system architecture diagram according to an embodiment of the present application.

[0112] FIG. 16 is a schematic block diagram of an interaction apparatus according to an embodiment of the present application.

[0113] FIG. 17 is a schematic block diagram of an interaction apparatus according to an embodiment of the present application. DETAILED DESCRIPTION

[0114] The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; in this document, "and / or" merely describes an association relationship of associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone.

[0115] The prefix words such as "first", "second" are used in the embodiments of the present application only to distinguish different description objects, and have no limiting effect on the position, order, priority, quantity or content of the described objects. The use of prefix words such as ordinal numbers in the embodiments of the present application does not constitute a limitation on the described objects, and the description of the described objects should be referred to the description of the context in the claims or embodiments, and should not constitute an unnecessary limitation because of the use of such prefix words. In addition, in the description of the embodiments, unless otherwise stated, the meaning of "a plurality of" is two or more.

[0116] FIG. 1 is a functional block diagram of a vehicle 100 according to an embodiment of the present application. The vehicle 100 can include a perception system 110, a computing platform 120, a display device 130, and a sound emitting device 140, wherein the perception system 110 can include one or more sensors that sense information about the environment around the vehicle 100. For example, the perception system 110 can include a positioning system, which can be a global positioning system (GPS), a Beidou system, or other positioning systems. The perception system 110 can also include one or more of an inertial measurement unit (IMU), a laser radar, a millimeter wave radar, an ultrasonic radar, and a camera.

[0117] Some or all functions of the vehicle 100 can be controlled by the computing platform 120. The computing platform 120 can include one or more processors, such as processors 121 through 12n (n is a positive integer), which are circuits having a processing capability of signals. In one implementation, the processors can be circuits having an instruction reading and running capability, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a kind of microprocessor), a digital signal processor (DSP), or the like. In another implementation, the processors can be circuits having a certain function implemented by a logic relationship of hardware circuits, which is fixed or reconfigurable. For example, the processors can be hardware circuits implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD) such as a field programmable gate array (FPGA). In the reconfigurable hardware circuit, the processor loads a configuration document to implement the hardware circuit configuration. It can be understood that the processor loads an instruction to implement the functions of the above part or all units. In addition, the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as a kind of ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), or the like. In addition, the computing platform 120 can further include a memory for storing instructions, and some or all of the processors 121 through 12n can call and execute the instructions in the memory to implement corresponding functions.

[0118] The display device 130 in the cabin is mainly divided into two categories, the first category is a vehicle display screen, and the second category is a projection display screen, such as a head-up display (HUD). The vehicle display screen is a physical display screen and is an important part of the in-vehicle infotainment system. Multiple display screens can be provided in the cabin, such as a digital instrument display screen, a central control screen, a display screen in front of a passenger at a front passenger seat (also referred to as a passenger in a front area), a display screen in front of a passenger at a left rear seat, and a display screen in front of a passenger at a right rear seat, or even a vehicle window can be used as a display screen. The head-up display, also known as a head-up display system, is mainly used to display driving information such as speed, navigation, etc. on a display device (such as a windshield) in front of the driver. To reduce the driver's eye movement time and avoid pupil changes caused by the driver's eye movement, and to improve driving safety and comfort. The HUD includes, for example, a combiner-HUD (C-HUD) system, a windshield-HUD (W-HUD) system, and an augmented reality HUD (AR-HUD) system. It should be understood that other types of systems can also appear as technology evolves, and the present application does not limit this.

[0119] The sound generating device 140 can be a loudspeaker, a sound device, or a horn.

[0120] As described above, when the vehicle is driving on the road, it will interact with the vehicles driving nearby. For example, a series of interactions occur, such as overtaking, following, lane changing, cutting in, sudden braking, or queuing. When these interactions occur between vehicles, the drivers cannot perceive each other's emotions, which can easily lead to misunderstandings and further cause the drivers to have angry emotions. In more serious cases, it can lead to the driver's retaliatory behavior (such as cutting off, stopping, etc.), resulting in traffic accidents.

[0121] The embodiments of the present application provide an interaction method and device and a vehicle, which are based on information described in natural language to locate the vehicle, achieve the effect of information connection, and further process the interaction information of the user to achieve the purpose of effective and friendly communication, and achieve the purpose of transmitting effective information without transmitting negative emotions. In this way, when the interaction occurs between vehicles, the driver's angry emotions and retaliatory behavior can be avoided, and further traffic accidents can be avoided, which helps to improve the driving experience of the user.

[0122] For example, FIGS. 2A-2C show a schematic diagram of an interaction scenario provided by the embodiments of the present application.

[0123] As shown in FIG. 2A, when driver A driving vehicle 100 finds that vehicle 200 is changing lanes from the current lane to the lane where vehicle 100 is located during driving on the road, driver A can control vehicle 100 to slow down. In this way, driver A can generate dissatisfaction with driver B in the cabin of vehicle 200. At this time, driver A can control vehicle 100 to honk to express dissatisfaction.

[0124] Driver B in the cabin of vehicle 200 can confirm that driver A in the cabin of vehicle 100 generates dissatisfaction after finding that vehicle 100 honks. At this time, driver B can send voice information "help me say to the driver behind, sorry, there is an urgent matter at home".

[0125] After vehicle 200 obtains the voice information sent by the user, vehicle 200 can first convert the voice information into text content through an automatic speech recognition (ASR) module. Vehicle 200 can understand the real intention of the user to express and the current emotion of the user according to the text content.

[0126] For example, through a natural language understanding (NLU) engine, the intention of the user is obtained, for example, "expressing apology because there is an urgent matter at home", the slot is "the driver behind", and the current emotion of the user is apology. Vehicle 200 can determine the license plate information of vehicle 100 through the image collected by the camera outside the cabin. Vehicle 200 can send the intention of the user and the license plate information of vehicle 100 to the cloud server. The cloud server can find the address of vehicle 100 based on the license plate information of vehicle 100, and send the intention of the user to vehicle 100 according to the address of vehicle 100. At the same time, in response to obtaining the intention of the user, vehicle 200 can send voice information "OK, I will say it now" to the user.

[0127] Optionally, the vehicle 200 can determine the abnormal driving record of the vehicle 200 and the driving record of the vehicle 100 through the data collected by the sensors outside the cabin. For example, the vehicle 200 detects that the vehicle 100 performs an emergency brake (for example, the deceleration of the vehicle 100 is greater than or equal to a preset deceleration) and detects that the vehicle 100 honks or switches between high beam and low beam during the process of changing from the right lane to the left lane, so it can be determined that the vehicle 100 performs non-active courtesy to the vehicle 200. After obtaining the user's intention, the vehicle 200 can determine that the user's intention includes an expression of apology. According to the abnormal driving record of the vehicle 200 and the driving record of the vehicle 100, the vehicle 200 can send the content of thanks (for example, send love, thank his courtesy) to the cloud server in addition to the user's intention to the cloud server. At this time, the vehicle 200 can send voice information to the user "OK, I will tell him now, and I will send love to him on your behalf."

[0128] After the vehicle 100 receives the user's intention and the content of thanks from the vehicle 200 from the cloud server, it generates the text content of expressing apology "Dear car owner, the driver of the preceding vehicle expresses apology, there is an urgent matter at home" and the text content of expressing thanks "He sends love to you, thank you for your courtesy".

[0129] Optionally, the vehicle 100 can also determine the abnormal driving record of the vehicle 200 (for example, the vehicle 200 does not turn on the left turn signal when changing to the left lane) and the driving record of the vehicle 100 (for example, the vehicle 100 performs an emergency brake and honks after the emergency brake) through the data collected by the sensors outside the cabin. The vehicle 100 can determine that the driver A in the vehicle 100 is currently very angry based on the abnormal driving record of the vehicle 200 and the driving record of the vehicle 100, and the vehicle 100 can also process the text content twice. For example, in addition to expressing apology and thanks to the driver A in the vehicle 100, it can also add text content of emotional pacification (for example, don't be angry).

[0130] After the above-mentioned secondary processing process, the vehicle 100 can convert the secondary processed text content into voice information through a text to speech (TTS) module, and send the voice information "Dear car owner, the driver of the preceding vehicle expresses apology, there is an urgent matter at home, don't be angry. He sends love to you, thank you for your courtesy" to the driver A through the speaker in the cabin of the vehicle 100.

[0131] In one embodiment, during the process of changing from the right lane to the left lane, the vehicle 200 detects that the vehicle 100 performs an emergency braking (for example, the deceleration of the vehicle 100 is greater than or equal to a preset deceleration) and detects that the vehicle 100 honks or switches between low beam and high beam, etc., so as to determine that the vehicle 100 performs a non-active courtesy to the vehicle 200. At this time, the vehicle 200 can also generate and send the license plate information of the vehicle 100, the text content of apology "apologize to the owner of the rear vehicle" and the text content of expressing thanks "thank you for your courtesy" to the cloud server based on the abnormal driving record of the vehicle 200 and the driving record of the vehicle 100 without receiving the voice information of the driver B. The cloud server can find the address of the vehicle 100 according to the license plate information of the vehicle 100, and send the text content of apology and the text content of expressing thanks to the vehicle 100 based on the address. The vehicle 100 can determine that the driver A in the vehicle 100 is currently very angry based on the abnormal driving record of the vehicle 200 and the driving record of the vehicle 100, and the vehicle 100 can also process the text content twice. For example, in addition to expressing apology and thanks to the driver A in the vehicle 100, the text content of emotional pacification (for example, don't be angry) can be added. The vehicle 100 can send the voice information "Dear owner, the driver of the front vehicle expresses apology to you, there is something urgent at home, don't be angry, don't be angry. He sends you love, thank you for your courtesy" to the driver A through the speaker in the cabin.

[0132] As shown in FIG. 2B, during the process of driving the vehicle 100 on the road, the driver A finds that the vehicle 200 changes lanes from the current lane to the lane where the vehicle 100 is located, and the driver A controls the vehicle 100 to decelerate for courtesy consideration, so as to perform courtesy to the vehicle 200.

[0133] After the driver B in the cabin of the vehicle 200 finds that the vehicle 100 performs active courtesy, the driver B can send the voice information "help me say thank you to the owner of the rear vehicle". After obtaining the voice information sent by the user, the vehicle 200 can first convert the voice information into text content through the ASR module. The vehicle 200 can understand the real intention that the user wants to express and the current emotion of the user according to the text content.

[0134] For example, the vehicle 200 can determine that the user's intention is "expressing thanks", the slot is "the driver behind", and the user's current emotion is thanks. In response to obtaining the user's intention and the slot, the vehicle 200 can issue voice information "OK, I will tell him right away" to the user. At the same time, the vehicle 200 can determine the information of the license plate number of the vehicle 100 through the data collected by the camera outside the cabin. The vehicle 200 can send the user's intention and the license plate information of the vehicle 100 to the cloud server. The cloud server can determine the address of the vehicle 100 based on the license plate information of the vehicle 100, and send the user's intention to the vehicle 100 according to the address of the vehicle 100.

[0135] Alternatively, the vehicle 200 can determine the abnormal driving record of the vehicle 200 through the data collected by the sensor outside the cabin. For example, the vehicle 200 detects that the vehicle 100 performs a slow brake during the process of changing from the right lane to the left lane, and does not detect that the vehicle 100 issues a horn and switches the high beam and the low beam, etc., so as to determine that the vehicle 100 actively gives way to the vehicle 200. After obtaining the user's intention, the vehicle 200 can determine that the user's intention includes the content of expressing thanks. According to the abnormal driving record of the vehicle 200 and the driving record of the vehicle 100, the vehicle 200 can also add other content of thanks (such as sending love) to the user's intention. At this time, the vehicle 200 can issue voice information "OK, I will tell him right away, and I will send love on your behalf" to the user.

[0136] When the vehicle 100 receives the user's intention, the vehicle 100 can generate text content "Dear driver, the driver in front thanks you for your active courtesy and sends love to you" according to the user's intention. The vehicle 100 can convert the text content into voice information through the TTS module, and issue the voice information "Dear driver, the driver in front thanks you for your active courtesy and sends love to you" to the driver A through the loudspeaker in the cabin of the vehicle 100.

[0137] As shown in FIG. 2C, during the process of driving on the road, the driver A driving the vehicle 100 finds that the vehicle 200 changes lanes from the lane where it is currently located to the lane where the vehicle 100 is located, and controls the vehicle 100 to slow down. In this way, it may cause the driver A to generate dissatisfaction emotion to the driver B in the cabin of the vehicle 200. At this time, the driver A controls the vehicle 100 to issue a horn to express dissatisfaction emotion.

[0138] The driver B in the cabin of the vehicle 200 issues the voice information "why not just add a plug, why not just apologize" after discovering that the vehicle 100 honks. After obtaining the voice information issued by the user, the vehicle 200 can first convert the voice information into text content through the ASR module. The vehicle 200 can understand the real intention of the user and the current emotion of the user according to the text content.

[0139] For example, through the NLU engine, it is obtained that the intention of the user is "why not just apologize for adding a plug" and the emotion expressing the intention of the user is an emotion of dissatisfaction or impatience. The vehicle 200 can determine the abnormal driving record of the vehicle 200 through the data collected by the sensor outside the cabin, for example, the vehicle 200 detects that the vehicle 100 performs an emergency brake (for example, the deceleration of the vehicle 100 is greater than or equal to a preset deceleration) during the process of changing from the right lane to the left lane and detects that the vehicle 100 honks or switches the high beam and the low beam, etc., so as to determine that the vehicle 100 performs non-active courtesy to the vehicle 200. After obtaining the intention of the user, the vehicle 200 can determine that the slot information is the vehicle 100 based on the abnormal driving record of the vehicle 200 and the driving record of the vehicle 100. The vehicle 200 can determine the license plate information of the vehicle 100 through the image collected by the camera outside the cabin.

[0140] Optionally, the vehicle 200 can edit the intention of the user, for example, perform de-emotional expression on the intention of the user, and convert the emotion of dissatisfaction or impatience into an emotion of sincere apology. For example, after performing de-emotional expression, the intention of the user is "apologize to the driver of the rear vehicle for adding a plug".

[0141] Optionally, according to the abnormal driving record of the vehicle 200 and the driving record of the vehicle 100, the vehicle 200 can also add other content of thanks (for example, send a heart) in the intention of the user. At this time, the vehicle 200 can issue the voice information "dear driver, I have expressed my apology to the driver of the rear vehicle and sent a heart to him" to the user.

[0142] After receiving the intention of the user and other content of thanks from the vehicle 200, the vehicle 100 generates the text content expressing thanks "dear driver, the driver of the front vehicle thanks your courtesy and sends a heart to you". The vehicle 100 can convert the text content into voice information through the TTS module and issue the corresponding voice information to the driver A through the speaker in the cabin of the vehicle 100.

[0143] For example, FIG. 3 shows a schematic diagram of an interaction scenario provided by an embodiment of the present application.

[0144] As shown in FIG. 3, when the driver A driving the vehicle 100 finds that the opposite vehicle 200 turns on the high beam during the night driving, it will affect the driving of the driver A. In this way, the driver A can be caused to have an unsatisfied emotion. At this time, the driver A sends the voice information "Can the vehicle opposite turn off the high beam?". After obtaining the voice information sent by the user, the vehicle 100 can first convert the voice information into text content through the ASR module. The vehicle 200 can understand the real intention of the user and the current emotion of the user according to the text content.

[0145] For example, through the NLU engine, the intention of the user is "turn off the high beam", the slot is "the vehicle opposite", and the current emotion of the user is dissatisfaction or impatience. After obtaining the intention of the user, the vehicle 100 can edit the intention of the user, for example, the intention of the user after the emotion is removed is "turn off the high beam", or the vehicle 100 can also add polite language in the intention of the user, for example, the intention of the user after adding the polite language is "please turn off the high beam".

[0146] Alternatively, the vehicle 100 can also translate the intention of the user. For example, the intention of the user after the emotion is removed is "turn off the high beam". After translation, the real intention of the user is "switch the high beam to the low beam".

[0147] At the same time, the vehicle 100 can obtain the information of the vehicle 200 opposite to the vehicle 100 according to the slot information. For example, the vehicle 100 can determine the license plate information of the vehicle 200 through the image collected by the camera outside the cabin. The vehicle 100 can send the real intention of the user and the license plate information of the vehicle 200 to the cloud server. The cloud server can find the address of the vehicle 200 based on the license plate information of the vehicle 200, and send the real intention of the user after the emotion is removed and translation to the vehicle 200 according to the address of the vehicle 200.

[0148] After receiving the real intention of the user, the vehicle 200 can convert the real intention of the user. For example, the vehicle 200 can convert the intention of the user into an expression form that the driver B is easy to accept and give effective suggestions to the driver B.

[0149] For example, when the vehicle 200 detects that the driver B is a novice driver, the vehicle 200 can introduce the driver B the reason for turning off the high beam (for example, affecting the driving of the opposite driver) and the driving suggestion (suggestion to switch to the low beam). For example, the vehicle 200 can send the voice information "Dear car owner, you are now driving with the high beam, which may affect the driving of the opposite driver. The car owner opposite hopes you to switch to the low beam" to the driver B.

[0150] For another example, when the vehicle 200 detects that the driver B is an experienced driver, the vehicle 200 can send a more concise expression mode to the driver B, such as sending a voice information "Please switch to low beam for oncoming vehicles".

[0151] For example, FIG. 4 shows a schematic diagram of an interaction scenario provided by an embodiment of the present application.

[0152] As shown in FIG. 4, the driver A driving the vehicle 100 finds that the vehicle 200 is driving at a slow speed (for example, 65 kph) on the overtaking lane. In this way, the driver A can generate dissatisfaction with the driver B in the cabin of the vehicle 200. At this time, the driver A can send a voice information "Help me talk to the car in front of me. Don't take the fast lane if you drive so slowly. You are hindering others." After the vehicle 100 obtains the voice information sent by the user, the vehicle 100 can first convert the voice information into text content through an ASR module. The vehicle 100 can understand the real intention of the user and the current emotion of the user according to the text content.

[0153] For example, through the NLU engine, the user's intention is "to convey that the car in front of me should not take the fast lane if it drives so slowly", the slot is "the car in front of me", and the current emotion of the user is dissatisfaction or impatience. After obtaining the user's intention, the vehicle 100 can edit the user's intention, for example, de-emotionalize the user's intention to obtain the de-emotionalized user's intention, such as "to convey that the current speed in the fast lane is too slow". At the same time, the vehicle 100 can send a voice information "OK, I have expressed it to the other party."

[0154] Optionally, the vehicle 100 can also translate the user's intention. For example, the user's intention obtained after de-emotionalization is "to convey to the vehicle that the current speed in the fast lane is too slow". After translation, the real intention of the user can be "to convey to the vehicle to speed up in the fast lane or switch to the slow lane".

[0155] The vehicle 100 can obtain the information of the vehicle 200 located in front of the vehicle 100 according to the slot information. The vehicle 100 can determine the license plate information of the vehicle 200 through the data collected by the camera outside the cabin. The vehicle 100 can send the de-emotionalized user's intention and the license plate information of the vehicle 200 to the cloud server. The cloud server can find the address of the vehicle 200 based on the license plate information of the vehicle 200, and send the real intention of the user obtained after de-emotionalization and translation to the vehicle 200 according to the address of the vehicle 200.

[0156] After receiving the real intention from the user of the vehicle 100 from the cloud server, the vehicle 200 can translate the real intention of the user. For example, the vehicle 200 can translate the intention of the user into an expression form that is easy for the driver B to receive.

[0157] For example, when the vehicle 200 detects that the number of times that the vehicle 200 travels on the current road is less than or equal to a preset number, or when the vehicle 200 detects that the driver B is a novice driver, the vehicle 200 can introduce the speed limit information of the current expressway to the driver B and translate the intention of the user into an expression form that is easy for the driver B to receive. For example, the vehicle 200 can send the voice information "Dear car owner, Xia A reminds you that you are now on the expressway, the speed limit is 80-100 km / h, the current driving speed of the vehicle is 65 km / h, the driver behind is a bit anxious, please speed up" to the driver B.

[0158] For another example, when the vehicle 200 detects that the number of times that the vehicle 200 travels on the current road is greater than a preset number, or when the vehicle 200 detects that the driver B is an experienced driver, the vehicle 200 can translate the intention of the user into an expression form that is easy for the driver B to receive. For example, a more concise expression form such as the voice information "The driver behind hopes you to speed up or change lanes" can be sent to the driver B.

[0159] For example, FIG. 5 shows a schematic diagram of an interaction scenario provided by an embodiment of the present application.

[0160] As shown in FIG. 5, the driver A driving the vehicle 100 finds that the vehicle 200 is driving on the lane line in the process of driving on the lane. In this way, the driver A may generate dissatisfaction with the driver B in the cabin of the vehicle 200. At this time, the driver A can send the voice information "What is wrong with this person, how to drive the car". After obtaining the voice information sent by the user, the vehicle 100 can first convert the voice information into text content through the ASR module. The vehicle 100 can understand the real intention that the user wants to express and the current emotion of the user according to the text content.

[0161] For example, through the NLU engine, it is obtained that the intention of the user is "complaint about abnormal driving behavior" and the current emotion of the user is complaint or anger. After obtaining the intention of the user and the current emotion, the vehicle 100 can send the user the voice information of emotional appeasement (for example, "Don't be angry, it's not worth getting angry") and recommend the user to perform certain operations for emotional appeasement (for example, ask the user whether to play light music or open fragrance, etc.). For example, the vehicle 100 can send the voice information "Dear car owner, don't be angry, it's not worth getting angry, I can play a little light music for you, how about it?" through the loudspeaker in the cabin.

[0162] Optionally, the vehicle 100 can determine the information of the vehicle 200 with abnormal driving record according to the intention of the user and the data collected by the out-cabin sensor, for example, the vehicle 100 determines that the vehicle 200 is driving on the lane line by the data collected by the sensor and obtains the license plate information of the vehicle 200. The vehicle 100 can translate the intention of the user according to the abnormal driving record. For example, the real intention of the user after translation is "please control the vehicle to drive in the lane". The vehicle 100 can send the real intention of the user and the license plate information of the vehicle 200 to the cloud server. The cloud server can determine the address of the vehicle 200 based on the license plate information of the vehicle 200 and send the real intention of the user to the vehicle 200 according to the address of the vehicle 200.

[0163] The vehicle 200 can generate an expression mode easy for the driver B to accept after receiving the real intention of the user from the vehicle 100 from the cloud server.

[0164] For example, the vehicle 200 can introduce the current abnormal driving behavior and driving suggestion to the driver B when detecting that the driver B is a novice driver. For example, the vehicle 200 can send voice information to the driver B "Dear owner, you are currently driving on the lane line, which may affect the opposite driver, please drive the vehicle in the lane as soon as possible".

[0165] For another example, the vehicle 200 can send a more concise expression mode to the driver B when detecting that the driver B is a driver with more experience, such as sending voice information "attention, driving on the line".

[0166] For example, FIGS. 6A-6B show a schematic diagram of an interaction scenario provided by the embodiments of the present application.

[0167] As shown in FIG. 6A, the vehicle 200 is in an intelligent driving state and is about to turn around in front. The vehicle 200 can determine that the vehicle 100 is located behind the vehicle 200 by the data collected by the out-cabin sensor (for example, a camera located at the tail of the vehicle 200) and obtain the license plate information of the vehicle 100. At this time, the vehicle 200 can send the license plate information of the vehicle 100 and the indication information 1 to the cloud server, the indication information 1 being used to indicate that the vehicle 200 is in an intelligent driving state and is about to turn around in front. The cloud server can determine the address of the vehicle 100 based on the license plate information of the vehicle 100 and send the indication information 1 to the vehicle 100 based on the address.

[0168] The vehicle 100 can generate a prompt information after receiving the indication information 1 from the vehicle 200 sent by the cloud server. The prompt information can include the state of the vehicle 200 and the content prompting the driver A to pay attention. For example, the vehicle 100 can convert the prompt information into voice information through the TTS module and play the voice information through the loudspeaker in the cabin: “Dear car owner, the vehicle in front is in the intelligent driving state and wants to turn around, please pay attention”.

[0169] As shown in FIG. 6B, the vehicle 100 is in the intelligent driving state. The driver in the vehicle 200 sends voice information “How does this car drive” and expresses dissatisfaction through the horn when finding that the driving trajectory of the vehicle 100 is abnormal. The vehicle 200 can determine that the driving trajectory of the vehicle 100 is abnormal (for example, the frequency of lateral movement is too high) based on the information of the vehicle 200 and the driving information of the surrounding vehicles. The vehicle 200 can obtain the user’s intention “Please don’t always move laterally, the vehicle in front” based on the information of the vehicle 200 and the driving information of the vehicle 100. The vehicle 200 can send the user’s intention and the license plate information of the vehicle 100 to the cloud server. The cloud server can determine the address of the vehicle 100 based on the license plate information of the vehicle 100 and send the user’s intention to the vehicle 100 based on the address.

[0170] In response to receiving the user’s intention, the vehicle 100 can determine that the vehicle 200 is located behind the vehicle 100 and obtain the license plate information of the vehicle 200 based on the data collected by the cabin external sensor (for example, the camera located at the tail of the vehicle 200) and the user’s intention from the vehicle 200. At this time, the vehicle 100 can send the license plate information of the vehicle 200 and the indication information 2 to the cloud server, the indication information 2 being used to indicate that the vehicle 100 is in the intelligent driving state. The cloud server can determine the address of the vehicle 200 based on the license plate information of the vehicle 200 and send the indication information 2 to the vehicle 200 based on the address, the indication information 2 including the indication information that the vehicle 100 is in the intelligent driving state and the apology information.

[0171] The vehicle 200 can generate a prompt information after receiving the indication information 2 from the vehicle 100 sent by the cloud server. The prompt information can include the state of the vehicle 100 and the content prompting the driver to pay attention. For example, the vehicle 200 can convert the prompt information into voice information through the TTS module and play the voice information through the loudspeaker in the cabin: “Dear car owner, the vehicle in front is in the intelligent driving state, please forgive me”.

[0172] The above embodiments are described by taking the ASR module and the NLU module as examples to analyze the user's intention and slot information. The embodiments of the present application are not limited thereto. For example, the input (for example, voice input) of the user, the information of the vehicle, and the environmental information around the vehicle can be input into an inference model (or referred to as a large model or a multi-modal model), and the information of another vehicle (for example, license plate information) and the information communicated to the other vehicle can be output. The vehicle can send the information of the other vehicle and the information communicated to the other vehicle to the cloud server. Thus, the cloud server can send the communicated information to the other vehicle through the information of the other vehicle.

[0173] For example, FIG. 7 shows a schematic diagram of an interaction scenario provided by the embodiments of the present application.

[0174] As shown in FIG. 7, during the driving of the vehicle 100 on the road, the driver A finds that the vehicle 200 changes lanes from the current lane to the lane where the vehicle 100 is located, and controls the vehicle 100 to slow down. In this way, the driver A may cause dissatisfaction with the driver B in the cabin of the vehicle 200. At this time, the driver A controls the vehicle 100 to honk to express dissatisfaction.

[0175] After detecting the honking of the vehicle 100, the vehicle 200 can determine the surrounding environmental information through the data collected by the sensor. For example, the vehicle 200 determines through the surrounding environmental information that the driver needs to change lanes to the left due to the construction diversion in front. After determining that the vehicles in the left lane and the right lane need to alternate in the construction diversion scene, the vehicle 200 can output voice information "The honking vehicle owner may not know the situation of the road occupation construction, I will convey it to him" through the loudspeaker in the cabin.

[0176] The vehicle 200 can determine the license plate information of the vehicle 100 through the image collected by the camera outside the cabin. Thus, the vehicle 200 can send the license plate information of the vehicle 100 and the indication information 3 to the cloud server, and the indication information 3 is used to indicate that the lane where the vehicle 200 is currently located is under road occupation construction.

[0177] After receiving the indication information 3 from the vehicle 200 sent by the cloud server, the vehicle 100 can determine that the driver A honks may not be aware of the road occupation construction in the right lane, or the driver A may be aware of the road occupation construction in the right lane and unaware of the need to alternate in this scene according to the indication information 3, the abnormal driving record of the vehicle 200, and the driving record of the vehicle 100. The vehicle 100 can convey the driving opinion in the current scene to the user, for example, output voice information "Dear vehicle owner, the right lane is under road occupation construction, and currently needs to alternate, don't be too anxious!" through the loudspeaker in the cabin.

[0178] Optionally, the vehicle 100 can generate an expression mode easy for the driver A to accept according to the information of the driver A.

[0179] For example, when the vehicle 100 detects that the driver A is a novice driver, the vehicle 100 can introduce the vehicle 200 to the driver A about the non-abnormal driving of the vehicle 200 and driving suggestions. For example, the vehicle 100 can send the voice information “Dear car owner, the right lane is under construction, and alternating traffic is required at present. Don't be too anxious!” to the driver A.

[0180] For another example, when the vehicle 200 detects that the driver B is a driver with rich experience, the vehicle 200 can send a relatively concise expression mode to the driver A, such as sending the voice information “Attention, right lane under construction”.

[0181] For example, FIGS. 8A-8C show schematic diagrams of an interaction scenario provided by an embodiment of the present application.

[0182] As shown in FIG. 8A, when the user drives the vehicle 100 to a narrow lane and finds that a vehicle in front is parked on the narrow lane, causing the vehicle 100 to be unable to pass through. At this time, the driver in the vehicle 100 can send the voice information “Can you talk to the person in front and move his car? He is parked here and makes it difficult for others to pass through”. When the vehicle 100 collects the voice information, the vehicle 100 can input the voice information, the information of the vehicle 100 (for example, the speed of the vehicle, the information of the road where the vehicle is located), and the environmental information outside the cabin of the vehicle 100 into the reasoning model, so as to obtain a reasoning result. The reasoning result can include the voice information “OK, it has been expressed to the other party” conveyed to the driver of the vehicle 100, the license plate information of the vehicle 200, and the first information (for example, the text content “Your car seems to block the car behind you. The driver behind you is a little anxious to pass through”) conveyed to the vehicle 200.

[0183] In the embodiment of the present application, the input of the reasoning model can include one or more of voice, picture, text, and video stream (the picture and video stream can be the image or video stream collected by the sensor inside the cabin and the sensor outside the cabin).

[0184] Optionally, the reasoning model can be externally connected with knowledge (for example, road regulations, road conditions, and golden processing methods (the best way)). In this way, the general multi-modal model can become a special multi-modal model for the user driving the vehicle. The output of the reasoning model can be the intention of the user and the information of the other vehicle.

[0185] According to the current scene, the plug-in knowledge in the search reasoning model (for example, playing games in the car (racing type), actual driving scene) can be searched. For example, the user plays games through the co-driver screen in the car, issues a voice instruction "the car in front is so slow", the voice information and the state of the co-driver screen can be input into the reasoning model, and then text or voice "overtake it!" can be output. For another example, the user honks the horn when the front car is slow on the urban road, issues a voice instruction "the car in front is so slow", the voice information, the current speed of the vehicle, the lane where the vehicle is located, and the environmental information around the vehicle can be input into the reasoning model, and then voice information "the speed limit of this section of road is 50-70km / h, the front car is driving normally, please do not be in a hurry!" can be output.

[0186] The reasoning model can also output the execution information (for example, tone, intonation, dynamic effect, color, etc.) of other devices in the vehicle interior. For example, when the voice information "the speed limit of this section of road is 50-70km / h, the front car is driving normally, please do not be in a hurry!" is output, the instruction to control the sound producing device to play light music can also be output.

[0187] The vehicle 100 can send the license plate information of the vehicle 200 and the first information conveyed to the vehicle 200 to the cloud server. The cloud server can determine that the account corresponding to the license plate information of the vehicle 200 includes multiple devices, for example, the vehicle 200, the mobile phone, the smart watch and the tablet, according to the association relationship between the license plate information of the vehicle 200 and the saved license plate information and the account. The cloud server can send the first information to the multiple devices under the account.

[0188] As shown in FIG. 8B, when the mobile phone under the account of which the vehicle 200 logs in receives the first information, the mobile phone can display a prompt box through the display interface of the mobile phone, the prompt box including prompt information "Dear car owner, your vehicle seems to block the following vehicle, the driver of the following vehicle is a bit anxious to pass, do you want to remotely control the vehicle?", a confirmation control and a cancel control. When it is detected that the user clicks the confirmation control, the mobile phone can display environmental information around the vehicle 200 and a prompt box, the environmental information including a dashed box, the dashed box being a suggested moving area, the prompt box including prompt information "It is suggested that you move the vehicle to the dashed box to facilitate the passing of the following vehicle", a confirmation control and a cancel control. When it is detected that the user clicks the confirmation control, the mobile phone can send a control instruction to the vehicle, the control instruction being used to instruct the vehicle 200 to move from the current position to the area in the dashed box, and the mobile phone can also display the process of the vehicle 200 moving to the dashed box. When the mobile phone detects that the user issues voice information "Please help me to express apologies to the driver of the following vehicle", the mobile phone can input the voice information into the reasoning model, so that the text content expressed to the driver of the following vehicle "has moved the vehicle and expressed apologies to you" can be obtained. The cloud server can forward the text content to the vehicle 100, and the vehicle 100 can input the text content, the information of the vehicle and the environmental information around the vehicle into the reasoning model, so that the voice information "Dear car owner, the driver of the front vehicle has moved the vehicle and expressed apologies to you, and has caused you to waste valuable time" broadcasted to the driver of the vehicle 100 can be obtained.

[0189] As shown in FIG. 8C, the vehicle 100 can control the loudspeaker to issue the voice information "Dear car owner, the driver of the front vehicle has moved the vehicle and expressed apologies to you, and has caused you to waste valuable time".

[0190] FIG. 9 shows a schematic flowchart of an interaction method 900 provided by an embodiment of the present application. The method 900 includes the following steps:

[0191] S910, obtaining a first intention of a user in a vehicle cabin, the first intention being associated with another vehicle and the first intention being associated with a negative emotion of the user.

[0192] For example, the first intention of the user can be obtained through voice instruction analysis, such as the voice information "Can the car opposite turn off the high beam?" in FIG. 3. For another example, the voice information "Can you talk to the car in front and tell it not to block the fast lane by driving so slowly, which is obstructing others" in FIG. 4.

[0193] For another example, the first intention of the user can be obtained through the driving record of the vehicle (for example, the driver of the ego vehicle frequently switches between low beam and high beam, suddenly brakes and suddenly turns the steering wheel) and the driving record of the opposite vehicle (the opposite vehicle is currently using high beam).

[0194] For another example, the first intention of the user can be determined by the driving record of the vehicle (honking in the fast lane) and the driving record of the opposite vehicle (for example, too slow in the fast lane).

[0195] For another example, the first intention of the user can also be determined by the action (for example, the limb action of the user) and the expression (for example, the expression of impatience) of the user.

[0196] For another example, the first intention of the user can also be determined by the information of the face (for example, the face turning red) determined by the infrared sensor and the thermal imaging sensor.

[0197] For another example, if the emotion of the driver is not exposed, the first intention can also be obtained by the emotion of other members in the cabin. For example, the other members send voice information "Can the light of the opposite vehicle be turned off?"

[0198] For another example, the first intention can also be obtained by the information such as heart rate or blood pressure detected by the watch of the user.

[0199] For another example, the first intention can also be determined by the gripping force of the steering wheel.

[0200] S920, performing de-emotional processing on the first intention to obtain a second intention.

[0201] For example, after determining the first intention of the user, the first intention can be de-emotional processed to obtain a second intention after de-emotional processing.

[0202] S930, sending the second intention to the other vehicle.

[0203] In the embodiment of the application, after the user sends an intention with negative emotion, the negative emotion is de-emotional processed to obtain a second intention, and the intention after de-emotional processing is sent to the other vehicle. By twice processing the intention of the user, the negative emotion is not transmitted. In this way, when there is abnormal interaction between vehicles, the purpose of effective and friendly communication can be achieved, and effective information can be transmitted.

[0204] Optionally, the de-emotional processing on the first intention to obtain a second intention comprises: de-emotional processing on the first intention to obtain an intention after de-emotional processing; and adding polite language in the intention after de-emotional processing to obtain the second intention.

[0205] In the embodiment of the application, polite language can be added in the intention, so that the purpose of effective and friendly communication is further achieved, friction between drivers is avoided, and traffic accidents are avoided. In addition, the adjustment of language style is more conducive to communication.

[0206] The intention of the user can be recognized by the ASR module and the NLU module in the embodiments of the present application. The de-emotionalization processing can be performed when the NLU recognizes the intention of the user.

[0207] Optionally, before the first intention of the user in the vehicle cabin is acquired, the method further includes: acquiring a first voice instruction of the user; and determining the first intention according to the first voice instruction; and before the second intention is sent to the other vehicle, the method further includes: when the first voice instruction does not include slot information corresponding to the first intention, determining information of the other vehicle according to the first intention and data collected by a sensor outside the vehicle cabin; or when the first voice instruction includes the slot information corresponding to the first intention, determining the information of the other vehicle according to the first slot information and the data collected by the sensor.

[0208] In the embodiments of the present application, after the intention of the user is acquired through the voice instruction, if the slot information is included, the information (which can be at least one of the license plate information, the body color and the brand) of the other vehicle can be determined based on the slot information and the driving record of the other vehicle; and if the slot information is not included, the information (which can be at least one of the license plate information, the body color and the brand) of the other vehicle can be determined based on the intention of the user and the data collected by the sensor.

[0209] In the embodiments of the present application, the intention of the user can also be recognized by an inference model (for example, a multi-modal large model).

[0210] Optionally, before the first intention of the user in the vehicle cabin is acquired, the method further includes: acquiring a second voice instruction of the user, the second voice instruction including the first intention; and the de-emotionalization processing of the first intention to obtain a second intention includes: inputting the second voice instruction, information of the vehicle and surrounding environment information of the vehicle into an inference model to obtain the second intention and information of the other vehicle.

[0211] Optionally, the acquisition of the first intention of the user includes: acquiring a driving behavior of the user and a driving record of the other vehicle; and determining the first intention according to the driving behavior of the user and the driving record of the other vehicle; and before the second intention is sent to the other vehicle, the method further includes: determining information of the other vehicle according to the first intention and data collected by a sensor outside the vehicle cabin.

[0212] In the embodiments of the present application, the intention of the user can be determined based on the driving behavior of the user in the ego vehicle and the driving record of the other vehicle. For example, the ego vehicle detects that the user long-presses the horn or frequently switches between the low beam and the high beam, etc., it can be determined that the other vehicle has abnormal driving behavior, and thus the intention of the user can be determined (the intention is generally to instruct the other vehicle to remove the abnormal driving behavior).

[0213] Optionally, before the second intention is sent to the other vehicle, the method further comprises: determining that the second intention meets the road regulations.

[0214] In the embodiments of the present application, when it is determined that the first intention meets the road regulations, for example, in the scene of the road construction in FIG. 7, the user wants the other vehicle not to cut in, which actually does not comply with the road regulations (in the scene of the road construction, the vehicles should alternate passing).

[0215] The above can also determine whether the voice instruction of the user is legal through the reasoning model, and if it is legal, the reasoning model can determine how to communicate with the other party and give an output result. Whether it is legal can be used as an internal implementation process of the reasoning model.

[0216] Optionally, the second intention is sent to the other vehicle, comprising: sending the second intention to the other vehicle according to the signal strength and / or signal quality of the environment in which the vehicle is located.

[0217] Optionally, the second intention is sent to the other vehicle according to the signal strength and / or signal quality of the environment in which the vehicle is located, comprising: when the signal strength is greater than or equal to a preset signal strength, and / or the signal quality is greater than or equal to a preset signal quality, sending the information of the other vehicle and the second intention to a cloud server, so that the cloud server sends the second intention to the other vehicle according to the information of the other vehicle.

[0218] Optionally, the second intention is sent to the other vehicle, comprising: when the signal strength is less than a preset signal strength, and / or the signal quality is less than a preset signal quality, the second intention is sent to the other vehicle through near field communication.

[0219] FIG. 10 shows a schematic flowchart of an interaction method 1000 provided by the embodiments of the present application. The method 1000 comprises:

[0220] S1010, receiving a second intention from a vehicle, the second intention being an intention obtained after the first intention of a user in a cabin of the vehicle is de-emotionalized.

[0221] S1020, controlling a prompting device to prompt the user with the second intention.

[0222] Optionally, the control prompting device prompts the second intention to the user, including: determining a first driving opinion according to the second intention; and controlling the prompting device to prompt the second intention and the first driving opinion to the user.

[0223] For example, the receiving end device can adjust at least one of the voice, tone or mood of the voice information based on the state of the user. For example, if the driver is currently in a bad mood, the second intention can be conveyed in a humorous way to relieve the tension. For example, if the driver is currently in a good mood, the second intention can be directly conveyed.

[0224] Optionally, the control prompting device prompts the second intention to the user, including: controlling the prompting device to prompt the second intention to the user according to the state of the user in the main driving area.

[0225] Optionally, the second intention indicates an abnormal driving behavior, and the method further includes: controlling the vehicle light and / or the vehicle exterior projection information to display first information, the first information being used to apologize and / or thank the user in the vehicle.

[0226] FIG. 11 shows a schematic flowchart of an interaction method 1100 provided by an embodiment of the present application. The method 1100 includes:

[0227] S1110, obtaining a first input of a user in a vehicle cabin, information of the vehicle and environmental information around the vehicle.

[0228] For example, the first input can be voice information issued by the user, an action of the user, an expression of the user. Here, the user can be a user in a main driving area, or can also be a user in another area.

[0229] For example, the information of the vehicle can include one or more of the speed of the vehicle, the operation of the actuator of the vehicle (for example, frequent switching of low beam and high beam, sudden braking, too large change rate of steering wheel angle, horn, gripping force on the steering wheel, etc.), position, type of lane, type of road, data collected by sensors in the cabin (for example, data collected by infrared sensors and thermal imaging sensors).

[0230] For example, the environmental information around the vehicle includes data collected by sensors outside the vehicle cabin, such as images and video streams collected by cameras outside the cabin.

[0231] Optionally, the first input further includes an input of an electronic device (for example, a mobile phone, a smart watch or a smart bracelet) of the user. For example, heart rate or blood pressure detected by the watch of the user.

[0232] S1120, determine a first output according to the first input, the information of the vehicle and the environment information.

[0233] Optionally, the first output includes information (e.g., text content or voice information) communicated to another vehicle.

[0234] Optionally, the method further includes: outputting a control instruction of an in-vehicle cabin actuator (e.g., air conditioner, atmosphere lamp, fragrance or car audio) according to the first input, the information of the vehicle and the environment information.

[0235] Optionally, the determining the first output according to the first input, the information of the vehicle and the environment information includes: inputting the first input, the information of the vehicle and the environment information into an inference model to obtain the first output.

[0236] Optionally, the information of the vehicle includes at least one of a speed of the vehicle, a type of a road where the vehicle is located, and a type of a lane where the vehicle is located.

[0237] S1130, sending the first output to the other vehicle.

[0238] Optionally, the environment information includes data collected by a sensor of the vehicle.

[0239] Embodiments of the present application also provide an interaction system. The interaction system includes a sending device and a receiving device, wherein the sending device is configured to obtain a first intention of a user in a vehicle cabin, the first intention being associated with another vehicle and the first intention being associated with negative emotion of the user; the sending device is further configured to perform de-emotionalization processing on the first intention to obtain a second intention; the sending device is further configured to send the second intention to the other vehicle; and the receiving device is configured to control a prompt device to prompt the second intention to the user.

[0240] FIG. 12 and FIG. 13 respectively show schematic diagrams of system architectures provided by embodiments of the present application.

[0241] In embodiments of the present application, through an out-of-cabin sensor (e.g., a camera), appearance information (e.g., color, brand, model) of a neighboring vehicle can be recognized, a license plate number can be recognized, a relative position relationship between vehicles can be recognized, and current driving environment information (congestion, intersection, small road) can be recognized. Through an out-of-cabin microphone, a honking feature (whether to honk, honking times, urgency level) of the other vehicle can be recognized, surrounding environment information (whether to honk) can be recognized, a driving state and an abnormal driving behavior of the vehicle or the other vehicle can be recognized through a camera, a vehicle sensor and an advanced driving system (ADS), and reference information can be provided to a driver.

[0242] Through the intelligent engine system, with the assistance of a large model, the following can be achieved: understanding the fuzzy semantics described by the user and mapping to a specific vehicle in the vicinity; combining the current user's expression with the current dynamic and static environment information, the current user state, and the general processing flow (knowledge) for the current situation to effectively perform secondary processing and form accurate, reasonable, polite, and orderly communication content. Combined with the current state information, appropriate feedback suggestions are given to the user, and the user can be assisted to perform operations such as editing light language, expressing gratitude, and expressing apologies.

[0243] In the cloud server, information is transmitted through information matching or through a mobile network, or information exchange is completed through near-field transmission. This information exchange can protect the privacy of the driver without exposing personal information and reduce ambiguity in understanding, thereby improving communication efficiency and safety without the need for indirect expression through non-verbal means (flashing lights, sounding horns).

[0244] Based on natural language descriptions, information is mapped to nearby vehicle information and exchanged, breaking the information barrier between drivers and allowing communication to take place and emotions to be effectively dissipated.

[0245] In the embodiments of the present application, information expressed by the driver can be processed and filtered according to the driver's state, environmental state, and driving behavior at the time, and after extracting key information, secondary processing and summarization are performed for interactive transmission. The driver can complete a series of explicit expressions of intent, such as gratitude, apologies, and opinions.

[0246] In the embodiments of the present application, vehicle information is located according to natural language descriptions, achieving the effect of information connection (vehicle license plates, VIN numbers, etc. can be used as IDs).

[0247] In the embodiments of the present application, interactive information is processed according to state information (people, vehicles, and environment) to achieve effective and friendly communication, effectively transmitting information while not transmitting negative emotions.

[0248] In the embodiments of the present application, point-to-point interaction between vehicles can be achieved, and broadcast-style interaction can also be performed.

[0249] In the embodiments of the present application, other hardware of the vehicle, such as vehicle lights and projection, can be used to transmit paralinguistic information.

[0250] In the embodiments of the present application, projection includes DLP projection to the ground for pedestrian interaction and projection to the vehicle window for surrounding interaction.

[0251] In the embodiments of the present application, vehicle-to-vehicle interaction can include mobile network interaction and near-field interaction (e.g., through star flashing).

[0252] The embodiment of the present application further provides an interaction device, which comprises a module or unit for executing the above interaction method.

[0253] FIG. 14 shows a schematic flowchart of an interaction method 1400 provided by the embodiment of the present application. The method 1400 comprises:

[0254] S1410, the first vehicle acquires the first input of the user in the cabin of the first vehicle and the environmental information around the first vehicle, wherein the environmental information comprises information of the second vehicle.

[0255] For example, the first input can comprise voice input of the user.

[0256] For example, taking the first vehicle as the vehicle 100 in FIG. 3 and the second vehicle as the vehicle 200 in FIG. 3, the first input can be the voice input of the driver A, i.e. “Can the car opposite turn off the high beam?”.

[0257] For example, taking the first vehicle as the vehicle 100 in FIG. 4 and the second vehicle as the vehicle 200 in FIG. 4, the first input can be the voice input of the driver A, i.e. “Help me talk to the car in front, don't take the fast lane if you drive so slowly, it hinders others!”.

[0258] For example, taking the first vehicle as the vehicle 100 in FIG. 5 and the second vehicle as the vehicle 200 in FIG. 5, the first input can be the voice input of the driver A, i.e. “What's wrong with this person, how does he drive the car?”.

[0259] For example, the first input can comprise input of one or more components in the vehicle by the user. For example, the one or more components comprise, but are not limited to, a steering wheel, a light, a horn, an accelerator pedal or a brake pedal.

[0260] For example, taking the first vehicle as the vehicle 100 in FIG. 2A and the second vehicle as the vehicle 200 in FIG. 2A, the first input can be the horn input of the driver A.

[0261] For example, taking the first vehicle as the vehicle 200 in FIG. 6B and the second vehicle as the vehicle 100 in FIG. 6B, the first input can be the horn input of the driver in the vehicle 200.

[0262] For example, the first input can comprise physiological feature information of the user. For example, the physiological feature information can comprise blood pressure, heart rate, facial expression, etc. of the user.

[0263] For example, taking the facial expression of the user as an example, the vehicle can acquire image data collected by a camera in the cabin and determine the facial expression of the user according to the image data. For example, the facial expression of the user is a frowning expression.

[0264] For example, the first input can include a user's limb input. For example, the limb input can be a gesture input.

[0265] For example, the first input can be data collected by a wearable device (e.g., a smart watch, a smart bracelet, etc.) of a user in the cabin.

[0266] For example, the first input can be input of one or more users in the cabin.

[0267] Optionally, the environmental information includes data collected by a sensor outside the cabin.

[0268] For example, the sensor outside the cabin includes one or more of a camera, a laser radar, a millimeter wave radar, and a microphone.

[0269] S1420, the first vehicle determines the first output according to the first input and the environmental information.

[0270] Optionally, before the first vehicle determines the first output according to the first input and the environmental information, the method 1400 further includes determining that the first input is associated with a negative emotion of the user.

[0271] Optionally, determining that the first input is associated with a negative emotion of the user includes determining that the first input satisfies a preset condition.

[0272] For example, taking the first input as a voice input as an example, the preset condition includes that a text content corresponding to the voice input includes a target semantic, and the target semantic includes a semantic related to a negative emotion such as a complaint, anger, impolite language, and a mood adverb. For example, the target semantic related to a complaint includes "how to drive like this", "obstruct others", etc. For example, the semantic related to a mood adverb includes "ah".

[0273] For example, taking the first input as input of the user to the horn as an example, the preset condition includes that a duration for the user to press the horn is greater than or equal to a preset duration.

[0274] For example, taking the first input as input of the user to the steering wheel as an example, the preset condition includes that a gripping force of the user to the steering wheel is greater than or equal to a preset gripping force; or, the preset condition includes that a change rate of a steering angle of the steering wheel is greater than or equal to a preset change rate within a preset duration.

[0275] For example, taking the first input as a facial expression of the user as an example, the preset condition includes that the facial expression of the user is a frowning, angry, or angry expression.

[0276] Exemplarily, the first input is a heart rate of the user, and the preset condition includes that the heart rate of the user is greater than or equal to a preset heart rate.

[0277] Optionally, the first input is associated with negative emotion of the user, and the first output includes a de-emotional output.

[0278] In the embodiment of the present application, when the input of the user is associated with negative emotion of the user, the information that needs to be interacted with the second vehicle can be processed again based on the input of the user and the environmental information, so as to realize de-emotional processing. In this way, the purpose of effective and friendly communication between vehicles can be achieved, and the purpose of transmitting effective information without transmitting negative emotion can be achieved, which helps to avoid conflicts between drivers.

[0279] Optionally, the first output includes impolite language, and the first output includes polite language converted from the impolite language.

[0280] Based on the above technical solution, while performing de-emotional processing, the vehicle can also convert the impolite language of the user, so as to send the converted polite language to the second vehicle. In this way, the purpose of effective and friendly communication between vehicles can be further achieved, the purpose of transmitting effective information without transmitting negative emotion can be achieved, and the probability of conflicts between drivers can be reduced.

[0281] Optionally, the first input is associated with negative emotion of the user, and the method 1400 further includes: controlling a prompt device to output a third output, the third output including an output result for soothing the user in the cabin.

[0282] Based on the above technical solution, when the first input is associated with negative emotion of the user, the output result for soothing the user in the cabin can be output based on the first input and the environmental information. In this way, while realizing friendly interaction between vehicles, the user in the cabin can also be soothed, which helps to relieve negative emotion of the user and improve the driving experience of the user.

[0283] Exemplarily, the third output can include soothing language.

[0284] Exemplarily, the first input can be a voice input of the driver A as shown in FIG. 4, and the third output can be a voice output 1 "good, it has been expressed to the other party" and a voice output 2 "the driver in front may be a novice driver".

[0285] Exemplarily, the third output can include an execution instruction for one or more components in the cabin.

[0286] Optionally, the first output is determined according to the first input and the environment information, including: inputting the first input and the environment information into a first inference model to obtain the first output.

[0287] Based on the above technical solution, the first input and the environment information can be input into the first inference model, so that the first output can be obtained. In this way, the end-to-end input and output can be realized through the first inference model.

[0288] For example, the first inference model can be a multi-modal model.

[0289] For example, in the scenario shown in FIG. 3, the voice input "Can the car opposite turn off the high beam?" of the driver A and the image data (for example, a picture or a video stream) collected by the camera outside the cabin of the vehicle 100 can be input into the first inference model. The first inference model can output the real intention of the user "Please switch the high beam to the low beam".

[0290] For example, in the scenario shown in FIG. 4, the voice input "Help me talk to the car in front of me. Don't block others by driving so slowly in the fast lane." of the driver A and the image data (for example, a picture or a video stream) collected by the camera outside the cabin of the vehicle 100 can be input into the first inference model. The first inference model can output the real intention of the user "Please speed up in the fast lane".

[0291] For example, in the scenario shown in FIG. 5, the voice input "What's wrong with this person, how does he drive the car?" of the driver A and the image data (for example, a picture or a video stream) collected by the camera outside the cabin of the vehicle 100 can be input into the first inference model. The first inference model can output the real intention of the user "Please do not drive on the lane line".

[0292] Optionally, the method 1400 further includes: obtaining information of the first vehicle; wherein the first vehicle determines the first output according to the first input and the environment information, including: the first vehicle determines the first output according to the first input, the information of the first vehicle and the environment information.

[0293] Based on the above technical solution, through the input of the user in the vehicle cabin, the information of the vehicle and the environment information around the vehicle, the first output can be determined, so that the first output can be sent to the second vehicle or the terminal device. In this way, the information isolation between the drivers of the two vehicles can be broken, and the interaction between the vehicles can be realized. At the same time, by combining the information of the vehicle, the accuracy of the first output result can be further improved.

[0294] For example, the information of the vehicle includes historical driving records of the vehicle.

[0295] Exemplarily, the information of the vehicle includes one or more of a speed, an acceleration, and a position of the vehicle.

[0296] Exemplarily, taking the scenario shown in FIG. 3 as an example, the voice input of the driver A “Can the car opposite turn off the high beam?” can be input into the second inference model together with the light state of the vehicle 100 (the current vehicle 100 turns on the low beam, or the driver frequently switches the low beam and the high beam is detected) and the image data collected by the camera outside the cabin of the vehicle 100. The second inference model can output the real intention of the user “Please switch the high beam to the low beam”.

[0297] Exemplarily, taking the scenario shown in FIG. 4 as an example, the voice input of the driver A “Help me talk to the car in front, don't block others by driving so slowly in the fast lane” can be input into the second inference model together with the speed of the vehicle 100 (for example, 90 km / h) and the image data collected by the camera outside the cabin of the vehicle 100. The second inference model can output the real intention of the user “Please speed up in the fast lane”.

[0298] Optionally, the first vehicle determines the first output according to the first input, the information of the first vehicle, and the environmental information, including: inputting the first input, the information of the first vehicle, and the environmental information into the second inference model to obtain the first output.

[0299] Based on the above technical solutions, the first input, the information of the first vehicle, and the environmental information can be input into the first inference model, so that the first output can be obtained. In this way, the end-to-end input and output can be realized through the second inference model.

[0300] Optionally, the first inference model and the second inference model can be the same model.

[0301] Taking the first inference model and the second inference model as the same multi-modal model as an example. The input of the multi-modal model includes but is not limited to one or more of voice, picture, text content, or video stream.

[0302] The multi-modal large model can be obtained by training a general multi-modal large model. For example, the multi-modal large model includes a knowledge base. The knowledge base includes but is not limited to one or more of road regulations of a country (or a region), speed limit information of a road section, and reasonable handling methods when vehicles interact. In this way, the general multi-modal large model can become a special multi-modal large model in the field of vehicle interaction. The output of the multi-modal large model can be one or more of the real intention of the user after de-emotion, the polite language after conversion, and the translation result of the real intention of the user.

[0303] Optionally, the method 1400 further includes: the first vehicle determines the information of the second vehicle according to the first input and the environmental information.

[0304] Based on the above technical solution, through the input of the user in the first vehicle cabin and the environmental information, the information of the second vehicle can be determined, so that the first output can be sent to the second vehicle or the terminal device. In this way, the user's input and environmental information can be combined when determining the information of the second vehicle, ensuring the accuracy of the determined vehicle to be interacted.

[0305] Optionally, the first vehicle determines the information of the second vehicle according to the first input and the environmental information, comprising: the first vehicle inputs the first input and the environmental information into the third inference model to obtain the information of the second vehicle.

[0306] For example, in the scenario shown in FIG. 3, the voice input of the driver A "Can the car opposite turn off the high beam?", the light state of the vehicle 100 (the current vehicle 100 turns on the low beam, or it is detected that the driver frequently switches the low beam and the high beam) and the image data collected by the camera outside the cabin of the vehicle 100 are input into the second inference model. The second inference model can output the real intention of the user "Please switch the high beam to the low beam" and the license plate information of the vehicle 200.

[0307] For example, in the scenario shown in FIG. 4, the voice input of the driver A "Help me talk to the car in front, don't take the fast lane if you drive so slow, it's obstructing others", the speed of the vehicle 100 (for example, 90km / h) and the image data collected by the camera outside the cabin of the vehicle 100 are input into the second inference model. The second inference model can output the real intention of the user "Please speed up in the fast lane" and the license plate information of the vehicle 200.

[0308] Optionally, the first inference model, the second inference model and the third inference model can be the same model.

[0309] Taking the first inference model, the second inference model and the third inference model as the same multi-modal model as an example. The output of the multi-modal large model can also include the information of the second vehicle. For example, the information of the second vehicle includes the license plate information of the second vehicle.

[0310] Optionally, the first inference model and the third inference model can not be the same model.

[0311] For example, the third inference model can be implemented by a semantic recognition algorithm and an image segmentation algorithm. For example, when the first vehicle obtains the voice input of the user "Can the red car in front drive faster?", the first vehicle can obtain image 1 captured by the camera outside the cabin. Through the semantic recognition algorithm, the text content (for example, "red car") related to the attribute of the target in the voice input can be determined. The first vehicle can extract ROI1 and ROI2 from image 1 based on the image segmentation algorithm, and ROI1 and ROI2 respectively include a red vehicle. For example, image 1 can be divided into multiple regions, ROI1 can be region a in the multiple regions, and ROI2 can be region b in the multiple regions. If the line-of-sight direction of the user points to region a when the user triggers the voice input, ROI1 can be selected as the target ROI. The first vehicle can send the target ROI to the cloud server. Thus, the cloud server can analyze the license plate information of the second vehicle based on the target ROI.

[0312] Optionally, the method 1400 further includes: determining, by the first vehicle, the first control instruction according to the first input and the environment information, the first control instruction being associated with one or more actuators; and controlling, by the first vehicle, the one or more actuators to execute the first control instruction.

[0313] For example, taking the first vehicle as vehicle 100 in FIG. 2A and the second vehicle as vehicle 200 in FIG. 2A as an example, the first input can be the horn input of driver A. The first vehicle can generate control instructions for the sound device and the atmosphere lamp according to the horn input and image data captured by the sensor outside the cabin. For example, the control instruction for the sound device is used to instruct the sound device to play music (for example, soothing music) to relieve dissatisfaction. For example, the control instruction for the atmosphere lamp includes controlling the color of the atmosphere lamp to be warm.

[0314] Based on the above technical solutions, the control instruction can also be obtained based on the first input and the environment information. In this way, by executing the control instruction for the one or more actuators, the user's angry, angry, or impatient emotions can be relieved.

[0315] S1430, the first vehicle sends the first output to the second vehicle or the terminal device, the terminal device being associated with the second vehicle.

[0316] Optionally, the information of the second vehicle includes information of a license plate of the second vehicle; and the first vehicle sending the first output to the second vehicle or the terminal device includes: the first vehicle sending the first output and the information of the license plate of the second vehicle to the cloud server, so that the cloud server sends the first output to the second vehicle or the terminal device based on the information of the license plate of the second vehicle.

[0317] Based on the above technical solution, the vehicle can send the first output and the information of the license plate of the second vehicle to the cloud server, so that the cloud server can send the first output to the second vehicle through the information of the license plate of the second vehicle. Through the cloud server forwarding the information, the information isolation between the drivers in the two vehicles can be broken, and the interaction between the vehicles can be realized.

[0318] Optionally, the cloud server stores an association relationship between the license plate of the vehicle and the identification information of the terminal device (for example, a mobile phone or a vehicle).

[0319] In the embodiment of the application, the cloud server can store a corresponding relationship between each account in a plurality of accounts, an identity document (ID) or address information of one or more devices corresponding to each account, and license plate information.

[0320] For example, Table 1 shows the corresponding relationship between each account, the address information of one or more devices corresponding to each account, and the license plate information.

[0321] Table 1

[0322] For example, user 1 can input the information of the license plate number xxx associated with vehicle B in vehicle B logged in account 1, so that vehicle B can send the binding relationship between account 1 and the license plate number xxx to the cloud server.

[0323] For example, user 2 can input the information of the license plate number yyy associated with vehicle C in mobile phone C logged in account 2, so that mobile phone C can send the binding relationship between account 2 and the license plate number yyy to the cloud server.

[0324] For example, the output result of the above multi-modal large model can include the information of the license plate number xxx. After receiving the first output sent by the first vehicle and the information of the license plate number xxx, the cloud server can determine to send the first output to vehicle B according to the association relationship shown in Table 1.

[0325] For example, the output result of the above multi-modal large model can include the information of the license plate number yyy. After receiving the first output sent by the first vehicle and the information of the license plate number yyy, the cloud server can determine to send the first output to mobile phone C according to the association relationship shown in Table 1. Mobile phone C can determine whether mobile phone C is located in the cabin of vehicle C after receiving the first output. If mobile phone C is located in the cabin of vehicle C, mobile phone C can prompt the user based on the first output.

[0326] Optionally, the first vehicle sending the first output to the second vehicle or the terminal device comprises: the first vehicle sending the first output to the second vehicle through a near field communication technology.

[0327] For example, in the scenario shown in FIG. 3, when the environmental information indicates that there is a vehicle 200 around the vehicle 100 and the current light state of the vehicle 200 indicates that the high beam of the vehicle 200 is turned on, the vehicle 100 can send the first output to the vehicle 200 through the star flash technology.

[0328] S1440, the second vehicle determines the second output according to the first output and the environmental information.

[0329] For example, in the scenario shown in FIG. 3, the first inference model can output the real intention of the user "please switch the high beam to the low beam". The vehicle 200 can input the real intention of the user in the vehicle 100 and the image data collected by the camera outside the cabin of the vehicle 200 into the fourth inference model, so as to obtain the text content "Dear driver, you are now driving with high beam, which may affect the driver on the opposite side, the driver on the opposite side hopes you to switch to low beam".

[0330] For example, in the scenario shown in FIG. 4, the first inference model can output the real intention of the user "please speed up in the fast lane". The vehicle 200 can input the real intention of the user in the vehicle 100 and the image data collected by the camera outside the cabin into the fourth inference model, so as to obtain the text content "Dear driver, Xia A reminds you that you are now in the fast lane, the speed limit is 80-100km / h, and the current driving speed of the vehicle is 65km / h, the driver behind is a little anxious, please speed up".

[0331] For example, in the scenario shown in FIG. 7, the first vehicle can be the vehicle 100 and the second vehicle can be the vehicle 200. The first input can be the horn input of the driver in the vehicle 100. The vehicle 100 can input the horn input and the data collected by the sensor outside the cabin of the vehicle 100 into the first inference model, so as to obtain the intention of the user "please do not add to the queue". Since the sensor outside the cabin of the vehicle 100 does not detect the cone barrel in front of the vehicle 200, the vehicle 100 will mistakenly think that the intention of the user conforms to the road regulations, and can send the intention of the user to the vehicle 200. After receiving the intention of the user, the vehicle 200 can input the intention of the user and the data collected by the sensor outside the cabin (including the data related to the road occupation construction) into the fourth inference model, so as to obtain the text content "the driver who honks may not know the situation of the road occupation construction, I will convey it to him".

[0332] Optionally, before the second output is determined according to the first output and the environment information, the method 1400 further includes: the second vehicle controlling the prompting device to prompt the user to receive the first output from the first vehicle and to prompt the user whether to make the prompt; and wherein the second vehicle determines the second output according to the first output and the environment information, including: in response to detecting the input of the user determining to make the prompt, the second vehicle determines the second output according to the first output and the environment information.

[0333] Optionally, the second vehicle determines the second output according to the first output and the environment information, including: the second vehicle determines the second output according to the data collected by the sensor in the vehicle cabin, the first output and the environment information.

[0334] Optionally, the second vehicle can output the first output and the environment information into the fourth inference model, so as to obtain the second output.

[0335] Optionally, the fourth inference model can be a multi-modal large model. The multi-modal large model can be obtained by training a general multi-modal large model. For example, the multi-modal large model includes a knowledge base. The knowledge base includes but is not limited to one or more national (or regional) road regulations, speed limit information of road sections, and reasonable handling methods when vehicles interact. In this way, the general multi-modal large model can become a specialized multi-modal large model in the field of vehicle interaction.

[0336] Optionally, taking the scenario shown in FIG. 3 as an example, the first vehicle can be vehicle 100 and the second vehicle can be vehicle 200. The first inference model can output the real intention of the user "please switch the high beam to the low beam". Vehicle 200 can input the real intention of the user in vehicle 100, the data collected by the sensor in the cabin (indicating that the driver in vehicle 200 is in a tense state) and the image data collected by the camera outside the cabin into the fourth inference model, so as to obtain the text content "Dear car owner, you are currently driving with high beam, which may affect the driver on the opposite side. The car owner on the opposite side hopes you to switch to low beam. You can pull the lever to the right of the steering wheel upwards to switch to low beam".

[0337] Optionally, taking the scenario shown in FIG. 4 as an example, the first vehicle can be vehicle 100 and the second vehicle can be vehicle 200. The first inference model can output the real intention of the user "please speed up in the fast lane". Vehicle 200 can input the real intention of the user in vehicle 100, the data collected by the sensor in the cabin (indicating that the driver in vehicle 200 is in a happy state) and the image data collected by the camera outside the cabin into the fourth inference model, so as to obtain the text content "Dear car owner, little A reminds you that the driver behind is a bit anxious, please speed up".

[0338] Based on the above technical scheme, the data collected by the sensors in the vehicle cabin can be combined when determining the second output. In this way, the second output can be more easily accepted by the user in the current cabin, thereby helping to improve the user's driving experience.

[0339] Optionally, the second vehicle determines the second output according to the first output and the environmental information, including: the second vehicle determines the second output according to the first information of the driver in the second vehicle cabin, the first output and the environmental information, and the first information includes one or more of driving proficiency, driving habit or physiological characteristic information.

[0340] For example, in the scenario shown in FIG. 4, the first vehicle can be vehicle 100 and the second vehicle can be vehicle 200. The first inference model can output the real intention of the user "please speed up in the fast lane". Vehicle 200 can input the real intention of the user in vehicle 100 and the driving proficiency of the driver in vehicle 200 (indicating that the driver in the second vehicle is a novice driver) into the fourth inference model, so as to obtain the text content "Dear car owner, A reminds you that you are now in the fast lane, the speed limit is 80-100km / h, the current vehicle speed is 65km / h, the driver behind is a bit anxious, please speed up".

[0341] For example, in the scenario shown in FIG. 4, vehicle 200 can input the real intention of the user in vehicle 100 and the driving proficiency of the driver in vehicle 200 (indicating that the driver in the second vehicle is a driver with high driving proficiency) into the fourth inference model, so as to obtain the text content "Dear car owner, the driver behind is a bit anxious, please speed up".

[0342] For example, in the scenario shown in FIG. 4, vehicle 200 can input the real intention of the user in vehicle 100 and the driving habit of the driver in vehicle 200 (the frequency of driving the vehicle in the slow lane in the past period of time is greater than or equal to the preset frequency) into the fourth inference model, so as to obtain the text content "Dear car owner, the driver behind is a bit anxious, you can switch to the slow lane".

[0343] For example, in the scenario shown in FIG. 4, vehicle 200 can input the real intention of the user in vehicle 100 and the physiological characteristic information of the driver in vehicle 200 (indicating that the driver is an old driver) into the fourth inference model, so as to obtain the text content "Dear car owner, A reminds you that you are now in the fast lane, the speed limit is 80-100km / h, the current vehicle speed is 65km / h, the driver behind is a bit anxious, if you are not in a hurry, you can turn on the turn signal to the right and switch to the slow lane".

[0344] Based on the above technical solution, the second output can be determined in combination with the information of the driver. In this way, the second output can be more in line with the image of the driver, so that different images of different drivers obtain different output results, which helps to improve the intelligent degree of the vehicle and also helps to improve the driving experience of the user.

[0345] Optionally, the second vehicle determines the second output according to the first output and the environment information, including: the second vehicle determines the second output according to the historical driving records of other vehicles around the second vehicle, the first output and the environment information.

[0346] For example, in the scenario shown in FIG. 7, the first output can be the user's intention "please don't add a queue". Since the sensor outside the cabin of the vehicle 100 does not detect the cone barrel in front of the vehicle 200, it is mistakenly believed that the user's intention complies with the road regulations, so that the user's intention can be sent to the vehicle 200. After receiving the user's intention, the vehicle 200 can input the user's intention, the historical driving records of other vehicles around the vehicle 200 (the historical driving records indicate that the surrounding other vehicles have alternately passed through the road section in the past period of time) and the data collected by the sensor outside the cabin (including the data related to the road occupation construction) into the fourth inference model, so as to obtain the text content "the driver who honks the horn may not know the situation of the road occupation construction, I will convey it to him".

[0347] Based on the above technical solution, the second output can be determined in combination with the historical driving records of other vehicles around the vehicle. In this way, the second output can be more accurate, and the second output can be more easily accepted by the user in the cabin.

[0348] Optionally, the second vehicle determines the second output according to the first output and the environment information, including: the second vehicle inputs the first output and the environment information into the inference model to obtain the second output.

[0349] S1450, the second vehicle controls the prompt device to output the second output.

[0350] For example, after obtaining the text content of the fourth inference model, the second vehicle can control the sound producing device in the cabin of the second vehicle to play the corresponding voice information.

[0351] FIG. 15 shows a schematic diagram of the architecture of the system 1500 provided by the embodiments of the present application. The system 1500 includes a vehicle 1510 and a vehicle 1520. The vehicle 1510 can include a first inference model, and the vehicle 1520 can include a fourth inference model.

[0352] The vehicle 1510 can be the vehicle 100, and the vehicle 1520 can be the vehicle 200. Alternatively, the vehicle 1510 can be the first vehicle, and the vehicle 1520 can be the second vehicle.

[0353] FIG. 16 shows a schematic block diagram of an interaction device 1600 according to an embodiment of the present application. The device 1600 includes an obtaining unit 1610 configured to obtain a first input of a user in a vehicle cabin and environment information of a vehicle surrounding, the environment information including information of another vehicle; a determining unit 1620 configured to determine a first output according to the first input and the environment information; and a sending unit 1630 configured to send the first output to the another vehicle or a terminal device associated with the another vehicle.

[0354] Optionally, the first input is associated with negative emotion of the user, and the first output includes an output after de-emotion.

[0355] Optionally, the determining unit 1620 is configured to input the first input and the environment information into a first inference model to obtain the first output.

[0356] Optionally, the obtaining unit 1610 is further configured to obtain information of the vehicle, and the determining unit is configured to determine the first output according to the first input, the information of the vehicle, and the environment information.

[0357] Optionally, the determining unit 1620 is configured to input the first input, the information of the vehicle, and the environment information into a second inference model to obtain the first output.

[0358] Optionally, the determining unit 1620 is further configured to determine the information of the another vehicle according to the first input and the environment information.

[0359] Optionally, the determining unit 1620 is configured to input the first input and the environment information into a third inference model to obtain the information of the another vehicle.

[0360] Optionally, the information of the another vehicle includes information of a license plate of the another vehicle, and the sending unit 1630 is configured to send the first output and the information of the license plate of the another vehicle to a cloud server, so that the cloud server sends the first output to the another vehicle or the terminal device based on the information of the license plate of the another vehicle.

[0361] Optionally, the device further includes a control unit, and the determining unit is further configured to determine a first control instruction associated with one or more actuators according to the first input and the environment information, and the control unit is configured to control the one or more actuators to execute the first control instruction.

[0362] FIG. 17 shows a schematic block diagram of an interaction device 1700 according to an embodiment of the present application. The device 1700 comprises: an acquisition unit 1710 configured to acquire a first output of another vehicle and environmental information around the vehicle; a determination unit 1720 configured to determine a second output according to the first output and the environmental information; and a control unit 1730 configured to control a prompting device to output the second output.

[0363] Optionally, the determination unit 1720 is configured to determine the second output according to data collected by a sensor in a cabin of the vehicle, the first output and the environmental information.

[0364] Optionally, the determination unit 1720 is configured to determine the second output according to first information of a driver in the vehicle, the first output and the environmental information, the first information comprising one or more of driving proficiency, driving habit or physiological characteristic information.

[0365] Optionally, the determination unit 1720 is configured to determine the second output according to historical driving records of other vehicles around the vehicle, the first output and the environmental information.

[0366] Optionally, the determination unit 1720 is configured to input the first output and the environmental information into an inference model to obtain the second output.

[0367] It should be understood that the division of units in the above device is only a logical division of functions, and all or part of the units can be integrated into one physical entity, or can be physically separated. In addition, the units in the device can be implemented in the form of processor calling software; for example, the device includes a processor, the processor is connected with a memory, the memory stores instructions, and the processor calls the instructions stored in the memory to implement any one of the above methods or to realize the functions of the units of the device, wherein the processor is, for example, a general processor such as a CPU or a microprocessor, and the memory is an internal memory of the device or an external memory of the device. Alternatively, the units in the device can be implemented in the form of hardware circuit, and the functions of part or all of the units can be realized by designing the hardware circuit, which can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of part or all of the units are realized by designing the logical relationship of elements in the circuit; for example, in another implementation, the hardware circuit is a PLD, and the functions of part or all of the units are realized by configuring the connection relationship between the logical gate circuits through a configuration file, for example, the FPGA can include a large number of logical gate circuits, and the functions of part or all of the units are realized by configuring the connection relationship between the logical gate circuits through a configuration file. All units of the above device can be implemented in the form of processor calling software, or all units can be implemented in the form of hardware circuit, or part of the units can be implemented in the form of processor calling software, and the remaining part can be implemented in the form of hardware circuit.

[0368] In the embodiments of the present application, the processor is a circuit with signal processing capability. In one implementation, the processor can be a circuit with instruction reading and running capability, such as a CPU, a microprocessor, a GPU, or a DSP, etc. In another implementation, the processor can implement certain functions through a logic relationship of a hardware circuit, which is fixed or can be reconfigured. For example, the processor is a hardware circuit implemented by an ASIC or a PLD, such as an FPGA. In the reconfigurable hardware circuit, the processor loads a configuration document to implement the hardware circuit configuration. It can be understood that the processor loads instructions to implement the functions of the above units.

[0369] It can be seen that each unit in the above apparatus can be one or more processors (or processing circuits) configured to implement the above methods, such as a CPU, a GPU, a NPU, a TPU, a DPU, a microprocessor, a DSP, an ASIC, an FPGA, or a combination of at least two of these processor forms.

[0370] In addition, each unit in the above apparatus can be integrated together or can be independently implemented. In one implementation, these units are integrated together to implement a SoC. The SoC can include at least one processor for implementing any of the above methods or the functions of the units of the apparatus. The at least one processor can be different, such as including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0371] The embodiments of the present application also provide an interaction apparatus, which includes a processing unit and a storage unit, wherein the storage unit is configured to store instructions, and the processing unit is configured to execute the instructions stored in the storage unit, so that the apparatus executes the method or steps executed by the above embodiments.

[0372] Optionally, if the interaction apparatus is located in a vehicle, the above processing unit can be one or more of the processors 121-12n shown in FIG. 1.

[0373] The embodiments of the present application also provide an interaction system, which includes the above interaction apparatus and a perception system.

[0374] The embodiments of the present application also provide a vehicle, which can include the above interaction apparatus or the above interaction system.

[0375] The embodiments of the present application also provide a computer program product, which includes computer program code. When the computer program code runs on a computer, it makes the computer execute the method in the above embodiments.

[0376] The embodiment of the present application further provides a computer readable medium, which stores program codes, and when the program codes are run on a computer, the computer executes the method in the above embodiment.

[0377] The embodiment of the present application further provides a chip, which comprises a circuit for executing the method in the above embodiment.

[0378] In the implementation process, each step of the above method can be completed by integrated logic circuits of hardware in a processor or instructions in the form of software. The method disclosed in the embodiment of the present application can be directly embodied as hardware processor execution completion or combined execution completion by hardware and software modules in the processor. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register or other mature storage media in the art. The storage medium is located in the memory, and the processor reads information in the memory and combines the hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0379] It should be understood that the memory in the embodiment of the present application can include a read-only memory and a random access memory, and provide instructions and data to the processor.

[0380] It should also be understood that in various embodiments of the present application, the size of the serial number of each process described above does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0381] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0382] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be described here.

[0383] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented in other manners. For example, the described device embodiments are merely schematic. The division of the units is merely logical function division. There can be other division manners in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0384] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0385] In addition, each functional unit in the various embodiments of the present application can be integrated into a processing unit, or each unit can be a physically separate unit, or two or more units can be integrated into one unit.

[0386] If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program code storage media.

[0387] The above is merely specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An interaction method, characterized in that, The method comprises: obtaining a first input of a user in a vehicle cabin and environmental information around the vehicle, the environmental information comprising information of another vehicle; determining a first output according to the first input and the environmental information; sending the first output to the another vehicle or a terminal device associated with the another vehicle.

2. The method of claim 1, wherein, The first input is associated with negative emotion of the user, and the first output comprises a de-emotional output.

3. The method according to claim 1 or 2, characterized in that, The determining of the first output according to the first input and the environmental information comprises: inputting the first input and the environmental information into a first inference model to obtain the first output.

4. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: obtaining information of the vehicle; The determining of the first output according to the first input and the environmental information comprises: determining the first output according to the first input, the information of the vehicle and the environmental information.

5. The method of claim 4, wherein, The determining of the first output according to the first input, the information of the vehicle and the environmental information comprises: inputting the first input, the information of the vehicle and the environmental information into a second inference model to obtain the first output.

6. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: determining information of the another vehicle according to the first input and the environmental information.

7. The method of claim 6, wherein, The determining of the information of the another vehicle according to the first input and the environmental information comprises: inputting the first input and the environmental information into a third inference model to obtain the information of the another vehicle.

8. The method according to claim 6 or 7, characterized in that, The information of the another vehicle comprises information of a license plate of the another vehicle. The sending of the first output to the another vehicle or the terminal device comprises: sending the first output and the information of the license plate of the another vehicle to a cloud server, so that the cloud server sends the first output to the another vehicle or the terminal device based on the information of the license plate of the another vehicle.

9. The method according to any one of claims 1 to 8, characterized in that, The method further comprises: determining a first control instruction associated with one or more actuators according to the first input and the environmental information; controlling the one or more actuators to execute the first control instruction.

10. An interaction method, characterized in that, The method comprises: obtaining a first output of another vehicle and environmental information around the vehicle; determining a second output according to the first output and the environmental information; controlling a prompt device to output the second output.

11. The method of claim 10, wherein, The determining of the second output according to the first output and the environmental information comprises: determining the second output according to data collected by a sensor in a vehicle cabin, the first output and the environmental information.

12. The method according to claim 10 or 11, characterized in that, The determining of the second output according to the first output and the environmental information comprises: determining the second output according to first information of a driver in the vehicle, the first output and the environmental information, the first information comprising one or more of driving proficiency, driving habit or physiological characteristic information.

13. The method according to any one of claims 10 to 12, characterized in that, The determining of the second output according to the first output and the environmental information comprises: determining the second output according to historical driving records of other vehicles around the vehicle, the first output and the environmental information.

14. The method according to any one of claims 10 to 13, characterized in that, The determining the second output according to the first output and the environment information comprises: inputting the first output and the environment information into an inference model to obtain the second output.

15. An interactive device, characterized by comprise: an acquisition unit, configured to acquire a first input of a user in a vehicle cabin and environment information around the vehicle, the environment information comprising information of another vehicle; a determination unit, configured to determine a first output according to the first input and the environment information; a sending unit, configured to send the first output to the another vehicle or a terminal device associated with the another vehicle.

16. The apparatus of claim 15, wherein, The first input is associated with negative emotion of the user, and the first output comprises an output after de-emotion.

17. The apparatus of claim 15 or 16, wherein, The determination unit is configured to: input the first input and the environment information into a first inference model to obtain the first output.

18. The apparatus according to any one of claims 15-17, wherein the acquisition unit is further configured to acquire information of the vehicle; the determination unit is configured to determine the first output according to the first input, the information of the vehicle and the environment information.

19. The apparatus of claim 18, wherein, The determination unit is configured to: input the first input, the information of the vehicle and the environment information into a second inference model to obtain the first output.

20. The apparatus according to any one of claims 15-19, wherein the determination unit is further configured to determine information of the another vehicle according to the first input and the environment information.

21. The apparatus of claim 20, wherein, The determination unit is configured to: input the first input and the environment information into a third inference model to obtain the information of the another vehicle.

22. The apparatus of claim 20 or 21, wherein, The information of the another vehicle comprises information of a license plate of the another vehicle; The sending unit is configured to send the first output and the information of the license plate of the another vehicle to a cloud server, so that the cloud server sends the first output to the another vehicle or the terminal device based on the information of the license plate of the another vehicle.

23. The apparatus of any one of claims 15-22, wherein, The apparatus further comprises a control unit, the determination unit is further configured to determine a first control instruction according to the first input and the environment information, the first control instruction being associated with one or more actuators; the control unit is configured to control the one or more actuators to execute the first control instruction.

24. An interactive device, characterized by comprise: an acquisition unit, configured to acquire a first output of another vehicle and environment information around the vehicle; a determination unit, configured to determine a second output according to the first output and the environment information; a control unit, configured to control a prompting apparatus to output the second output.

25. The apparatus of claim 24, wherein, The determination unit is configured to: determine the second output according to data collected by a sensor in a vehicle cabin, the first output and the environment information.

26. The apparatus of claim 24 or 25, wherein, The determination unit is configured to: determine the second output according to first information of a driver in the vehicle, the first output and the environment information, the first information comprising one or more of driving proficiency, driving habit or physiological characteristic information.

27. The apparatus of any one of claims 24-26, wherein, The determination unit is configured to: The second output is determined according to historical driving records of other vehicles around the vehicle, the first output, and the environmental information.

28. The apparatus of any one of claims 24-27, wherein, The determining unit is configured to: input the first output and the environmental information into an inference model to obtain the second output.

29. An interactive device, characterized by comprise: a memory configured to store a computer program; a processor configured to execute the computer program stored in the memory, so that the apparatus executes the method according to any one of claims 1 to 14.

30. An interactive system, characterized by The interaction system comprises a computing platform and a perception system, and the computing platform comprises the interaction apparatus according to claim 29.

31. A vehicle characterized by comprise the interaction apparatus according to any one of claims 15 to 29, or comprise the interaction system according to claim 30.

32. A computer-readable storage medium, comprising: instructions stored thereon, which, when executed by a processor, cause the processor to implement the method according to any one of claims 1 to 14.

33. A computer program product, characterised in that, The computer program product comprises computer program code which, when executed on a computer, causes the computer to implement the method according to any one of claims 1 to 14.

34. A chip, characterized by The chip comprises a circuit configured to execute the method according to any one of claims 1 to 14.

Images