Interaction method and device, computer equipment and storage medium

Abstract

The invention relates to an interaction method and device, computer equipment and a storage medium, and the method comprises the steps: determining a plurality of target intentions related to a user according to multi-mode information related to a target vehicle; a dialogue service flow is started, and the dialogue service flow comprises the steps that for each target intention in the multiple target intentions, an instruction corresponding to the target intention is sent to an application, to which an atomization service function used for achieving the target intention belongs, in the at least one application, so that the application calls the atomization service function used for achieving the target intention according to the instruction; and providing information for guiding user interaction for the user, wherein the information for guiding user interaction comprises a calling result corresponding to the atomization service function for realizing the target intention.

Description

Technical Field

[0001] The present invention relates to the field of vehicles, and in particular to an interaction method, device, computer equipment and storage medium. Background Art

[0002] With the rapid development of intelligent technology, in-vehicle voice interaction systems have become an important part of modern cars. In recent years, with the continuous breakthroughs in artificial intelligence, natural language processing and speech recognition technology, the performance of in-vehicle voice interaction systems has been significantly improved. These technologies enable vehicles to understand the driver's voice commands more accurately and perform corresponding operations through voice feedback, greatly improving the convenience and safety of driving. The development of in-vehicle voice technology not only simplifies traditional manual operations, but also enhances driving safety by reducing driver distraction.

[0003] At present, the in-vehicle voice interaction system is able to recognize and process multiple intentions in one sentence, that is, the system can recognize multiple needs or intentions expressed by the user at the same time in a sentence, and distribute these intentions to different in-vehicle applications for processing.

[0004] However, when processing multi-intention commands, the current in-vehicle voice interaction system simply distributes the commands to various applications for execution. The execution results of multiple commands are not fed back on the interface, resulting in users being unable to intuitively see the operation results corresponding to each command. Summary of the invention

[0005] One of the purposes of the present invention is to provide an interaction method, apparatus, computer device and storage medium to solve the problem of how to improve the convenience of interaction between a vehicle and a user in the vehicle.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0007] An interaction method comprises: determining multiple target intentions related to a user based on multimodal information related to a target vehicle; starting a dialogue service flow, the dialogue service flow comprising: for each target intention among the multiple target intentions, sending an instruction corresponding to the target intention to an application to which an atomic service function for realizing the target intention belongs in at least one application, so that the application calls the atomic service function for realizing the target intention according to the instruction; and providing information for guiding user interaction to a user, the information for guiding user interaction comprising: a call result corresponding to the atomic service function for realizing the target intention.

[0008] In one possible implementation, the method also includes: providing information to the user for guiding user interaction, including: displaying the call result corresponding to the atomic service function used to achieve the target intent in the dialogue service flow interface or playing a voice indicating the call result corresponding to the atomic service function used to achieve the target intent.

[0009] In one possible implementation, the information for guiding user interaction also includes: information indicating recommended atomic service functions corresponding to the atomic service functions for achieving the target intent, providing the user with information for guiding user interaction, and also including: displaying a user interface component of the recommended atomic service functions corresponding to the atomic service functions for achieving the target intent in the dialogue service flow interface or playing a voice indicating the recommended atomic service functions corresponding to the atomic service functions for achieving the target intent.

[0010] In one possible implementation, multiple target intentions related to the user are determined based on multimodal information related to the target vehicle, including: using a target model on a cloud server to determine multiple target intentions related to the user based on multimodal information related to the target vehicle.

[0011] In one possible implementation, the method further includes: after providing the user with information for guiding user interaction, in response to the user's operation on the corresponding information item in the information for guiding user interaction, calling an atomic service function corresponding to the corresponding information item.

[0012] An interactive device, the interactive device comprising: an intention determination unit, used to determine multiple target intentions related to a user based on multimodal information related to a target vehicle; a starting unit, used to start a dialogue service flow, the dialogue service flow comprising: for each target intention among the multiple target intentions, sending an instruction corresponding to the target intention to at least one application to which an atomic service function for realizing the target intention belongs, so that the application calls the atomic service function for realizing the target intention according to the instruction; a guiding unit, used to provide information for guiding user interaction to the user, the information for guiding user interaction comprising: a calling result corresponding to the atomic service function for realizing the target intention.

[0013] A computer device comprises: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the interaction method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0014] A vehicle comprises: a memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the interaction method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0015] A computer-readable storage medium stores computer instructions, wherein the computer instructions are used to enable a computer to execute the interactive method of the first aspect or any corresponding embodiment thereof.

[0016] A computer program product includes computer instructions, wherein the computer instructions are used to enable a computer to execute the interactive method of the first aspect or any corresponding embodiment thereof.

[0017] Beneficial effects of the present invention:

[0018] By processing multimodal data, multiple target intentions related to the user are determined based on multimodal information related to the target vehicle; the dialogue service flow is started, and the dialogue service flow includes: for each target intention among the multiple target intentions, the instruction corresponding to the target intention is sent to at least one application to which the atomic service function for realizing the target intention belongs, so that the target intention can be realized. The user does not need to perform a series of operations on the vehicle screen to trigger the operation of executing the intention, thereby simplifying the user's operation and improving the convenience of interaction between the vehicle and the user. At the same time, information for guiding user interaction is provided to the user, and the information for guiding user interaction includes: the call results corresponding to the atomic service function for realizing the target intention, so that the user can intuitively see each call result. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A schematic diagram of an example architecture of an interactive system that can be used to execute the interactive method provided in an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of a flow chart of an interactive method provided by an embodiment of the present invention;

[0022] Figure 3 A schematic diagram of an example for determining target intent;

[0023] Figure 4A schematic diagram of the effect of calling the corresponding atomic service function of the navigation APP on the target vehicle to realize the target intention;

[0024] Figure 5 A schematic diagram of the effect of calling the corresponding atomic service function of the music APP on the target vehicle to realize the target intention;

[0025] Figure 6 A schematic diagram of the effect of calling the corresponding atomic service function of the music APP on the target vehicle to realize the target intention;

[0026] Figure 7 A flowchart showing an example of interaction between a target vehicle and a user;

[0027] Figure 8 Schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0029] refer to Figure 1 , which shows a schematic diagram of an example architecture of an interactive system that can be used to execute the interactive method provided by an embodiment of the present invention.

[0030] An example architecture of an interactive system that can be used to execute the interactive method provided by the embodiment of the present invention includes: a data collection unit, a data processing unit, and an atomic service function unit.

[0031] The data acquisition unit is used to collect voice data of the user's interactive voice, the internal environment image of the target vehicle, the external environment image of the target vehicle, the sensor data of the target vehicle, etc. The data processing unit includes a voice data processing module, an image processing module, a sensor data processing module, an encoding module, and a target model. The atomic service function includes: the atomic service function of each APP with atomic service function on the target vehicle. The atomic service function can be understood as the smallest functional unit, and the atomic service function can perform a single operation.

[0032] The voice data processing module can perform voice recognition on the user's interactive voice to obtain a voice recognition result of the user's interactive voice. The voice data processing unit can use a sound source localization algorithm to determine the seat position of the user who speaks the interactive voice in the target vehicle. The voice data processing unit can determine the identity of the user by matching the voiceprint feature of the user's interactive voice with the preset voiceprint feature. The preset voiceprint feature is the voiceprint feature of the user who has recorded the voiceprint feature.

[0033] The image processing module can process the internal environment image of the target vehicle to determine the age group to which the user belongs, the user's gender, the user's sitting position, the user's identity information, etc. The image processing unit can include a driver monitoring system (DMS) and an occupant monitoring system (OMS). The driver monitoring system monitors the direction of the driver's line of sight in the target vehicle in real time. The occupant monitoring system can process the vehicle's internal environment image captured by the camera of the target vehicle to determine the gender, age, gesture, user's sitting position, etc. of the passenger in the target vehicle.

[0034] The sensor data processing module can process the sensor data collected by the sensor of the target vehicle to determine the information related to the driving of the target vehicle and the information related to the weather in the area where the target vehicle is located. The information related to the driving of the target vehicle may include: the speed of the target vehicle, the remaining fuel of the target vehicle or the remaining power of the battery of the target vehicle, etc. The information related to the weather in the area where the target vehicle is located may include: the temperature of the weather and the type of the weather. The type of weather may be raining, snowing, etc.

[0035] The encoding module is used to encode the multimodal information related to the target vehicle to obtain a representation of the multimodal information related to the target vehicle.

[0036] It should be noted that the target model can be on the target vehicle or on a cloud server.

[0037] The target model is used to receive the representation of multimodal information related to the target vehicle and predict the representation of the target intention. Thus, the target intention is determined. The target model is a trained neural network. Before the interactive method provided by the embodiment of the present invention is executed for the first time, the target model is trained using the training samples of the target model and the labeling operation of the training samples of the target model.

[0038] As an example, the target model is a pre-trained language model. As an example, the pre-trained language model used as the target model is a large language model (LLM). When the input of the target model is a natural language text corresponding to a natural language speech, the pre-trained language model performs a natural language processing (NLP) task.

[0039] Training the target model can enable the target model to learn: which information in the input of the target model to use to predict the operation that satisfies the user's interactive voice instruction intention, and how to determine the target intention. The training samples of the target model may include: information used for training on the type of information involved in the multimodal information related to the target vehicle. For a piece of information, if the multimodal information related to the target vehicle includes the information, then the type of the information is the type of information involved in the multimodal information related to the target vehicle. The labeled operation of the training sample of the target model can be understood as: when the training sample of the target model is input into the target model, the operation that the target model is expected to output. When training the target model, the loss between the operation predicted by the target model and the labeled operation of the training sample can be calculated, and the parameters of the target model can be updated based on the loss.

[0040] refer to Figure 2 , which shows a schematic diagram of an example of an interaction method provided by an embodiment of the present invention.

[0041] Step S201, determining multiple target intentions related to the user based on multimodal information related to the target vehicle.

[0042] The multimodal information related to the target vehicle includes: speech recognition results of the user's interactive speech, parameter information of the target vehicle, external environment information of the target vehicle, user's action information, user's riding position information, user's attribute information, user's identity information, information related to the APP on the target vehicle determined within a historical time period of using the APP on the target vehicle, navigation information related to the target vehicle, music information related to the target vehicle, and point of interest information related to the target vehicle, wherein the point of interest information related to the target vehicle indicates the points of interest reached by the target vehicle within a target historical time period and / or the points of interest reached by the target vehicle multiple times.

[0043] The speech recognition result of the user's interactive speech is obtained by performing speech recognition on the user's interactive speech. Performing speech recognition on the user's interactive speech specifically refers to performing speech recognition on the speech data of the user's interactive speech.

[0044] As an example, the parameter information of the target vehicle may include: information related to the driving of the target vehicle, and information related to the cabin of the target vehicle.

[0045] As an example, the information related to the driving of the target vehicle includes: the speed of the target vehicle, the remaining fuel of the target vehicle, or the remaining power of the battery of the target vehicle.

[0046] As an example, information related to the target vehicle's cabin includes: the brightness of the target vehicle's screen, the volume of the target vehicle's audio system, the temperature of the target vehicle's air conditioner, the air volume of the target vehicle's air conditioner, the angle of the target vehicle's seat, etc.

[0047] The external environment information of the target vehicle is information related to the external environment of the target vehicle.

[0048] As an example, the external environment information of the target vehicle includes: information related to the weather in the area where the target vehicle is located, information related to the driving safety of the road where the target vehicle is currently located, points of interest (POI) near the target vehicle, etc.

[0049] As an example, the information related to the weather in the area where the target vehicle is located includes the temperature of the weather, the type of the weather, and the like.

[0050] As an example, the information related to the driving safety of the vehicle on the road that the target vehicle is currently on includes one or more of the following: information indicating whether the road that the target vehicle is currently on is slippery, information indicating whether the road that the target vehicle is currently on is icy, information indicating whether the road that the target vehicle is currently on is bumpy, information indicating whether the road that the target vehicle is currently on is a plateau road, etc.

[0051] As an example, the user's action information includes at least some of the following items: the user's gesture, the direction in which the user's hand points, the direction in which the user's line of sight is directed, the user's action, and the like.

[0052] As an example, the attribute information of the user includes: the age group to which the user belongs, the gender of the user, etc.

[0053] The user's identity information is used to identify the user.

[0054] As an example, the user's identity information includes: the user's identifier.

[0055] The user's sitting position information indicates the user's sitting position. As an example, the user's sitting position is one of the following items: main driving position, co-pilot position, rear left position, rear middle position, rear right position.

[0056] The navigation information related to the target vehicle may include: a departure point of the current navigation of the target vehicle, a destination of the current navigation of the target vehicle, and a navigation route of the current navigation of the target vehicle.

[0057] The music information related to the target vehicle may include: a song playlist to which the identifier of the song currently played by the target vehicle belongs, and a play progress of the song to which the identifier belongs in the song playlist. The play progress indicates which song is being played.

[0058] The end time of the historical time period of using the APP on the target vehicle is before the current time. The APP on the target vehicle is used during the historical time period of using the APP on the target vehicle.

[0059] As an example, for a navigation APP on a target vehicle, the multimodal information related to the target vehicle includes: information related to the navigation APP on the target vehicle determined within a historical period of time when the navigation APP on the target vehicle was used. The information related to the navigation APP on the target vehicle determined within a historical period of time when the APP on the target vehicle was used includes: a navigation route determined within a historical period of time when the APP on the target vehicle was used, the name of the departure point of the navigation route determined within a historical period of time when the navigation APP on the target vehicle was used, the name of the destination of the navigation route determined within a historical period of time when the navigation APP on the target vehicle was used, etc.

[0060] As an example, for a music app on a target vehicle, the multimodal information related to the target vehicle includes: information related to the music app on the target vehicle determined during a historical period of use of the music app of the target vehicle. The information related to the music app of the target vehicle includes: an identification of a determined recommended song.

[0061] refer to Figure 3 , which shows a schematic diagram of an example of determining target intent.

[0062] The user speaks an interactive voice, triggering the target vehicle to obtain multimodal information related to the target vehicle. The user's interactive voice is the user's voice for interaction. The user's interactive voice can be a natural language voice, that is, a voice in the form of a natural language. The user's interactive voice can reflect the operation that the user expects the target vehicle to perform. In order to obtain multimodal information related to the target vehicle, the voice data of the user's interactive voice, sensor data, the internal environment image of the target vehicle, the external environment image of the target vehicle and other data collected by the target vehicle are obtained. The data collected by the target vehicle is processed to obtain multimodal information related to the target vehicle. The multimodal information related to the target vehicle includes at least some of the following items: the voice recognition result of the user's interactive voice, the parameter information of the target vehicle, the external environment information of the target vehicle, the user's action information, the user's attribute information, the user's identity information, and the information related to the APP on the target vehicle determined during the historical time period of using the APP on the target vehicle.

[0063] Each information in the multimodal information related to the target vehicle is encoded to obtain a representation of the multimodal information related to the target vehicle.

[0064] A representation of multimodal information related to the target vehicle is input into the target model, and the target model outputs the target intention.

[0065] In step S201, multimodal information related to the target vehicle may be input into a target model, and the target model outputs a target intention.

[0066] The target model can be on the target vehicle or on a cloud server.

[0067] It should be noted that the target model can use part of the multimodal information related to the target vehicle to predict the target intention.

[0068] As an example, the interior temperature of the target vehicle is high, the user's hand is toward the window and makes a downward gesture, and the target intention is to open the window.

[0069] As an example, a user says an interactive voice "play songs suitable for people of my age group". The target intention is to play songs suitable for users of the age group to which the user belongs.

[0070] As an example, the user says the interactive voice "Move the seat back a little bit". The target intention is to move the seat of the user's sitting position back.

[0071] Step S202, start the dialogue service flow, the dialogue service flow includes: for each target intent among multiple target intents, send an instruction corresponding to the target intent to an application to which the atomic service function for implementing the target intent belongs in at least one application, so that the application calls the atomic service function for implementing the target intent according to the instruction.

[0072] In an embodiment of the invention, each APP with an atomic service function on the target vehicle can have multiple atomic service functions. When starting the dialog flow service, for each of the multiple target intents, an instruction corresponding to the target intent can be sent to at least one application to which the atomic service function for realizing the target intent belongs, so that the application calls the atomic service function for realizing the target intent according to the instruction.

[0073] As an example, the vehicle center APP on the target vehicle has atomic service functions such as the open window atomic service function for opening the window, the close window atomic service function for closing the window, etc. The air conditioning APP on the target vehicle has atomic service functions such as the turn on air conditioning atomic service function for turning on the air conditioning, the turn off air conditioning atomic service function for turning off the air conditioning, and the adjust air conditioning temperature atomic service function for adjusting the air conditioning temperature.

[0074] Once the target intent is determined, the dialogue service flow can be started to call the atomic service function for realizing the target intent. After calling the atomic service function for realizing the target intent, the call result corresponding to the atomic service function for realizing the target intent can be displayed in the dialogue service flow interface, wherein the dialogue service flow interface is displayed by the vehicle computer screen of the target vehicle.

[0075] The call result corresponding to the atomic service function for achieving the target intent is the result obtained by the atomic service function for achieving the target intent or the result indicating the completion of the target intent indication. For example, if the atomic service function for achieving the target intent is that the air-conditioning APP on the target vehicle has an atomic service function for turning on the air-conditioning, then the call result corresponding to the atomic service function for achieving the target intent is text indicating that the air-conditioning has been turned on. If the atomic service function for achieving the target intent is the atomic service function for searching for POIs of the search APP on the target vehicle, then the call result corresponding to the atomic service function for achieving the target intent is the search result searched out by calling the atomic service function for searching for POIs of the search APP on the target vehicle.

[0076] As an example, the user speaks an interactive voice related to a music APP on the target vehicle. The target model determines the target intent related to the music APP on the target vehicle, and calls the corresponding atomic service function of the music APP on the target vehicle to realize the target intent.

[0077] refer to Figure 4 , which shows a schematic diagram of the effect of calling the corresponding atomic service function of the navigation APP on the target vehicle to achieve the target intention.

[0078] When the user interacts with the interactive system of the target vehicle for the tth time, the user says the interactive voice "Go to the... subway station to pick up someone, then go to... and find a place to eat", and two target intentions are determined, namely, generating a navigation route passing through the... subway station and the "..." in "then to..." and searching for multiple restaurants near the "..." in "then to...". Start the dialogue service flow, and call the atomic service function for generating navigation routes of the navigation APP on the target vehicle to generate a navigation route passing through the... subway station and the "..." in "then to...". Call the atomic service function for searching POIs of the search APP on the target vehicle to search for multiple restaurants near the "..." in "then to...". When the user interacts with the target vehicle's interactive system for the t+1th time, the user says the interactive voice "Is there a hot pot restaurant?" The target intention is to find a hot pot restaurant from multiple restaurants, start the dialogue service flow, and call the atomic service function for searching POIs of the search APP on the target vehicle to find a hot pot restaurant from multiple restaurants. When the user interacts with the target vehicle's interactive system for the t+1th time, "These 5 hot pots are all good" is displayed in the dialogue service flow interface. When the user interacts with the target vehicle's interactive system for the t+2th time, the user says the interactive voice "Choose one that is easy to park", start the dialogue service flow, call the atomic service function for searching POIs of the search APP on the target vehicle, and the atomic service function for searching POIs of the search APP on the target vehicle finds a hot pot restaurant with easy parking from the 5 hot pot restaurants based on the supporting facilities of the points of interest near the target vehicle in the external environment information of the target vehicle. When the user interacts with the interactive system of the target vehicle for the t+3th time, the user utters the interactive voice "Just select...", and the target intention is to navigate to the "..." in "Just select...", which starts the dialogue service flow and calls the atomic service function for generating navigation routes of the navigation APP on the target vehicle to generate a navigation route from the current location of the target vehicle to the "..." in "Just select...".

[0079] refer to Figure 5 , which shows a schematic diagram of the effect of calling the corresponding atomic service function of the music APP on the target vehicle to achieve the target intention.

[0080] When the user interacts with the target vehicle's interactive system for the tth time, the user utters the interactive voice "Which song is the famous song of...". The target intention is to search for the name of the famous song of "...", and the dialogue service flow is started, and the atomic service function for searching songs of the music APP on the target vehicle is called to search for the name of the famous song of "..." from the knowledge base. When the user interacts with the target vehicle's interactive system for the t+1th time, the user utters the interactive voice "What is the creative background of the song?", and the target intention is to search for the creative background of the famous song of "...". The dialogue service flow is started, and the atomic service function for searching music background knowledge of the music APP on the target vehicle is called to search for the creative background of the famous song of "..." from the knowledge base. When the user interacts with the target vehicle's interactive system for the t+2th time, the user utters the interactive voice "Which album is it in?", and the target intention is to search for the album to which the song belongs.

[0081] Start the dialogue service flow, call the atomic service function for searching the album to which the song belongs of the music APP on the target vehicle, and search the name of the album to which the famous song of "..." belongs from the knowledge base. When the user in the target vehicle interacts with the interactive system of the target vehicle for the t+2th time, the user in the target vehicle utters the interactive voice "What other songs are there in this album?". The target intent is to find out the names of other songs in the album except the famous song of "...". Start the dialogue service flow, call the atomic service function for searching the song of the music APP on the target vehicle, and search the names of other songs in the album except the famous song of "...". When the user in the target vehicle interacts with the interactive system of the target vehicle for the t+3th time, the user in the target vehicle utters the interactive voice "Add these songs to the song playlist." The target intent is to add other songs to the song playlist.

[0082] Start the dialogue service flow and call the atomic service function of the music APP on the target vehicle for adding songs to the playlist to add other songs to the song playlist.

[0083] In step S203, information for guiding user interaction is provided to the user, where the information for guiding user interaction includes: a call result corresponding to the atomic service function for achieving the target intention.

[0084] The information used to guide user interaction can be played via voice or displayed via text in the dialogue service flow interface.

[0085] In this embodiment, by processing multimodal data, multiple target intentions related to the user are determined according to the multimodal information related to the target vehicle; the dialogue service flow is started, and the dialogue service flow includes: for each target intention among the multiple target intentions, the application to which the atomic service function used to achieve the target intention belongs in at least one application corresponding to the multiple target intentions can achieve the target intention. The user does not need to perform a series of operations on the car screen to trigger the operation that the user intends to perform, which simplifies the user's operation and improves the convenience of interaction between the vehicle and the user. At the same time, information for guiding user interaction is provided to the user, and the information for guiding user interaction includes: the call results corresponding to the atomic service functions used to achieve the target intention, so that the user can intuitively see each call result.

[0086] In one possible implementation, providing information to the user for guiding user interaction includes: displaying the call result corresponding to the atomic service function used to achieve the target intent in the dialogue service flow interface or playing a voice indicating the call result corresponding to the atomic service function used to achieve the target intent.

[0087] As an example, when the target vehicle is driving, the user becomes interested in a community that the target vehicle passes through, and the user says the interactive voice "What community is ahead?" The target intention is to search for the name of the community ahead of the target vehicle, and then the call result is "The community ahead is community A." The atomic service function for searching POIs of the search APP on the target vehicle is called to search for the name of the community ahead of the target vehicle. The user says the interactive voice "How much does it cost to rent a three-bedroom apartment?" The target intention is to search for the rent of a three-bedroom apartment in the community ahead of the target vehicle, and the call result is "Rent xxx yuan."

[0088] In one possible implementation, the information for guiding user interaction also includes: information indicating recommended atomic service functions corresponding to the atomic service functions for achieving the target intent, providing the user with information for guiding user interaction, and also including: displaying a user interface component of the recommended atomic service functions corresponding to the atomic service functions for achieving the target intent in the dialogue service flow interface or playing a voice indicating the recommended atomic service functions corresponding to the atomic service functions for achieving the target intent.

[0089] In addition to the call result corresponding to the atomic service function for achieving the target intent, the information provided to the user for guiding the user interaction also includes information indicating the recommended atomic service function corresponding to the atomic service function for achieving the target intent. The information indicating the recommended atomic service function corresponding to the atomic service function for achieving the target intent can be played by voice or displayed on the dialogue service flow interface.

[0090] In order to determine the recommended atomic service function corresponding to the atomic service function for achieving the target intent, the association relationship between the atomic service functions can be preset. Other atomic service functions related to the atomic service function for achieving the target intent are determined as the recommended atomic service function corresponding to the atomic service function for achieving the target intent. The target model can also predict the recommended atomic service function corresponding to the atomic service function for achieving the target intent based on the association between the atomic service functions.

[0091] As an example, the target intent is to recommend songs, and the atomic service function used to achieve the target intent is the atomic service function of searching songs in the music app on the target vehicle. Start the conversation service flow, and call the atomic service function of searching songs in the music app on the target vehicle to search for recommended songs. The atomic service function corresponding to the atomic service function used to achieve the target intent is the atomic service function of the music app on the target vehicle for setting the sound effects of the recommended songs.

[0092] refer to Figure 6 , which shows a schematic diagram of the effect of calling the corresponding atomic service function of the music APP on the target vehicle to achieve the target intention.

[0093] The target intention is that each of the first songs currently recommended has been played, and the second song will continue to be recommended. Start the dialogue service flow, call the atomic service function of searching songs on the music APP on the target vehicle to search for the recommended second song. Among them, the recommended second song is different from the recommended first song. In the dialogue service flow interface, it is displayed "The recommended songs have been played, and some new songs are recommended for you." The recommended song in "The recommended songs have been played, and some new songs are recommended for you" refers to the first song. The identifier of the new song is displayed in the dialogue service flow interface. The new song refers to the second song.

[0094] In a possible implementation, after the information for guiding user interaction is provided to the user, in response to the user's operation on a corresponding information item in the information for guiding user interaction, a feedback operation corresponding to the corresponding information item is performed.

[0095] The feedback operation may be an output generated according to the user's interactive voice and operation result, and used to convey information to the user. Specifically, after providing the user with information for guiding user interaction, in response to the user's operation on the corresponding information item in the information for guiding user interaction, a feedback operation corresponding to the corresponding information item is performed.

[0096] In a possible implementation manner, the user's operation may be clicking on a corresponding information item in the information for guiding user interaction or speaking a voice for the corresponding information item in the information for guiding user interaction.

[0097] In one possible implementation, if the user's operation on a corresponding information item in the information for guiding user interaction is the user's operation on information related to the recommended atomic service function corresponding to the atomic service function for achieving the target intention in the information for guiding user interaction, the feedback operation corresponding to the corresponding information item is to call the recommended atomic service function corresponding to the atomic service function for achieving the target intention.

[0098] As an example, the atomic service function used to achieve the target intention is the atomic service function of the seat massage APP for turning on the seat massage. The recommended atomic service function corresponding to the atomic service function for turning on the seat massage is used to set the parameter values ​​of the working parameters of the seat massage function. The corresponding information items in the information for guiding user interaction include: the parameter values ​​of the working parameters of the seat massage function. The user speaks the voice for the parameter values ​​of the working parameters of the seat massage function to turn on the recommended atomic service function corresponding to the atomic service function of the seat massage.

[0099] refer to Figure 7 , which shows a flowchart of an example of interaction between a target vehicle and a user.

[0100] The multimodal information related to the target vehicle is input into the target model, and the target model outputs the target intent. The skill distribution function of the voice APP of the target vehicle will send the instruction corresponding to the target intent to the APP of the vehicle center APP, navigation APP, music APP and other APPs of the target vehicle for each target intent in at least one target intent. The APP to which the atomic service function for realizing the target intent belongs calls the atomic service function for realizing the target intent according to the instruction corresponding to the target intent, and obtains the call result corresponding to the atomic service function for realizing the target intent. When providing information for guiding user interaction to the user, in this example, in the dialogue service flow interface, the call result corresponding to the atomic service function for realizing the target intent and the UI component of the recommended atomic service function corresponding to the atomic service function for realizing the target intent are displayed. In this example, the user's operation on the corresponding information item in the information for guiding user interaction is the user's operation on the information related to the recommended atomic service function corresponding to the atomic service function for realizing the target intent in the information for guiding user interaction. In this example, the feedback operation corresponding to the corresponding information item is to call the recommended atomic service function corresponding to the atomic service function for achieving the target intention.

[0101] In the embodiment of the present invention, it is considered that the user in the vehicle often needs to further search for points of interest that meet corresponding conditions from the points of interest that have been searched. Points of interest that meet the conditions indicated by the speech recognition result of the interactive speech of the user in the target vehicle can be searched from the multiple points of interest that have been searched, without the user in the target vehicle having to determine the points of interest that meet the corresponding conditions by himself. The operation of the user in the target vehicle is simplified and the user experience is improved.

[0102] In this embodiment, an interactive device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0103] An interactive device is also provided in an embodiment of the present invention, and the device is used to implement the above-mentioned method embodiment and preferred implementation mode, which have been described and will not be repeated here. As used below, the term "unit" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived. The device in the embodiment of the present invention is presented in the form of a functional unit, where the functional unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0104] An interactive device, the interactive device comprising:

[0105] an intention determination unit, configured to determine a plurality of target intentions associated with the user based on multimodal information associated with the target vehicle;

[0106] A starting unit, configured to start a dialog service flow, wherein the dialog service flow includes: for each target intent among a plurality of target intents, sending an instruction corresponding to the target intent to an application to which an atomic service function for realizing the target intent belongs in at least one application, so that the application calls the atomic service function for realizing the target intent according to the instruction;

[0107] The guiding unit is used to provide information for guiding user interaction to the user, and the information for guiding user interaction includes: a call result corresponding to the atomic service function for realizing the target intention.

[0108] Furthermore, the guiding unit is further configured to:

[0109] The call result corresponding to the atomic service function used to achieve the target intention is displayed in the dialogue service flow interface, or a voice indicating the call result corresponding to the atomic service function used to achieve the target intention is played.

[0110] Furthermore, the information for guiding the user interaction also includes: information indicating the recommended atomic service function corresponding to the atomic service function for achieving the target intention, and the guiding unit is further used to:

[0111] A user interface component of a recommended atomic service function corresponding to the atomic service function for achieving the target intent is displayed in the dialogue service flow interface, or a voice indicating the recommended atomic service function corresponding to the atomic service function for achieving the target intent is played.

[0112] Furthermore, the intention determination unit is used to utilize the target model on the cloud server to determine multiple target intentions related to the user based on multimodal information related to the target vehicle.

[0113] Furthermore, the interaction device further includes: an execution unit, configured to execute a feedback operation corresponding to the corresponding information item in response to an operation of the user on the corresponding information item in the information for guiding user interaction.

[0114] The interactive device in this embodiment is presented in the form of a functional unit, where the functional unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0115] refer to Figure 8 , Figure 81 is a schematic diagram of the hardware structure of a computer device provided by an embodiment of the present invention. The computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used with multiple memories and multiple memories. Similarly, multiple vehicles can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). The processor 10 can be a central processing unit, a network processor, or a combination thereof. Among them, the processor 10 can further include a hardware chip. The above-mentioned hardware chip can be a dedicated integrated circuit, a programmable logic device, or a combination thereof. The above-mentioned programmable logic device can be a complex programmable logic device, a field programmable gate array, a general array logic, or any combination thereof. Among them, the memory 20 stores instructions that can be executed by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment. The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created according to the use of the vehicle, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof. The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state hard disk; the memory 20 may also include a combination of the above types of memories. The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30, and the output device 40 may be connected via a bus or in other ways. The input device 30 can receive input digital or character information, and generate key signal input related to user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, an indicator stick, one or more mouse buttons, a trackball, a joystick, etc.The output device 40 may include a display device, an auxiliary lighting device (e.g., LED), a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display, and a plasma display. In some optional embodiments, the display device may be a touch screen.

[0116] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0117] A portion of the embodiments of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in a computer-readable medium includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which the computer program instructions are executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0118] The above embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Any equivalent substitution or change made by a person skilled in the art based on the present invention is within the protection scope of the present invention.

Claims

1. An interactive method, characterized in that: The method comprises: Determining multiple target intentions associated with the user based on multimodal information associated with the target vehicle; Start the dialogue service flow, which includes: For each target intent among the multiple target intents, sending an instruction corresponding to the target intent to an application to which an atomic service function for realizing the target intent belongs in at least one application, so that the application calls the atomic service function for realizing the target intent according to the instruction; Information for guiding user interaction is provided to the user, and the information for guiding user interaction includes: a call result corresponding to an atomic service function for achieving a target intent.

2. The interactive method according to claim 1, characterized in that: Information provided to users to guide user interaction includes: The call result corresponding to the atomic service function used to achieve the target intention is displayed in the dialogue service flow interface, or a voice indicating the call result corresponding to the atomic service function used to achieve the target intention is played.

3. The interactive method according to claim 2, characterized in that: The information for guiding user interaction also includes: information indicating the recommended atomic service function corresponding to the atomic service function for achieving the target intention, and providing the information for guiding user interaction to the user, and also includes: A user interface component of a recommended atomic service function corresponding to the atomic service function for achieving the target intent is displayed in the dialogue service flow interface, or a voice indicating the recommended atomic service function corresponding to the atomic service function for achieving the target intent is played.

4. The interactive method according to claim 1, characterized in that: Based on the multimodal information related to the target vehicle, multiple target intentions related to the user are determined, including: Using the target model on the cloud server, multiple target intentions related to the user are determined based on the multimodal information related to the target vehicle.

5. The interactive method according to any one of claims 1 to 4, characterized in that: The method further comprises: After the information for guiding user interaction is provided to the user, in response to the user's operation on the corresponding information item in the information for guiding user interaction, a feedback operation corresponding to the corresponding information item is performed.

6. An interactive device, characterized in that: The device comprises: an intention determination unit, configured to determine a plurality of target intentions associated with the user based on multimodal information associated with the target vehicle; A starting unit, configured to start a dialog service flow, the dialog service flow comprising: for each target intent among a plurality of target intents, sending an instruction corresponding to the target intent to an application to which an atomic service function for realizing the target intent belongs in at least one application, so that the application calls the atomic service function for realizing the target intent according to the instruction; The guiding unit is used to provide information for guiding user interaction to the user, and the information for guiding user interaction includes: a call result corresponding to the atomic service function for realizing the target intention.

7. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the interactive method according to any one of claims 1 to 5 by executing the computer instructions.

8. A vehicle, characterized in that: The vehicle comprises: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the interactive method according to any one of claims 1 to 5 by executing the computer instructions.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the interactive method according to any one of claims 1 to 5.

10. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the interactive method according to any one of claims 1 to 5.

Images