Interaction method and device, computer equipment and storage medium

Abstract

The invention relates to an interaction method and device, computer equipment and a storage medium. The method comprises the following steps: determining a target intention related to a user according to multi-modal information related to a target vehicle; and starting a dialogue service flow, wherein the dialogue service flow comprises calling an atomization service function for realizing the target intention.

Description

Technical Field

[0001] The present invention relates to the field of vehicles, and in particular to an interaction method, device, computer equipment and storage medium. Background Art

[0002] In the related art, when a user intends to have the vehicle perform a certain operation, the user needs to perform a series of operations on the vehicle's screen, such as opening the APP to which the operation belongs and opening the page of the APP related to the operation, in order to trigger the execution of the operation. The user needs to perform cumbersome operations to make the vehicle perform a certain operation, resulting in poor convenience in the interaction between the vehicle and the user. How to improve the convenience of the interaction between the vehicle and the user has become a problem that needs to be solved. Summary of the invention

[0003] One of the purposes of the present invention is to provide an interaction method, apparatus, computer device and storage medium to solve the problem of how to improve the convenience of interaction between a vehicle and a user in the vehicle.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0005] An interactive method, comprising:

[0006] Determine the target intention associated with the user based on the multimodal information associated with the target vehicle;

[0007] Start a dialog service flow, which includes calling an atomic service function for realizing a target intent.

[0008] According to the above technical means, the target intent related to the user is determined based on the multimodal information related to the target vehicle; the dialogue service flow is started, and the dialogue service flow includes: calling the atomic service function used to achieve the target intent to achieve the target intent, and the user does not need to perform a series of operations on the vehicle screen to trigger the operation that the user intends to perform, which simplifies the user's operation and improves the convenience of interaction between the vehicle and the user. At the same time, the multimodal information related to the target vehicle is obtained, and the target intent is determined based on the target model and the multimodal information related to the target vehicle. The multimodal information related to the target vehicle has a strong correlation with the user's intention. The accuracy of the determined target intent is high, and the atomic service function is called more accurately.

[0009] Furthermore, the dialogue service flow also includes:

[0010] Determine the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent;

[0011] Information for guiding user interaction is provided to the user, and the information for guiding user interaction includes: call results corresponding to the atomic service functions for achieving the target intent, and information related to the recommended atomic service functions corresponding to the atomic service functions for achieving the target intent.

[0012] Further, providing information to the user for guiding user interaction includes:

[0013] Information for guiding user interaction is displayed in the dialog service flow interface, or a voice indicating an information item in the information for guiding user interaction is played.

[0014] Further, the recommended atomic service functions corresponding to the atomic service functions for achieving the target intent include:

[0015] According to the information indicating the user's usage habits for the atomic service functions, a recommended atomic service function corresponding to the atomic service function for achieving the target intention is determined.

[0016] Furthermore, the dialogue service flow also includes:

[0017] After the information for guiding user interaction is provided to the user, in response to the user's operation on a corresponding information item in the information for guiding user interaction, a feedback operation corresponding to the corresponding information item is performed.

[0018] Furthermore, the recommended atomic service function is used to set parameter values ​​of working parameters of vehicle functions corresponding to the atomic service functions for achieving target intentions.

[0019] Further, the atomic service function for achieving the target intention is used to activate the vehicle function for coping with the target situation, and the recommended atomic service function is used to activate the alternative vehicle function for coping with the target situation.

[0020] Furthermore, the atomic service functions called to achieve the target intent include:

[0021] When there are multiple target intents, the atomic service function corresponding to each of the multiple target intents is called, wherein the atomic service function corresponding to the target intent is used to realize the target intent.

[0022] Furthermore, the atomic service functions called to achieve the target intent include:

[0023] For each target intent in at least one target intent, an instruction corresponding to the target intent is sent to an application to which the atomic service function for implementing the target intent belongs in at least one application corresponding to the at least one target intent, so that the application calls the atomic service function for implementing the target intent according to the instruction.

[0024] Further, based on the multimodal information related to the target vehicle, determining the target intention includes:

[0025] The target model on the cloud server is used to determine the target intention based on the multimodal information related to the target vehicle.

[0026] Furthermore, it also includes:

[0027] When it is detected that a user-defined scenario trigger condition is met, an atomic service function is called to implement the intent corresponding to the scenario trigger condition.

[0028] Furthermore, it also includes:

[0029] In response to detecting an operation performed on the dialogue service flow interface indicating reduction or enlargement of the size of the image frame, calculating a ratio of a height of the image frame adjusted as indicated by the operation to a height of a display screen of the target vehicle;

[0030] According to the ratio, the original image frame including the information to be displayed in the dialogue service flow interface is reduced or enlarged to obtain the target image frame, and the target image frame is displayed in the dialogue service flow interface.

[0031] An interactive device, the interactive device comprising:

[0032] an intention determination unit, for determining a target intention associated with the user based on multimodal information associated with the target vehicle;

[0033] The starting unit is used to start the dialogue service flow, and the dialogue service flow includes: calling an atomic service function for realizing the target intention.

[0034] Furthermore, the dialogue service flow also includes:

[0035] Determine the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent;

[0036] Information for guiding user interaction is provided to the user, and the information for guiding user interaction includes: call results corresponding to the atomic service functions for achieving the target intent, and information related to the recommended atomic service functions corresponding to the atomic service functions for achieving the target intent.

[0037] Further, providing information to the user for guiding user interaction includes:

[0038] Information for guiding user interaction is displayed in the dialog service flow interface, or a voice indicating an information item in the information for guiding user interaction is played.

[0039] Further, the recommended atomic service functions corresponding to the atomic service functions for achieving the target intent include:

[0040] According to the information indicating the user's usage habits for the atomic service functions, a recommended atomic service function corresponding to the atomic service function for achieving the target intention is determined.

[0041] Furthermore, the dialogue service flow also includes:

[0042] After the information for guiding user interaction is provided to the user, in response to the user's operation on a corresponding information item in the information for guiding user interaction, a feedback operation corresponding to the corresponding information item is performed.

[0043] Furthermore, the recommended atomic service function is used to set parameter values ​​of working parameters of vehicle functions corresponding to the atomic service functions for achieving target intentions.

[0044] Further, the atomic service function for achieving the target intention is used to activate the vehicle function for coping with the target situation, and the recommended atomic service function is used to activate the alternative vehicle function for coping with the target situation.

[0045] Furthermore, the atomic service functions called to achieve the target intent include:

[0046] When there are multiple target intents, the atomic service function corresponding to each of the multiple target intents is called, wherein the atomic service function corresponding to the target intent is used to realize the target intent.

[0047] Furthermore, the atomic service functions called to achieve the target intent include:

[0048] For each target intent in at least one target intent, an instruction corresponding to the target intent is sent to an application to which the atomic service function for implementing the target intent belongs in at least one application corresponding to the at least one target intent, so that the application calls the atomic service function for implementing the target intent according to the instruction.

[0049] Furthermore, the intention determination unit is further used to utilize the target model on the cloud server to determine the target intention based on the multimodal information related to the target vehicle.

[0050] Furthermore, the interactive device also includes:

[0051] The scene trigger unit is used to call an atomic service function for realizing the intention corresponding to the scene trigger condition when it detects that the scene trigger condition defined by the user is met.

[0052] Furthermore, the interactive device also includes:

[0053] a scaling unit, for, in response to detecting an operation performed on the dialogue service flow interface indicating reduction or enlargement of the size of the image frame, calculating a ratio of a height of the image frame adjusted as indicated by the operation to a height of a display screen of the target vehicle;

[0054] According to the ratio, the original image frame including the information to be displayed in the dialogue service flow interface is reduced or enlarged to obtain the target image frame, and the target image frame is displayed in the dialogue service flow interface.

[0055] A computer device comprising:

[0056] A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the above method by executing the computer instructions.

[0057] A computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the above method.

[0058] A vehicle comprising:

[0059] A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the above method by executing the computer instructions.

[0060] A computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the above method.

[0061] A computer program product, characterized in that it includes computer instructions, wherein the computer instructions are used to enable a computer to execute the above method.

[0062] Beneficial effects of the present invention:

[0063] According to the multimodal information related to the target vehicle, the target intent related to the user is determined; the dialogue service flow is started, and the dialogue service flow includes: calling the atomic service function used to achieve the target intent to achieve the target intent. The user does not need to perform a series of operations on the vehicle screen to trigger the operation that the user intends to perform, which simplifies the user's operation and improves the convenience of interaction between the vehicle and the user. At the same time, the multimodal information related to the target vehicle is obtained, and the target intent is determined according to the target model and the multimodal information related to the target vehicle. The multimodal information related to the target vehicle has a strong correlation with the user's intention. The accuracy of the determined target intent is high, and the atomic service function is called more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A schematic diagram of an example architecture of an interactive system that can be used to execute the interactive method provided in an embodiment of the present invention;

[0065] Figure 2 A schematic diagram of a flow chart of an interactive method provided by an embodiment of the present invention;

[0066] Figure 3 A schematic diagram of an example for determining target intent;

[0067] Figure 4 A flowchart for an example of determining target intent;

[0068] Figure 5 A flow chart of an example for determining a target intent associated with a navigation app on a target vehicle;

[0069] Figure 6 A flow chart of an example for determining a target intent associated with a music app on a target vehicle;

[0070] Figure 7 A schematic diagram of the effect of calling the corresponding atomic service function of the music APP on the target vehicle to realize the target intention;

[0071] Figure 8 A schematic diagram of the effect of calling the corresponding atomic service function of the music APP on the target vehicle to realize the target intention;

[0072] Fig. 9 A schematic diagram of the effect of calling the corresponding atomic service function of the navigation APP on the target vehicle to realize the target intention;

[0073] Fig.10 A schematic diagram of the effect of calling the corresponding atomic service function of the navigation APP on the target vehicle to realize the target intention;

[0074] Fig.11 A flowchart showing an example of interaction between a target vehicle and a user;

[0075] Fig.12 A schematic diagram of an example of recommending songs and sound effects;

[0076] Fig.13 A schematic diagram showing the effect of a UI component of a recommended atomic service function corresponding to an atomic service function for achieving a target intention;

[0077] Fig.14 A schematic diagram showing the effects of the vehicle functions that have been turned on and the alternative vehicle functions;

[0078] Fig.15A schematic diagram of an example of simultaneously enabling the atomic service function corresponding to each of multiple intents;

[0079] Fig.16 A schematic diagram of an example for determining a scene triggering condition and an intention corresponding to the scene triggering condition;

[0080] Fig.17 A schematic diagram of the effect of calling an atomic service function for realizing an intention corresponding to a scenario trigger condition;

[0081] Fig.18 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0082] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Users skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0083] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0084] refer to Figure 1 , which shows a schematic diagram of an example architecture of an interactive system that can be used to execute the interactive method provided by an embodiment of the present invention.

[0085] An example architecture of an interactive system that can be used to execute the interactive method provided by the embodiment of the present invention includes: a data collection unit, a data processing unit, and an atomic service function unit.

[0086] The data acquisition unit is used to collect voice data of the user's interactive voice, the internal environment image of the target vehicle, the external environment image of the target vehicle, the sensor data of the target vehicle, etc. The data processing unit includes a voice data processing module, an image processing module, a sensor data processing module, an encoding module, and a target model. The atomic service function includes: the atomic service function of each APP with atomic service function on the target vehicle. The atomic service function can be understood as the smallest functional unit, and the atomic service function can perform a single operation.

[0087] The voice data processing module can perform voice recognition on the user's interactive voice to obtain a voice recognition result of the user's interactive voice. The voice data processing unit can use a sound source localization algorithm to determine the seat position of the user who speaks the interactive voice in the target vehicle. The voice data processing unit can determine the identity of the user by matching the voiceprint feature of the user's interactive voice with the preset voiceprint feature. The preset voiceprint feature is the voiceprint feature of the user who has recorded the voiceprint feature.

[0088] The image processing module can process the internal environment image of the target vehicle to determine the age group to which the user belongs, the user's gender, the user's sitting position, the user's identity information, etc. The image processing unit can include a driver monitoring system (DMS) and an occupant monitoring system (OMS). The driver monitoring system monitors the direction of the driver's line of sight in the target vehicle in real time. The occupant monitoring system can process the vehicle's internal environment image captured by the camera of the target vehicle to determine the gender, age, gesture, user's sitting position, etc. of the passenger in the target vehicle.

[0089] The sensor data processing module can process the sensor data collected by the sensor of the target vehicle to determine the information related to the driving of the target vehicle and the information related to the weather in the area where the target vehicle is located. The information related to the driving of the target vehicle may include: the speed of the target vehicle, the remaining fuel of the target vehicle or the remaining power of the battery of the target vehicle, etc. The information related to the weather in the area where the target vehicle is located may include: the temperature of the weather and the type of the weather. The type of weather may be raining, snowing, etc.

[0090] The encoding module is used to encode the multimodal information related to the target vehicle to obtain a representation of the multimodal information related to the target vehicle.

[0091] It should be noted that the target model can be on the target vehicle or on a cloud server.

[0092] The target model is used to receive the representation of multimodal information related to the target vehicle and predict the representation of the target intention. Thus, the target intention is determined. The target model is a trained neural network. Before the interactive method provided by the embodiment of the present invention is executed for the first time, the target model is trained using the training samples of the target model and the labeling operation of the training samples of the target model.

[0093] As an example, the target model is a pre-trained language model. As an example, the pre-trained language model used as the target model is a large language model (LLM). When the input of the target model is a natural language text corresponding to a natural language speech, the pre-trained language model performs a natural language processing (NLP) task.

[0094] Training the target model can enable the target model to learn: which information in the input of the target model to use to predict the operation that satisfies the user's interactive voice instruction intention, and how to determine the target intention. The training samples of the target model may include: information used for training on the type of information involved in the multimodal information related to the target vehicle. For a piece of information, if the multimodal information related to the target vehicle includes the information, then the type of the information is the type of information involved in the multimodal information related to the target vehicle. The labeled operation of the training sample of the target model can be understood as: when the training sample of the target model is input into the target model, the operation that the target model is expected to output. When training the target model, the loss between the operation predicted by the target model and the labeled operation of the training sample can be calculated, and the parameters of the target model can be updated based on the loss.

[0095] refer to Figure 2 , which shows a flow chart of an interactive method provided by an embodiment of the present invention. The method can be executed by a computer device such as a vehicle-mounted terminal or a mobile terminal of a target vehicle.

[0096] Step S201, determining the target intention based on the multimodal information related to the target vehicle.

[0097] The multimodal information related to the target vehicle includes: speech recognition results of the user's interactive speech, parameter information of the target vehicle, external environment information of the target vehicle, user's action information, user's riding position information, user's attribute information, user's identity information, information related to the APP on the target vehicle determined within a historical time period of using the APP on the target vehicle, navigation information related to the target vehicle, music information related to the target vehicle, and point of interest information related to the target vehicle, wherein the point of interest information related to the target vehicle indicates the points of interest reached by the target vehicle within a target historical time period and / or the points of interest reached by the target vehicle multiple times.

[0098] The speech recognition result of the user's interactive speech is obtained by performing speech recognition on the user's interactive speech. Performing speech recognition on the user's interactive speech specifically refers to performing speech recognition on the speech data of the user's interactive speech.

[0099] As an example, the parameter information of the target vehicle may include: information related to the driving of the target vehicle, and information related to the cabin of the target vehicle.

[0100] As an example, the information related to the driving of the target vehicle includes: the speed of the target vehicle, the remaining fuel of the target vehicle, or the remaining power of the battery of the target vehicle.

[0101] As an example, information related to the target vehicle's cabin includes: the brightness of the target vehicle's screen, the volume of the target vehicle's audio system, the temperature of the target vehicle's air conditioner, the air volume of the target vehicle's air conditioner, the angle of the target vehicle's seat, etc.

[0102] The external environment information of the target vehicle is information related to the external environment of the target vehicle.

[0103] As an example, the external environment information of the target vehicle includes: information related to the weather in the area where the target vehicle is located, information related to the driving safety of the road where the target vehicle is currently located, points of interest (POI) near the target vehicle, etc.

[0104] As an example, the information related to the weather in the area where the target vehicle is located includes the temperature of the weather, the type of the weather, and the like.

[0105] As an example, the information related to the driving safety of the vehicle on the road that the target vehicle is currently on includes one or more of the following: information indicating whether the road that the target vehicle is currently on is slippery, information indicating whether the road that the target vehicle is currently on is icy, information indicating whether the road that the target vehicle is currently on is bumpy, information indicating whether the road that the target vehicle is currently on is a plateau road, etc.

[0106] As an example, the user's action information includes at least some of the following items: the user's gesture, the direction in which the user's hand points, the direction in which the user's line of sight is directed, the user's action, and the like.

[0107] As an example, the attribute information of the user includes: the age group to which the user belongs, the gender of the user, etc.

[0108] The user's identity information is used to identify the user.

[0109] As an example, the user's identity information includes: the user's identifier.

[0110] The user's sitting position information indicates the user's sitting position. As an example, the user's sitting position is one of the following items: main driving position, co-pilot position, rear left position, rear middle position, rear right position.

[0111] The navigation information related to the target vehicle may include: a departure point of the current navigation of the target vehicle, a destination of the current navigation of the target vehicle, and a navigation route of the current navigation of the target vehicle.

[0112] The music information related to the target vehicle may include: a song playlist to which the identifier of the song currently played by the target vehicle belongs, and a play progress of the song to which the identifier belongs in the song playlist. The play progress indicates which song is being played.

[0113] The end time of the historical time period of using the APP on the target vehicle is before the current time. The APP on the target vehicle is used during the historical time period of using the APP on the target vehicle.

[0114] As an example, for a navigation APP on a target vehicle, the multimodal information related to the target vehicle includes: information related to the navigation APP on the target vehicle determined within a historical period of time when the navigation APP on the target vehicle was used. The information related to the navigation APP on the target vehicle determined within a historical period of time when the APP on the target vehicle was used includes: a navigation route determined within a historical period of time when the APP on the target vehicle was used, the name of the departure point of the navigation route determined within a historical period of time when the navigation APP on the target vehicle was used, the name of the destination of the navigation route determined within a historical period of time when the navigation APP on the target vehicle was used, etc.

[0115] As an example, for a music app on a target vehicle, the multimodal information related to the target vehicle includes: information related to the music app on the target vehicle determined during a historical period of use of the music app of the target vehicle. The information related to the music app of the target vehicle includes: an identification of a determined recommended song.

[0116] refer to Figure 3 , which shows a schematic diagram of an example of determining target intent.

[0117] The user speaks an interactive voice, triggering the target vehicle to obtain multimodal information related to the target vehicle. The user's interactive voice is the user's voice for interaction. The user's interactive voice can be a natural language voice, that is, a voice in the form of a natural language. The user's interactive voice can reflect the operation that the user expects the target vehicle to perform. In order to obtain multimodal information related to the target vehicle, the voice data of the user's interactive voice, sensor data, the internal environment image of the target vehicle, the external environment image of the target vehicle and other data collected by the target vehicle are obtained. The data collected by the target vehicle is processed to obtain multimodal information related to the target vehicle. The multimodal information related to the target vehicle includes at least some of the following items: the voice recognition result of the user's interactive voice, the parameter information of the target vehicle, the external environment information of the target vehicle, the user's action information, the user's attribute information, the user's identity information, and the information related to the APP on the target vehicle determined during the historical time period of using the APP on the target vehicle.

[0118] Each information in the multimodal information related to the target vehicle is encoded to obtain a representation of the multimodal information related to the target vehicle.

[0119] A representation of multimodal information related to the target vehicle is input into the target model, and the target model outputs the target intention.

[0120] In step S201, multimodal information related to the target vehicle may be input into a target model, and the target model outputs a target intention.

[0121] The target model can be on the target vehicle or on a cloud server.

[0122] It should be noted that the target model can use part of the multimodal information related to the target vehicle to predict the target intention.

[0123] As an example, the interior temperature of the target vehicle is high, the user's hand is toward the window and makes a downward gesture, and the target intention is to open the window.

[0124] As an example, a user says an interactive voice "play songs suitable for people of my age group". The target intention is to play songs suitable for users of the age group to which the user belongs.

[0125] As an example, the user says the interactive voice "Move the seat back a little bit". The target intention is to move the seat of the user's sitting position back.

[0126] refer to Figure 4 , which shows a flowchart of an example of determining target intent.

[0127] The internal environment image of the target vehicle indicates the internal environment of the target vehicle. By processing the internal environment image of the target vehicle, the age group to which the user's age belongs, the user's gestures, etc. can be obtained. The external environment image of the target vehicle indicates the external environment of the target vehicle. The sensor data and the external environment image and the sensor data are processed to obtain the external environment information of the target vehicle. The external environment information of the target vehicle may include: information related to the weather, i.e., information related to the weather in the area where the target vehicle is located, information related to the driving safety of the vehicle, i.e., information related to the driving safety of the vehicle on the road where the target vehicle is currently located, points of interest (POI) near the target vehicle, etc. The voice data of the user's interactive voice is processed to obtain the user's seating position and the voice recognition results of the user's interactive voice.

[0128] It should be noted that Figure 4 The various information shown in is information in the multimodal information related to the target vehicle that may be used to determine the target intention. The target model can use part of the multimodal information related to the target vehicle to predict the target intention. In other words, only the multimodal information related to the target vehicle is provided to the target model, and the target model decides which information in the multimodal information related to the target vehicle is used to determine the target intention.

[0129] Each information in the multimodal information related to the target vehicle is encoded to obtain a representation of the multimodal information related to the target vehicle.

[0130] A representation of multimodal information related to the target vehicle is input to a target model, and the target model outputs a representation of the target intent. Thus, the target intent is determined.

[0131] refer to Figure 5 , which shows a flowchart of an example of determining a target intent associated with a navigation app on a target vehicle.

[0132] The user speaks the interactive voice "What kind of community is that?" The speech recognition result of the user's interactive voice in the multimodal information related to the target vehicle is "What kind of community is that?" The external environment information of the target vehicle in the multimodal information related to the target vehicle includes: multiple points of interest near the target vehicle, supporting facilities of the points of interest near the target vehicle, etc. The user's action information in the multimodal information related to the target vehicle includes: the direction in which the user's hand is pointing. The target intention is to find the name of the point of interest in the direction in which the user's hand is pointing and the point of interest type is a community from the multiple points of interest near the target vehicle.

[0133] refer to Figure 6 , which shows a flowchart of an example of determining a target intent associated with a music APP on a target vehicle.

[0134] When the target vehicle is traveling according to the navigation route in the navigation information related to the target vehicle, the user speaks the interactive voice "play a song", and the multiple users in the target vehicle include children. The internal image of the target vehicle is processed to determine the age group to which the age of the user belongs. The destination in the navigation information related to the target vehicle is an amusement park. The target model determines the target intent based on the age group of the children and the destination amusement park, and the target intent is to play cheerful children's songs.

[0135] In step S202, a dialog service flow is started, and the dialog service flow includes: calling an atomic service function for realizing a target intent.

[0136] In an embodiment of the invention, each APP with an atomic service function on the target vehicle may respectively have multiple atomic service functions.

[0137] As an example, the vehicle center APP on the target vehicle has atomic service functions such as the open window atomic service function for opening the window, the close window atomic service function for closing the window, etc. The air conditioning APP on the target vehicle has atomic service functions such as the turn on air conditioning atomic service function for turning on the air conditioning, the turn off air conditioning atomic service function for turning off the air conditioning, and the adjust air conditioning temperature atomic service function for adjusting the air conditioning temperature.

[0138] Once the target intent is determined, the dialogue service flow can be started to call the atomic service function for realizing the target intent. After calling the atomic service function for realizing the target intent, the call result corresponding to the atomic service function for realizing the target intent can be displayed in the dialogue service flow interface, wherein the dialogue service flow interface is displayed by the vehicle computer screen of the target vehicle.

[0139] The call result corresponding to the atomic service function for achieving the target intent is the result obtained by the atomic service function for achieving the target intent or the result indicating the completion of the target intent indication. For example, if the atomic service function for achieving the target intent is that the air-conditioning APP on the target vehicle has an atomic service function for turning on the air-conditioning, then the call result corresponding to the atomic service function for achieving the target intent is text indicating that the air-conditioning has been turned on. If the atomic service function for achieving the target intent is the atomic service function for searching for POIs of the search APP on the target vehicle, then the call result corresponding to the atomic service function for achieving the target intent is the search result searched out by calling the atomic service function for searching for POIs of the search APP on the target vehicle.

[0140] As an example, when the target vehicle is driving, the user becomes interested in a community that the target vehicle passes through, and the user says the interactive voice "What community is ahead?" The target intention is to search for the name of the community ahead of the target vehicle. The atomic service function for searching POIs of the search APP on the target vehicle is called to search for the name of the community ahead of the target vehicle. The user says the interactive voice "How much does it cost to rent a three-bedroom apartment?" The target intention is to search for the rent of a three-bedroom apartment in the community ahead of the target vehicle. The atomic service function for searching the rent of a house agency APP on the target vehicle is called to search for the rent of a three-bedroom apartment in the community ahead of the target vehicle. The user says the interactive voice "Call and ask about the situation," and the target intention is to call the relevant personnel of the housing transaction platform. The atomic service function of making a call of the Bluetooth phone APP is called to call the relevant personnel of the housing transaction platform.

[0141] As an example, the user speaks an interactive voice related to a music APP on the target vehicle. The target model determines the target intent related to the music APP on the target vehicle, and calls the corresponding atomic service function of the music APP on the target vehicle to realize the target intent.

[0142] refer to Figure 7 , which shows a schematic diagram of the effect of calling the corresponding atomic service function of the music APP on the target vehicle to achieve the target intention.

[0143] When the user interacts with the interactive system of the target vehicle for the tth time, the user says the interactive voice "Which song is the famous song of...". The target intention is to search for the name of the famous song of "...", start the dialogue service flow, and call the atomic service function for searching songs on the music APP on the target vehicle to search for the name of the famous song of "..." from the knowledge base. When the user interacts with the interactive system of the target vehicle for the t+1th time, the user says the interactive voice "What is the creative background?", and the target intention is to search for the creative background of the famous song of "...". Start the dialogue service flow, and call the atomic service function for searching music background knowledge on the music APP on the target vehicle to search for the creative background of the famous song of "..." from the knowledge base. When the user interacts with the interactive system of the target vehicle for the t+2th time, the user says the interactive voice "Which album is it in?", and the target intention is to search for the album to which the song belongs. Start the dialogue service flow, and call the atomic service function for searching the album to which the music APP on the target vehicle to search for the name of the album to which the famous song of "..." belongs from the knowledge base. When the user in the target vehicle interacts with the interactive system of the target vehicle for the t+2th time, the user in the target vehicle utters the interactive voice "What other songs are there in this album?" The target intent is to find out the names of other songs in the album except for the famous song "...". Start the dialogue service flow, and call the atomic service function for searching songs of the music APP on the target vehicle to search for the names of other songs in the album except for the famous song "...". When the user in the target vehicle interacts with the interactive system of the target vehicle for the t+3th time, the user in the target vehicle utters the interactive voice "Add these songs to the song playlist." The target intent is to add other songs to the song playlist.

[0144] Start the dialogue service flow and call the atomic service function of the music APP on the target vehicle for adding songs to the playlist to add other songs to the song playlist.

[0145] refer to Figure 8 , which shows a schematic diagram of the effect of calling the corresponding atomic service function of the music APP on the target vehicle to achieve the target intention.

[0146] The target intention is that each of the first songs currently recommended has been played, and the second song will continue to be recommended. Start the dialogue service flow, call the atomic service function of searching songs on the music APP on the target vehicle to search for the recommended second song. Among them, the recommended second song is different from the recommended first song. In the dialogue service flow interface, it is displayed "The recommended songs have been played, and some new songs are recommended for you." The recommended song in "The recommended songs have been played, and some new songs are recommended for you" refers to the first song. The identifier of the new song is displayed in the dialogue service flow interface. The new song refers to the second song.

[0147] As an example, the user speaks an interactive voice related to the navigation APP on the target vehicle. The target model determines the target intent related to the navigation APP on the target vehicle, and calls the corresponding atomic service function of the navigation APP on the target vehicle to realize the target intent.

[0148] refer to Fig. 9 , which shows a schematic diagram of the effect of calling the corresponding atomic service function of the navigation APP on the target vehicle to achieve the target intention.

[0149] When the user interacts with the interactive system of the target vehicle for the tth time, the user says the interactive voice "Go to the... subway station to pick up someone, then go to... and find a place to eat", and two target intentions are determined, namely, generating a navigation route passing through the... subway station and the "..." in "then to..." and searching for multiple restaurants near the "..." in "then to...". Start the dialogue service flow, and call the atomic service function for generating navigation routes of the navigation APP on the target vehicle to generate a navigation route passing through the... subway station and the "..." in "then to...". Call the atomic service function for searching POIs of the search APP on the target vehicle to search for multiple restaurants near the "..." in "then to...". When the user interacts with the target vehicle's interactive system for the t+1th time, the user says the interactive voice "Is there a hot pot restaurant?" The target intention is to find a hot pot restaurant from multiple restaurants, start the dialogue service flow, and call the atomic service function for searching POIs of the search APP on the target vehicle to find a hot pot restaurant from multiple restaurants. When the user interacts with the target vehicle's interactive system for the t+1th time, "These 5 hot pots are all good" is displayed in the dialogue service flow interface. When the user interacts with the target vehicle's interactive system for the t+2th time, the user says the interactive voice "Choose one that is easy to park", start the dialogue service flow, call the atomic service function for searching POIs of the search APP on the target vehicle, and the atomic service function for searching POIs of the search APP on the target vehicle finds a hot pot restaurant with easy parking from the 5 hot pot restaurants based on the supporting facilities of the points of interest near the target vehicle in the external environment information of the target vehicle. When the user interacts with the interactive system of the target vehicle for the t+3th time, the user utters the interactive voice "Just select...", and the target intention is to navigate to the "..." in "Just select...", which starts the dialogue service flow and calls the atomic service function for generating navigation routes of the navigation APP on the target vehicle to generate a navigation route from the current location of the target vehicle to the "..." in "Just select...".

[0150] refer to Fig.10 , which shows a schematic diagram of the effect of calling the corresponding atomic service function of the navigation APP on the target vehicle to achieve the target intention.

[0151] The user says the interactive voice "Navigate to the barbecue restaurant I went to last week". The target intention is to generate a navigation route from the current position of the target vehicle to the barbecue restaurant the user went to last week, start the dialogue service flow, call the atomic service function for searching POIs of the search APP on the target vehicle, search for the barbecue restaurant the user went to last week, and call the atomic service function for displaying information to guide the user to say the interactive voice to display the information instructing the user to choose a barbecue restaurant "I went to two barbecue restaurants last week, which one should I go to". The user says the interactive voice "Go to the nearest one", and the atomic service function for generating navigation routes of the navigation APP on the target vehicle is called to generate a navigation route from the current position of the target vehicle to the barbecue restaurant that is closest to the target vehicle among the two barbecue restaurants the user went to last week, and the navigation route is displayed in the dialogue service flow interface. In addition, "Okay,...nearest, navigate to..." is also displayed in the dialogue service flow interface.

[0152] In one possible implementation, calling the atomic service function for realizing the target intent includes: for each target intent in at least one target intent, sending an instruction corresponding to the target intent to the APP to which the atomic service function for realizing the target intent belongs in at least one application corresponding to the at least one target intent, so that the APP to which the atomic service function for realizing the target intent belongs calls the atomic service function for realizing the target intent according to the instruction corresponding to the target intent.

[0153] Among them, at least one application corresponding to the at least one target intent is used to achieve the at least one target intent.

[0154] For a target intent, the instruction corresponding to the target intent may include: the ID of the atomic service function used to realize the target intent. The instruction corresponding to the target intent may also include: the data required to realize the target intent.

[0155] In one possible implementation, it also includes: determining a recommended atomic service function corresponding to the atomic service function used to achieve the target intent; providing information to the user for guiding user interaction, and the information for guiding user interaction includes: the call result corresponding to the atomic service function used to achieve the target intent, and information related to the recommended atomic service function corresponding to the atomic service function used to achieve the target intent.

[0156] As an example, the target intent is to recommend songs, and the atomic service function used to achieve the target intent is the atomic service function of searching songs on the music app on the target vehicle. Start the conversation service flow, and call the atomic service function of searching songs on the music app on the target vehicle to search for recommended songs. The atomic service function corresponding to the atomic service function used to achieve the target intent is the atomic service function of the music app on the target vehicle for setting the sound effects of the recommended songs.

[0157] In order to determine the recommended atomic service function corresponding to the atomic service function for achieving the target intent, the association relationship between the atomic service functions can be preset. Other atomic service functions related to the atomic service function for achieving the target intent are determined as the recommended atomic service functions corresponding to the atomic service function for achieving the target intent. The target model can also predict the recommended atomic service function corresponding to the atomic service function for achieving the target intent based on the association between the atomic service functions.

[0158] In one possible implementation, determining the recommended atomic service function corresponding to the atomic service function for achieving the target intent includes: determining the recommended atomic service function corresponding to the atomic service function for achieving the target intent based on information indicating the user's usage habits for the atomic service function.

[0159] The information indicating the user's usage habits for the atomic service functions indicates: the atomic service functions that have been called due to the target intent associated with the user. The target model can be used to determine the recommended atomic service functions corresponding to the atomic service functions for achieving the target intent based on the information indicating the user's usage habits for the atomic service functions.

[0160] In a possible implementation, providing the user with information for guiding user interaction includes: displaying the information for guiding user interaction in the dialog service flow interface or playing a voice indicating an information item in the information for guiding user interaction.

[0161] Among them, the call result corresponding to the atomic service function used to achieve the target intention is used as an information item in the information used to guide user interaction, and each information related to the recommended atomic service function corresponding to the atomic service function used to achieve the target intention is used as an information item in the information used to guide user interaction.

[0162] When information for guiding user interaction is displayed in the dialogue service flow interface, the user interface (User Interface, UI for short) components of the call results corresponding to the atomic service functions for achieving the target intent and the recommended atomic service functions corresponding to the atomic service functions for achieving the target intent are displayed in the dialogue service flow interface.

[0163] The user interface component of the recommended atomic service function corresponding to the atomic service function for achieving the target intent may include: the name of the recommended atomic service function corresponding to the atomic service function for achieving the target intent, and parameters related to the recommended atomic service function corresponding to the atomic service function for achieving the target intent.

[0164] When playing a voice indicating information used to guide user interaction, playing a voice indicating a call result corresponding to an atomic service function used to achieve a target intent, and playing a voice indicating information related to a recommended atomic service function corresponding to an atomic service function used to achieve a target intent.

[0165] In one possible implementation, the dialogue service flow also includes: after providing information for guiding user interaction to the user, in response to the user's operation on the corresponding information item in the information for guiding user interaction, performing a feedback operation corresponding to the user's operation on the corresponding information item in the information for guiding user interaction.

[0166] In a possible implementation manner, the user's operation may be clicking on a corresponding information item in the information for guiding user interaction or speaking a voice for the corresponding information item in the information for guiding user interaction.

[0167] In one possible implementation, if the user's operation on a corresponding information item in the information for guiding user interaction is the user's operation on information related to the recommended atomic service function corresponding to the atomic service function for achieving the target intention in the information for guiding user interaction, the feedback operation corresponding to the corresponding information item is to call the recommended atomic service function corresponding to the atomic service function for achieving the target intention.

[0168] As an example, the atomic service function used to achieve the target intention is the atomic service function of the seat massage APP for turning on the seat massage. The recommended atomic service function corresponding to the atomic service function for turning on the seat massage is used to set the parameter values ​​of the working parameters of the seat massage function. The corresponding information items in the information for guiding user interaction include: the parameter values ​​of the working parameters of the seat massage function. The user speaks the voice for the parameter values ​​of the working parameters of the seat massage function to turn on the recommended atomic service function corresponding to the atomic service function of the seat massage.

[0169] refer to Fig.11 , which shows a flowchart of an example of interaction between a target vehicle and a user.

[0170] The multimodal information related to the target vehicle is input into the target model, and the target model outputs the target intent. The skill distribution function of the voice APP of the target vehicle will send the instruction corresponding to the target intent to the APP of the vehicle center APP, navigation APP, music APP and other APPs of the target vehicle for each target intent in at least one target intent. The APP to which the atomic service function for realizing the target intent belongs calls the atomic service function for realizing the target intent according to the instruction corresponding to the target intent, and obtains the call result corresponding to the atomic service function for realizing the target intent. When providing information for guiding user interaction to the user, in this example, in the dialogue service flow interface, the call result corresponding to the atomic service function for realizing the target intent and the UI component of the recommended atomic service function corresponding to the atomic service function for realizing the target intent are displayed. In this example, the user's operation on the corresponding information item in the information for guiding user interaction is the user's operation on the information related to the recommended atomic service function corresponding to the atomic service function for realizing the target intent in the information for guiding user interaction. In this example, the feedback operation corresponding to the corresponding information item is to call the recommended atomic service function corresponding to the atomic service function for achieving the target intention.

[0171] refer to Fig.12 , which shows a schematic diagram of an example of recommended songs and sound effects.

[0172] The user says the interactive voice "Recommend a song". The dialogue service flow is started, and the atomic service function of searching songs of the music APP on the target vehicle is called to search for the recommended song. The screen of the target vehicle displays the logo of the recommended song.

[0173] The recommended atomic service function corresponding to the atomic service function for achieving the target intention is the atomic service function of the music APP on the target vehicle for setting the sound effects of the recommended songs. Fig.12 The UI component of the atomic service function for setting the sound effects of the recommended songs displayed in the dialogue service flow interface is shown. The UI component of the atomic service function for setting the sound effects of the recommended songs includes multiple sound effect names such as HIFI original sound, surging sound effects, immersive surround, concert hall, LiveHouse, and cinema.

[0174] The user interface component of the recommended atomic service function corresponding to the atomic service function for achieving the target intention allows the user to call the recommended atomic service function corresponding to the atomic service function for achieving the target intention by clicking on the user interface component of the recommended atomic service function corresponding to the atomic service function for achieving the target intention, speaking a voice instruction to call the recommended atomic service function corresponding to the atomic service function for achieving the target intention, etc., thereby facilitating the user to continue interacting with the vehicle and improving the convenience of interaction between the vehicle and the user.

[0175] In one possible implementation, the recommended atomic service function corresponding to the atomic service function for achieving the target intent is used to set the parameter value of the working parameter of the vehicle function corresponding to the atomic service function for achieving the target intent.

[0176] Among them, the vehicle function is a function provided by the components of the target vehicle.

[0177] If the recommended atomic service function corresponding to the atomic service function for achieving the target intention is used to set the parameter value of the working parameter of the vehicle function corresponding to the atomic service function for achieving the target intention, the UI component of the recommended atomic service function corresponding to the atomic service function for achieving the target intention includes: multiple candidate values ​​of the working parameter of the vehicle function corresponding to the atomic service function for achieving the target intention. The user can trigger the call of the recommended atomic service function corresponding to the atomic service function for achieving the target intention by clicking on the candidate value of the working parameter or speaking a voice indicating the selection of the candidate value of the working parameter, so as to set the parameter value of the working parameter to the candidate value selected by the user, so that the target atomic service function works according to the set parameter value.

[0178] refer to Fig.13 , which shows an effect schematic diagram of a UI component that displays recommended atomic service functions corresponding to atomic service functions for achieving target intent.

[0179] The user says the interactive voice "turn on seat massage", and the atomic service function used to achieve the target intention is the atomic service function of the seat massage APP for turning on seat massage. The dialogue service flow is started, and the atomic service function of the seat massage APP for turning on seat massage is called to turn on the seat massage vehicle function. Fig.13 The call result corresponding to the atomic service function for turning on seat massage displayed in the dialogue service flow interface is shown, namely, “OK, seat massage is turned on.” The recommended atomic service function corresponding to the atomic service function for turning on seat massage is used to set parameter values ​​of working parameters of the seat massage function. Fig.13The UI component of the recommended atomic service function corresponding to the atomic service function for starting seat massage displayed in the dialogue service flow interface is shown. The recommended atomic service function corresponding to the atomic service function for starting seat massage is an atomic service function for setting parameter values ​​of the working parameters of seat massage. The UI component of the recommended atomic service function corresponding to the atomic service function for starting seat massage includes: massage duration parameters, massage intensity parameters and other working parameters of the seat massage function. Fig.13 Candidate values ​​of the massage intensity parameter, namely, "level 1", "level 2", and "level 3", are shown. Fig.13 Multiple parameter values ​​of the massage duration parameter such as "15 minutes", "20 minutes", and "30 minutes" are shown. If the user says the interactive voice "level 1" and "30 minutes" in sequence, the seat massage function will work according to "level 1" and "30 minutes", that is, it will work at the massage intensity of level 1 for 30 minutes.

[0180] In one possible implementation, the atomic service function for achieving the target intent is used to activate the vehicle function for coping with the target situation, and the recommended atomic service function corresponding to the atomic service function for achieving the target intent is used to activate the alternative vehicle function for coping with the target situation. The UI component of the recommended atomic service function corresponding to the atomic service function for achieving the target intent includes: the name of the alternative vehicle function for coping with the target situation. The user can click on the UI component of the recommended atomic service function corresponding to the atomic service function for achieving the target intent or speak a voice indicating to activate the alternative vehicle function for coping with the target situation, and the UI component of the recommended atomic service function corresponding to the atomic service function for achieving the target intent triggers the call to the recommended atomic service function corresponding to the atomic service function for achieving the target intent to activate the alternative vehicle function for coping with the target situation.

[0181] For each of a plurality of situations encountered by the target vehicle, a vehicle function for coping with the situation and an alternative vehicle function for coping with the situation may be preset.

[0182] refer to Fig.14 , which shows a schematic diagram showing the effects of vehicle functions that have been turned on and alternative vehicle functions.

[0183] As an example, the user says the interactive voice "It's snowing heavily, what should I do", the target situation encountered by the vehicle is "snowing", and the atomic service function used to achieve the target intention is the atomic service function for turning on the vehicle function for dealing with "snowing". Call the atomic service function for turning on the rearview mirror heating for dealing with "snowing", call the atomic service function for turning on the rearview mirror heating for dealing with "snowing", call the atomic service function for turning on the steering wheel heating for dealing with "snowing", call the atomic service function for turning on the automatic wiper for dealing with "snowing", and call the atomic service function for turning on the driver's seat heating for dealing with "snowing". Thus, turn on the vehicle functions for dealing with "snowing", such as rearview mirror heating, steering wheel heating, automatic wiper, and driver's seat heating. The alternative vehicle function for dealing with "snowing" is defrosting. The recommended atomic service function is the atomic service function for turning on defrosting, an alternative vehicle function for dealing with "snowing". Fig.14 The UI component for recommending atomic service functions is shown, and the UI component for recommending atomic service functions includes the name of the alternative vehicle function for coping with “snowing”, namely, defrost.

[0184] In an embodiment of the present invention, it is considered that when the vehicle encounters a target situation, the user is unclear about which vehicle function should be activated to deal with the target situation encountered by the vehicle. When the target vehicle encounters a target situation, the vehicle function to deal with the target situation can be automatically activated, and a UI component for activating an atomic service function corresponding to the alternative vehicle function to deal with the target situation can be displayed. Thus, the situation encountered by the target vehicle can be dealt with. At the same time, the user is made aware that an alternative vehicle function to deal with the target situation can also be activated. The user can activate the alternative vehicle function to deal with the target situation by clicking on the UI component of the recommended atomic service function corresponding to the atomic service function for achieving the target intention or by speaking a voice indicating to activate the alternative vehicle function to deal with the target situation, thereby improving the convenience of user interaction.

[0185] In one possible implementation, calling an atomic service function for realizing a target intent includes: when there are multiple target intents, calling an atomic service function corresponding to each target intent.

[0186] For one target intent among multiple target intents, the atomic service function corresponding to the target intent is used to realize the target intent.

[0187] As an example, the user says the interactive voice "play..., navigate to...". Two target intentions are determined, which are: the music APP on the target vehicle plays the "..." in "play..."; the navigation APP on the target vehicle generates a navigation route from the location of the target vehicle to the "..." in "navigate to...".

[0188] refer to Fig.15 , which shows a schematic diagram of an example of simultaneously turning on the atomic service functions corresponding to each of the multiple intents. In this example, the user says the interactive voice "turn on the seat massage and air conditioning". Two target intents are determined, namely: turn on the seat massage and turn on the air conditioning. Fig.15 The call result displayed in the dialogue service flow interface is shown, namely, "seat massage and air conditioning are both turned on."

[0189] In the embodiment of the present invention, the user can speak a voice indicating multiple intentions, and the multiple intentions can be realized at the same time, without the user having to speak a voice corresponding to each of the multiple intentions one by one, thereby simplifying the user's operation and improving the user's user experience.

[0190] In one possible implementation, when it is detected that a user-defined scenario trigger condition is met, an atomic service function is called to implement the intent corresponding to the scenario trigger condition.

[0191] As an example, the user-defined scene trigger condition is "a female passenger sits in the front passenger seat", and the intention corresponding to the scene trigger condition is "turn on the fragrance", and the atomic service function for turning on the fragrance function of the vehicle center APP of the target vehicle is called to turn on the fragrance function.

[0192] The user-defined scene trigger condition and the intention corresponding to the scene trigger condition can be determined according to the user's voice indicating the user-defined scene trigger condition and the intention corresponding to the scene trigger condition.

[0193] refer to Fig.16 , which shows a schematic diagram of an example of determining a scene trigger condition and an intention corresponding to the scene trigger condition.

[0194] The user's customized interactive scene operation and the voice of the interactive scene operation trigger condition corresponding to the customized interactive scene operation are "...after getting on the bus, broadcast "Happy birthday to...". The scene trigger condition is determined to be: "...get on the bus". The intention corresponding to the scene trigger condition is to broadcast "Happy birthday to...".

[0195] refer to Fig.17 , which shows a schematic diagram of the effect of calling an atomic service function to implement the intention corresponding to the scene trigger condition.

[0196] A user-defined scenario trigger condition is: User A gets on the vehicle. After User A enters the target vehicle, the target model determines that the scenario trigger condition of User A getting on the vehicle is met based on the identity information of User A and the action information of User A indicating the action of User A getting on the vehicle, and calls the voice playback atomic service function of the voice broadcast APP on the target vehicle to broadcast "Happy birthday to...".

[0197] In one possible implementation, in response to detecting an operation performed on the dialogue service flow interface to instruct reduction or enlargement of the size of an image frame, the ratio of the height of an adjusted image frame indicated by the operation performed on the dialogue service flow interface to instruct reduction or enlargement of the size of an image frame to the height of a display screen of a target vehicle is calculated; based on the ratio of the height of an adjusted image frame indicated by the operation performed on the dialogue service flow interface to instruct reduction or enlargement of the size of an image frame to the height of the display screen of the target vehicle, an original image frame including information to be displayed in the dialogue service flow interface is reduced or enlarged to obtain a target image frame including information to be displayed in the dialogue service flow interface, and the target image frame is displayed in the dialogue service flow interface.

[0198] If the ratio of the adjusted image frame height to the height of the target vehicle's display screen indicated by the operation indicating reducing or enlarging the size of the image frame is greater than the ratio of the height of the original image frame to the height of the target vehicle's display screen, the original image frame including the information that needs to be displayed in the dialogue service flow interface is enlarged.

[0199] If the ratio of the adjusted image frame height to the height of the target vehicle's display screen indicated by the operation indicating reducing or enlarging the size of the image frame is smaller than the ratio of the height of the original image frame to the height of the target vehicle's display screen, the original image frame including the information that needs to be displayed in the dialogue service flow interface is reduced.

[0200] The height of the target image frame including the information to be displayed in the dialogue service flow interface has an adjusted image frame height. The ratio of the width of the target image frame including the information to be displayed in the dialogue service flow interface to the width of the display screen of the target vehicle may be: the ratio of the adjusted image frame height of the operation indication indicating the reduction or enlargement of the size of the image frame to the height of the display screen of the target vehicle.

[0201] When the original image frame including the information that needs to be displayed in the dialogue service flow interface is reduced or enlarged according to the ratio to obtain the target image frame including the information that needs to be displayed in the dialogue service flow interface, the component for scaling the image frame at the system layer can be used to reduce or enlarge the original image frame including the information that needs to be displayed in the dialogue service flow interface according to the ratio to obtain the target image frame including the information that needs to be displayed in the dialogue service flow interface. In this way, there is no need to adapt to each APP on the target vehicle. As an example, the operating system running on the vehicle computer of the target vehicle is the Android system, and the DecorCaptionView component at the system layer is used to reduce or enlarge the original image frame including the information that needs to be displayed in the dialogue service flow interface according to the ratio to obtain the target image frame including the information that needs to be displayed in the dialogue service flow interface.

[0202] In an embodiment of the present invention, according to the ratio of the height of the adjusted image frame indicated by the operation to the height of the display screen of the target vehicle, the original image frame including the target information is reduced or enlarged to obtain a target image frame including the information to be displayed in the dialogue service flow interface, and the target image frame is displayed in the dialogue service flow interface. This can avoid the following situation in the related art: when adjusting the size of the image frame to be displayed, the module for displaying the image frame application layer notifies the module for adjusting the image frame of the kernel of the operating system, and the module for adjusting the image frame of the kernel of the operating system modifies the configuration item of the image frame size from the original size to the adjusted size. The overhead of adjusting the size of the image frame to be displayed is large. The notification and modification will cause the page to refresh during the scaling process, resulting in page jumping and the corresponding life cycle method being called back due to the page refresh, which will have an adverse effect on the performance of the vehicle computer.

[0203] An interactive device is also provided in an embodiment of the present invention, and the device is used to implement the above-mentioned method embodiment and preferred implementation mode, which have been described and will not be repeated here. As used below, the term "unit" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived. The device in the embodiment of the present invention is presented in the form of a functional unit, where the functional unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0204] An interactive device, the interactive device comprising:

[0205] an intention determination unit, for determining a target intention associated with the user based on multimodal information associated with the target vehicle;

[0206] The starting unit is used to start the dialogue service flow, and the dialogue service flow includes: calling an atomic service function for realizing the target intention.

[0207] Furthermore, the dialogue service flow also includes:

[0208] Determine the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent;

[0209] Information for guiding user interaction is provided to the user, and the information for guiding user interaction includes: call results corresponding to the atomic service functions for achieving the target intent, and information related to the recommended atomic service functions corresponding to the atomic service functions for achieving the target intent.

[0210] Further, providing information to the user for guiding user interaction includes:

[0211] Information for guiding user interaction is displayed in the dialog service flow interface, or a voice indicating an information item in the information for guiding user interaction is played.

[0212] Further, the recommended atomic service functions corresponding to the atomic service functions for achieving the target intent include:

[0213] According to the information indicating the user's usage habits for the atomic service functions, a recommended atomic service function corresponding to the atomic service function for achieving the target intention is determined.

[0214] Furthermore, the dialogue service flow also includes:

[0215] After the information for guiding user interaction is provided to the user, in response to the user's operation on a corresponding information item in the information for guiding user interaction, a feedback operation corresponding to the corresponding information item is performed.

[0216] Furthermore, the recommended atomic service function is used to set parameter values ​​of working parameters of vehicle functions corresponding to the atomic service functions for achieving target intentions.

[0217] Further, the atomic service function for achieving the target intention is used to activate the vehicle function for coping with the target situation, and the recommended atomic service function is used to activate the alternative vehicle function for coping with the target situation.

[0218] Furthermore, the atomic service functions called to achieve the target intent include:

[0219] When there are multiple target intents, the atomic service function corresponding to each of the multiple target intents is called, wherein the atomic service function corresponding to the target intent is used to realize the target intent.

[0220] Furthermore, the atomic service functions called to achieve the target intent include:

[0221] For each target intent in at least one target intent, an instruction corresponding to the target intent is sent to an application to which the atomic service function for implementing the target intent belongs in at least one application corresponding to the at least one target intent, so that the application calls the atomic service function for implementing the target intent according to the instruction.

[0222] Furthermore, the intention determination unit is further used to utilize the target model on the cloud server to determine the target intention based on the multimodal information related to the target vehicle.

[0223] Furthermore, the interactive device also includes:

[0224] The scene trigger unit is used to call an atomic service function for realizing the intention corresponding to the scene trigger condition when it detects that the scene trigger condition defined by the user is met.

[0225] Furthermore, the interactive device also includes:

[0226] a scaling unit, for, in response to detecting an operation performed on the dialogue service flow interface indicating reduction or enlargement of the size of the image frame, calculating a ratio of a height of the image frame adjusted as indicated by the operation to a height of a display screen of the target vehicle;

[0227] According to the ratio, the original image frame including the information to be displayed in the dialogue service flow interface is reduced or enlarged to obtain the target image frame, and the target image frame is displayed in the dialogue service flow interface.

[0228] refer to Fig.18 , Fig.181 is a schematic diagram of the hardware structure of a computer device provided by an embodiment of the present invention. The computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed in the computer device, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used with multiple memories and multiple memories. Similarly, multiple vehicles can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). The processor 10 can be a central processing unit, a network processor, or a combination thereof. Among them, the processor 10 can further include a hardware chip. The above-mentioned hardware chip can be a dedicated integrated circuit, a programmable logic device, or a combination thereof. The above-mentioned programmable logic device can be a complex programmable logic device, a field programmable gate array, a general array logic, or any combination thereof. The memory 20 stores instructions executable by at least one processor 10 so that the at least one processor 10 executes the method shown in the above embodiment. The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required by at least one function; the data storage area may store data created according to the use of the vehicle, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof. The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state hard disk; the memory 20 may also include a combination of the above-mentioned types of memory. The computer device also includes an input device 30 and an output device 40. The processor 10, the memory 20, the input device 30, and the output device 40 may be connected via a bus or in other ways. The input device 30 can receive input digital or character information, and generate key signal input related to the user settings and function control of the computer device, such as a touch screen, a keypad, a mouse, a trackpad, a touch pad, an indicator stick, one or more mouse buttons, a trackball, a joystick, etc.The output device 40 may include a display device, an auxiliary lighting device (e.g., LED), a tactile feedback device (e.g., a vibration motor), etc. The above-mentioned display device includes but is not limited to a liquid crystal display, a light emitting diode, a display, and a plasma display. In some optional embodiments, the display device may be a touch screen.

[0229] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0230] A portion of the embodiments of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of computer program instructions in a computer-readable medium includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the way in which the computer program instructions are executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0231] The above embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Any equivalent substitution or change made by a person skilled in the art on the basis of the present invention is within the protection scope of the present invention.

Claims

1. An interactive method, characterized in that: The method comprises: Determine the target intention associated with the user based on the multimodal information associated with the target vehicle; Start a dialog service flow, which includes calling an atomic service function for realizing a target intent.

2. The method according to claim 1, characterized in that: The conversation service flow also includes: Determine the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent; Information for guiding user interaction is provided to the user, and the information for guiding user interaction includes: call results corresponding to the atomic service functions for achieving the target intent, and information related to the recommended atomic service functions corresponding to the atomic service functions for achieving the target intent.

3. The method according to claim 2, characterized in that: Information provided to users to guide user interaction includes: Information for guiding user interaction is displayed in the dialog service flow interface, or a voice indicating an information item in the information for guiding user interaction is played.

4. The method according to claim 2, characterized in that: Determine the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent; According to the information indicating the user's usage habits for the atomic service functions, a recommended atomic service function corresponding to the atomic service function for achieving the target intention is determined.

5. The method according to claim 2, characterized in that: The conversation service flow also includes: After the information for guiding user interaction is provided to the user, in response to the user's operation on a corresponding information item in the information for guiding user interaction, a feedback operation corresponding to the corresponding information item is performed.

6. The method according to claim 2, characterized in that: The recommended atomic service function is used to set the parameter value of the working parameter of the vehicle function corresponding to the atomic service function used to achieve the target intention.

7. The method according to claim 2, characterized in that: The atomic service function for achieving the target intention is used to activate the vehicle function for coping with the target situation, and the recommended atomic service function is used to activate the alternative vehicle function for coping with the target situation.

8. The method according to claim 1, characterized in that: The atomic service functions called to achieve the target intent include: When there are multiple target intents, the atomic service function corresponding to each of the multiple target intents is called, wherein the atomic service function corresponding to the target intent is used to realize the target intent.

9. The method according to claim 1, characterized in that: The atomic service functions called to achieve the target intent include: For each target intent in at least one target intent, an instruction corresponding to the target intent is sent to the application to which the atomic service function for implementing the target intent belongs in at least one application corresponding to the at least one target intent, so that the application to which the atomic service function for implementing the target intent belongs calls the atomic service function for implementing the target intent according to the instruction.

10. The method according to claim 1, characterized in that: Based on the multimodal information related to the target vehicle, the target intention is determined including: The target model on the cloud server is used to determine the target intention based on the multimodal information related to the target vehicle.

11. The method according to claim 1, characterized in that: The method further comprises: When it is detected that the user-defined scene trigger condition is met, the atomic service function used to implement the intention corresponding to the scene trigger condition is called.

12. The method according to claim 1, characterized in that: The method further comprises: In response to detecting an operation performed on the dialogue service flow interface indicating reduction or enlargement of the size of the image frame, calculating a ratio of a height of the image frame adjusted as indicated by the operation to a height of a display screen of the target vehicle; According to the ratio, the original image frame including the information to be displayed in the dialogue service flow interface is reduced or enlarged to obtain a target image frame including the information to be displayed in the dialogue service flow interface, and the target image frame is displayed in the dialogue service flow interface.

13. An interactive device, characterized in that: The device comprises: an intention determination unit, for determining a target intention associated with the user based on multimodal information associated with the target vehicle; The starting unit is used to start the dialogue service flow, and the dialogue service flow includes: calling an atomic service function for realizing the target intention.

14. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 12 by executing the computer instructions.

15. A vehicle, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 12 by executing the computer instructions.

16. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 12.

17. A computer program product, characterized in that The method comprises computer instructions for causing a computer to execute the method according to any one of claims 1 to 12.

Images