Interaction method and apparatus, and computer device, vehicle, storage medium and program product

Abstract

The present application relates to an interaction method and apparatus, and a computer device and a storage medium. The method comprises: on the basis of multi-modal information related to a target vehicle, determining a target intent related to a user; and starting a dialog service flow, wherein the dialog service flow comprises: invoking an atomic service function for implementing the target intent.

Description

An interactive method, apparatus, computer equipment, vehicle, storage medium, and program product.

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese Patent Application No. 202411635904.1, filed on November 15, 2024, entitled "An Interactive Method, Apparatus, Computer Equipment and Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of vehicles, specifically to an interactive method, apparatus, computer equipment vehicle, storage medium, and program product. Background Technology

[0004] In related technologies, when a user intends for the vehicle to perform a certain operation, the user needs to perform a series of actions on the vehicle's infotainment screen, such as opening the app associated with the operation and opening the relevant page within that app, before the operation can be triggered. This cumbersome process results in poor convenience for the interaction between the vehicle and the user. Improving the convenience of vehicle-user interaction is a problem that needs to be addressed. Summary of the Invention

[0005] One of the purposes of this application is to provide an interaction method, device, computer equipment, vehicle, storage medium, and program product to address the problem of how to improve the convenience of interaction between a vehicle and its users.

[0006] To achieve the above objectives, the technical solution adopted in this application is as follows:

[0007] An interaction method, comprising:

[0008] Based on multimodal information related to the target vehicle, determine the target intent related to the user;

[0009] Initiate a dialogue service flow, which includes invoking atomic service functions used to achieve the target intent.

[0010] Based on the aforementioned technical means, and using multimodal information related to the target vehicle, the target intent related to the user is determined. A dialogue service flow is then initiated, which includes calling atomic service functions to realize the target intent. This allows the user to achieve the intended action without needing to perform a series of operations on the vehicle's screen, simplifying the user's operation and improving the convenience of vehicle-user interaction. Simultaneously, multimodal information related to the target vehicle is acquired, and the target intent is determined based on the target model and the multimodal information related to the target vehicle. The multimodal information related to the target vehicle has a strong correlation with the user's intent. The determined target intent is highly accurate, and the atomic service functions are invoked relatively accurately.

[0011] Optionally, the dialogue service flow also includes:

[0012] Determine the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent;

[0013] Provide users with information to guide user interaction. This information includes: the call results corresponding to the atomic service functions used to achieve the target intent, and information related to the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent.

[0014] Optionally, providing users with information to guide user interaction includes:

[0015] Display information that guides user interaction or play voice prompts for information items in the dialogue service flow interface.

[0016] Optionally, the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent include:

[0017] Based on information about users' usage habits of atomic service functions, determine the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent.

[0018] Optionally, the dialogue service flow also includes:

[0019] After providing the user with information to guide user interaction, in response to the user's operation on the corresponding information item in the information to guide user interaction, a feedback operation corresponding to the corresponding information item is executed.

[0020] Optionally, the recommended atomic service function is used to set the parameter values ​​of the working parameters of the vehicle function corresponding to the atomic service function used to achieve the target intent.

[0021] Optionally, the atomic service function for achieving the target intent is used to enable the vehicle function for responding to the target situation, and the recommended atomic service function is used to enable the alternative vehicle function for responding to the target situation.

[0022] Optionally, invoking atomic service functions to achieve the target intent includes:

[0023] When there are multiple target intents, the atomic service function corresponding to each target intent is invoked, wherein the atomic service function corresponding to the target intent is used to implement the target intent.

[0024] Optionally, invoking atomic service functions to achieve the target intent includes:

[0025] For each of the at least one target intent, an instruction corresponding to the target intent is sent to the application to which the atomic service function for implementing the target intent belongs in at least one application corresponding to the at least one target intent, so that the application invokes the atomic service function for implementing the target intent according to the instruction.

[0026] Optionally, determining the target intent based on multimodal information related to the target vehicle includes:

[0027] By utilizing the target model on the cloud server, the target intent is determined based on multimodal information related to the target vehicle.

[0028] Optionally, it also includes:

[0029] When a user-defined scene trigger condition is detected, an atomic service function is invoked to implement the intent corresponding to the scene trigger condition.

[0030] Optionally, it also includes:

[0031] In response to detecting an operation on the dialogue service flow interface that instructs the user to zoom in or out on the size of an image frame, the ratio of the adjusted image frame height indicated by the operation to the height of the target vehicle's display screen is calculated.

[0032] Based on the stated ratio, the original image frame, including the information to be displayed in the dialog service flow interface, is reduced or enlarged to obtain the target image frame, and the target image frame is displayed in the dialog service flow interface.

[0033] An interactive device, the interactive device comprising:

[0034] The intent determination unit is used to determine the target intent related to the user based on multimodal information related to the target vehicle;

[0035] The startup unit is used to initiate a dialogue service flow, which includes invoking atomic service functions used to achieve the target intent.

[0036] Optionally, the dialogue service flow also includes:

[0037] Determine the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent;

[0038] Provide users with information to guide user interaction. This information includes: the call results corresponding to the atomic service functions used to achieve the target intent, and information related to the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent.

[0039] Optionally, providing users with information to guide user interaction includes:

[0040] Display information that guides user interaction or play voice prompts for information items in the dialogue service flow interface.

[0041] Optionally, the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent include:

[0042] Based on information about users' usage habits of atomic service functions, determine the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent.

[0043] Optionally, the dialogue service flow also includes:

[0044] After providing the user with information to guide user interaction, in response to the user's operation on the corresponding information item in the information to guide user interaction, a feedback operation corresponding to the corresponding information item is executed.

[0045] Optionally, the recommended atomic service function is used to set the parameter values ​​of the working parameters of the vehicle function corresponding to the atomic service function used to achieve the target intent.

[0046] Optionally, the atomic service function for achieving the target intent is used to enable the vehicle function for responding to the target situation, and the recommended atomic service function is used to enable the alternative vehicle function for responding to the target situation.

[0047] Optionally, invoking atomic service functions to achieve the target intent includes:

[0048] When there are multiple target intents, the atomic service function corresponding to each target intent is invoked, wherein the atomic service function corresponding to the target intent is used to implement the target intent.

[0049] Optionally, invoking atomic service functions to achieve the target intent includes:

[0050] For each of the at least one target intent, an instruction corresponding to the target intent is sent to the application to which the atomic service function for implementing the target intent belongs in at least one application corresponding to the at least one target intent, so that the application invokes the atomic service function for implementing the target intent according to the instruction.

[0051] Optionally, the intent determination unit is further configured to utilize the target model on the cloud server to determine the target intent based on multimodal information related to the target vehicle.

[0052] Optionally, the interactive device also includes:

[0053] The scene triggering unit is used to invoke atomic service functions that implement the intent corresponding to the scene triggering conditions when a user-defined scene triggering condition is detected.

[0054] Optionally, the interactive device also includes:

[0055] A scaling unit is configured to, in response to detecting an operation on the dialogue service flow interface indicating to zoom in or out of an image frame, calculate the ratio of the adjusted image frame height indicated by the operation to the height of the target vehicle's display screen.

[0056] Based on the stated ratio, the original image frame, including the information to be displayed in the dialog service flow interface, is reduced or enlarged to obtain the target image frame, and the target image frame is displayed in the dialog service flow interface.

[0057] A computer device, comprising:

[0058] The system includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the above-described method.

[0059] A computer-readable storage medium storing computer instructions for causing a computer to perform the above-described method.

[0060] A vehicle comprising:

[0061] The system includes a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the above-described method.

[0062] A computer-readable storage medium storing computer instructions for causing a computer to perform the above-described method.

[0063] A computer program product, characterized in that it includes computer instructions, the computer instructions being used to cause a computer to perform the above-described method.

[0064] The beneficial effects of this application are:

[0065] Based on multimodal information related to the target vehicle, the target intent related to the user is determined. A dialogue service flow is then initiated, which includes calling atomic service functions to realize the target intent. This allows the user to achieve the intended action without needing to perform a series of operations on the vehicle's screen, simplifying the user experience and improving the convenience of vehicle-user interaction. Simultaneously, multimodal information related to the target vehicle is acquired, and the target intent is determined based on the target model and the multimodal information related to the target vehicle. The multimodal information related to the target vehicle has a strong correlation with the user's intent. The determined target intent is highly accurate, and the atomic service functions are invoked precisely. Attached Figure Description

[0066] Figure 1 is a schematic diagram of an example architecture of an interactive system that can be used to execute the interactive methods provided in the embodiments of this application;

[0067] Figure 2 is a flowchart illustrating the interaction method provided in an embodiment of this application;

[0068] Figure 3 is a schematic diagram illustrating an example of determining target intent;

[0069] Figure 4 is a flowchart of an example of determining target intent;

[0070] Figure 5 is a flowchart of an example of determining the target intent related to the navigation APP on the target vehicle;

[0071] Figure 6 is a flowchart of an example of determining target intent related to a music app on a target vehicle;

[0072] Figure 7 is a schematic diagram of the effect of calling the corresponding atomic service function of the music APP on the target vehicle to achieve the target intent;

[0073] Figure 8 is a schematic diagram illustrating the effect of calling the corresponding atomic service function of the music APP on the target vehicle to achieve the target intent;

[0074] Figure 9 is a schematic diagram illustrating the effect of calling the corresponding atomic service function of the navigation APP on the target vehicle to achieve the target intent;

[0075] Figure 10 is a schematic diagram of the effect of calling the corresponding atomic service function of the navigation APP on the target vehicle to achieve the target intent;

[0076] Figure 11 is a flowchart illustrating an example of the interaction between the target vehicle and the user;

[0077] Figure 12 is a schematic diagram of an example of recommended songs and sound effects;

[0078] Figure 13 is a schematic diagram of the UI components that show the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent;

[0079] Figure 14 is a schematic diagram showing the effect of the activated vehicle functions and the alternative vehicle functions;

[0080] Figure 15 is a schematic diagram of an example of simultaneously enabling the atomic service function corresponding to each of multiple intents;

[0081] Figure 16 is a schematic diagram of an example of determining scene triggering conditions and the intent corresponding to the scene triggering conditions;

[0082] Figure 17 is a schematic diagram of the effect of calling an atomic service function to realize the intent corresponding to the scene triggering condition;

[0083] Figure 18 is a schematic diagram of the hardware structure of the computer device provided in the embodiment of this application. Detailed Implementation

[0084] The embodiments of this application will be described below with reference to the accompanying drawings and preferred embodiments. Users skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be understood that the preferred embodiments are only for illustrating this application and are not intended to limit the scope of protection of this application.

[0085] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0086] Referring to Figure 1, it shows a schematic diagram of an example architecture of an interactive system that can be used to perform the interactive methods provided in the embodiments of this application.

[0087] An example architecture of an interactive system that can be used to execute the interactive methods provided in the embodiments of this application includes: a data acquisition unit, a data processing unit, and an atomic service function unit.

[0088] The data acquisition unit collects voice data from user interactions, images of the target vehicle's internal and external environments, and sensor data from the target vehicle. The data processing unit includes a voice data processing module, an image processing module, a sensor data processing module, an encoding module, and a target model. Atomized service functions include the atomic service functions of each app on the target vehicle that possesses atomic service capabilities. An atomic service function can be understood as the smallest functional unit, capable of performing a single operation.

[0089] The voice data processing module can perform speech recognition on the user's interactive speech to obtain the speech recognition results. The voice data processing unit can use a sound source localization algorithm to determine the user's location within the target vehicle. The voice data processing unit can also identify the user by matching the user's interactive voiceprint features with preset voiceprint features. The preset voiceprint features are those of users whose voiceprint features have been previously recorded.

[0090] The image processing module can process images of the interior environment of the target vehicle to determine the user's age group, gender, seating position, and identity information. The image processing unit may include a Driver Monitoring System (DMS) and an Occupant Monitoring System (OMS). The Driver Monitoring System monitors the driver's gaze direction in real time. The Occupant Monitoring System can process images of the vehicle's interior environment captured by the vehicle's cameras to determine the passengers' gender, age, gestures, and seating position.

[0091] The sensor data processing module can process sensor data collected by the target vehicle's sensors to determine information related to the target vehicle's driving and weather-related information about the target vehicle's location. Driving-related information may include the target vehicle's speed, remaining fuel level, or remaining battery charge. Weather-related information about the target vehicle's location may include the temperature and weather type, such as rain or snow.

[0092] The encoding module is used to encode the multimodal information related to the target vehicle to obtain a representation of the multimodal information related to the target vehicle.

[0093] It should be noted that the target model can be on the target vehicle or on a cloud server.

[0094] The target model is used to receive representations of multimodal information related to the target vehicle and predict the representation of the target intent. Thus, the target intent is determined. The target model is a trained neural network. Before the interaction method provided in this application embodiment is executed for the first time, the target model is trained using training samples and annotation operations on the training samples.

[0095] As an example, the target model is a pre-trained language model. As another example, the pre-trained language model used as the target model is a Large Language Model (LLM). When the input to the target model is natural language text corresponding to natural language speech, the pre-trained language model performs Natural Language Processing (NLP) tasks.

[0096] Training the target model allows it to learn: which information from the target model's input to predict actions that satisfy the user's interactive voice instructions, and how to determine the target intent. Training samples for the target model can include training information related to the multimodal information relevant to the target vehicle. For any given piece of information, if the multimodal information relevant to the target vehicle includes that information, then that information is classified as belonging to the multimodal information relevant to the target vehicle. The labeled operations of the target model's training samples can be understood as: the expected output operations of the target model when the training samples are input. During training, the loss between the operations predicted by the target model and the labeled operations of the training samples can be calculated, and the parameters of the target model can be updated based on this loss.

[0097] Referring to Figure 2, a flowchart illustrating the interaction method provided in an embodiment of this application is shown. This method can be executed by a computer device, such as an in-vehicle terminal of the target vehicle or a mobile terminal.

[0098] Step S201: Determine the target intent based on the multimodal information related to the target vehicle.

[0099] The multimodal information related to the target vehicle includes: the speech recognition results of the user's interactive voice, the parameter information of the target vehicle, the external environment information of the target vehicle, the user's action information, the user's seating location information, the user's attribute information, the user's identity information, information related to the APP on the target vehicle determined during the historical time period of using the APP on the target vehicle, navigation information related to the target vehicle, music information related to the target vehicle, and point of interest information related to the target vehicle. Among them, the point of interest information related to the target vehicle indicates the point of interest reached by the target vehicle during the target historical time period and / or the point of interest reached by the target vehicle multiple times.

[0100] The speech recognition result of the user's interactive voice is obtained by performing speech recognition on the user's interactive voice. Specifically, performing speech recognition on the user's interactive voice refers to performing speech recognition on the voice data of the user's interactive voice.

[0101] As an example, the parameter information of the target vehicle may include: information related to the driving of the target vehicle and information related to the cockpit of the target vehicle.

[0102] As an example, information related to the driving of the target vehicle includes: the target vehicle's speed, the target vehicle's remaining fuel, or the target vehicle's remaining battery charge.

[0103] As an example, information related to the target vehicle's cabin includes: the brightness of the vehicle's infotainment screen, the volume of the vehicle's audio system, the temperature of the vehicle's air conditioning, the airflow of the vehicle's air conditioning, and the angle of the vehicle's seats.

[0104] The external environment information of the target vehicle refers to information related to the external environment of the target vehicle.

[0105] As an example, the external environment information of the target vehicle includes: weather-related information about the area where the target vehicle is located, driving safety-related information about the road the target vehicle is currently on, and points of interest (POIs) near the target vehicle.

[0106] As an example, weather-related information for the area where the target vehicle is located includes the temperature, the type of weather, etc.

[0107] As an example, information related to the driving safety of the target vehicle on the road it is currently on includes one or more of the following: information indicating whether the road is slippery, information indicating whether the road is icy, information indicating whether the road is bumpy, information indicating whether the road is a high-altitude road, etc.

[0108] As an example, user action information includes at least some of the following: user gestures, the direction the user's hand is pointing, the direction the user's gaze is in, and the user's actions.

[0109] As an example, user attribute information includes: the user's age group, the user's gender, etc.

[0110] User identity information is used to identify the user's identity.

[0111] As an example, a user's identity information includes: the user's identifier.

[0112] The user's seating position information indicates the user's seating location. As an example, the user's seating position is one of the following: driver's seat, front passenger seat, rear left seat, rear middle seat, or rear right seat.

[0113] Navigation information related to the target vehicle may include: the origin of the current navigation to the target vehicle, the destination of the current navigation to the target vehicle, and the navigation route of the current navigation to the target vehicle.

[0114] Music information related to the target vehicle may include: the playlist to which the currently playing song belongs, and the playback progress of the song in that playlist. The playback progress indicates which song has been played so far.

[0115] The end time of the historical period in which the app on the target vehicle was used is before the current time. The app on the target vehicle was used within the historical period in which the app was used.

[0116] As an example, for a navigation app on a target vehicle, the multimodal information related to the target vehicle includes: information related to the navigation app determined during the historical time period of its use. This information includes: the navigation route determined during the historical time period of the app's use, the name of the starting point of the navigation route determined during the historical time period of the app's use, and the name of the destination of the navigation route determined during the historical time period of the app's use.

[0117] As an example, for a music app on a target vehicle, the multimodal information related to the target vehicle includes: information related to the music app on the target vehicle determined during the historical time period of using the music app on the target vehicle. Information related to the music app on the target vehicle includes: identifiers of identified recommended songs.

[0118] Referring to Figure 3, which shows a schematic diagram of an example of determining a target intent.

[0119] The user speaks an interactive voice command, triggering the target vehicle to acquire multimodal information related to the vehicle. The user's interactive voice command is the voice used for interaction. This voice command can be natural language speech, reflecting the user's desired action from the target vehicle. To acquire this multimodal information, the system acquires data collected by the target vehicle, including the user's interactive voice command, sensor data, images of the target vehicle's internal environment, and images of the target vehicle's external environment. The data collected by the target vehicle is processed to obtain the multimodal information related to the target vehicle. This multimodal information includes at least some of the following: the speech recognition result of the user's interactive voice command, the target vehicle's parameter information, the target vehicle's external environment information, the user's action information, the user's attribute information, the user's identity information, and information related to the app on the target vehicle determined during the historical time period of app usage on the target vehicle.

[0120] Each piece of information in the multimodal information related to the target vehicle is encoded to obtain a representation of the multimodal information related to the target vehicle.

[0121] The representation of multimodal information related to the target vehicle is input into the target model, and the target model outputs the target intent.

[0122] In step S201, multimodal information related to the target vehicle can be input into the target model, and the target model can output the target intent.

[0123] The target model can be on the target vehicle or on a cloud server.

[0124] It should be noted that the target model can use a portion of the multimodal information related to the target vehicle to predict the target's intent.

[0125] As an example, the interior temperature of the target vehicle is high, and the user's hand is pointing towards the window and making a downward gesture, indicating the user's intention to open the window.

[0126] As an example, a user says the interactive voice command, "Play songs suitable for people of my age." The intended message is to play songs suitable for users in the age group to which this user belongs.

[0127] As an example, a user says the interactive voice command, "Move your seat back a little." The intended action is to move the user's seat back.

[0128] Referring to Figure 4, a flowchart of an example of determining target intent is shown.

[0129] The interior environment image of the target vehicle indicates the internal environment of the target vehicle. Processing the interior environment image of the target vehicle can yield information such as the user's age group and gestures. The exterior environment image of the target vehicle indicates the external environment of the target vehicle. Processing sensor data and the exterior environment image and sensor data yields information about the exterior environment of the target vehicle. The exterior environment information of the target vehicle can include: weather-related information, i.e., weather-related information about the region where the target vehicle is located; information related to vehicle driving safety, i.e., road-related information about the target vehicle's current location; and nearby points of interest (POIs). Processing the user's interactive voice data yields the user's seating location and the speech recognition results of the user's interactive voice.

[0130] It should be noted that the various information shown in Figure 4 represents the multimodal information related to the target vehicle that may be used to determine the target's intent. The target model can use a portion of the multimodal information related to the target vehicle to predict the target's intent. In other words, only the multimodal information related to the target vehicle is provided to the target model, and the target model decides which information from the multimodal information related to the target vehicle to use to determine the target's intent.

[0131] Each piece of information in the multimodal information related to the target vehicle is encoded to obtain a representation of the multimodal information related to the target vehicle.

[0132] The multimodal information related to the target vehicle is input into the target model, and the target model outputs a representation of the target intent. Thus, the target intent is determined.

[0133] Referring to Figure 5, it shows a flowchart of an example of determining the target intent associated with a navigation app on a target vehicle.

[0134] The user speaks the interactive voice command, "What kind of community is that?". The speech recognition result of the user's interactive voice command, obtained from the multimodal information related to the target vehicle, is "What kind of community is that?". The external environment information of the target vehicle in the multimodal information related to the target vehicle includes: multiple points of interest (POIs) near the target vehicle, and the supporting facilities for those POIs. The user's action information in the multimodal information related to the target vehicle includes: the direction the user's hand is pointing. The target intent is: to find the name of the POI with the type "community" in the direction the user is pointing from the multiple POIs near the target vehicle.

[0135] Referring to Figure 6, it shows a flowchart of an example of determining a target intent related to a music app on a target vehicle.

[0136] When the target vehicle is traveling along the navigation route provided in the navigation information associated with the target vehicle, the user utters the interactive voice command "Play a song." Multiple users inside the target vehicle, including children, are processed. The interior image of the target vehicle is processed to determine the age group to which the users belong. The destination in the navigation information associated with the target vehicle is an amusement park. Based on the children's age group and the destination (amusement park), the target model determines the target intent: to play cheerful children's songs.

[0137] In step S202, a dialogue service flow is initiated, which includes invoking atomic service functions used to achieve the target intent.

[0138] In the application embodiment, each APP with atomic service function on the target vehicle can have multiple atomic service functions.

[0139] As an example, the vehicle center app on the target vehicle has atomic service functions such as an atomic service function for opening windows and an atomic service function for closing windows. The air conditioning app on the target vehicle has atomic service functions such as an atomic service function for turning on the air conditioning, an atomic service function for turning off the air conditioning, and an atomic service function for adjusting the air conditioning temperature.

[0140] Once the target intent is determined, the dialogue service flow can be initiated, invoking the atomic service functions used to achieve the target intent. After invoking the atomic service functions used to achieve the target intent, the invocation results corresponding to the atomic service functions used to achieve the target intent can be displayed in the dialogue service flow interface, which is displayed on the in-vehicle infotainment screen of the target vehicle.

[0141] The result of an atomic service function used to achieve a target intent is either the result obtained through that atomic service function or an indication that the target intent has been completed. For example, if the atomic service function used to achieve the target intent is an "Air Conditioner Turn On" atomic service function in the air conditioning app of the target vehicle, then the result of the call to that atomic service function is a text indicating that the air conditioning has been turned on. If the atomic service function used to achieve the target intent is an atomic service function for searching POIs in the search app of the target vehicle, then the result of the call to that atomic service function is the search result obtained by calling the atomic service function for searching POIs in the search app of the target vehicle.

[0142] As an example, a user becomes interested in a neighborhood the target vehicle is passing through while it is driving. The user says, "What neighborhood is ahead?" The target intent is to search for the name of the neighborhood ahead of the target vehicle. The search app on the target vehicle uses its atomic service function for searching Points of Interest (POIs) to search for the name of the neighborhood ahead. The user then says, "How much does it cost to rent a three-bedroom apartment?" The target intent is to search for the rent of a three-bedroom apartment in the neighborhood ahead of the target vehicle. The real estate agency app on the target vehicle uses its atomic service function for searching rent to search for the rent of a three-bedroom apartment in the neighborhood ahead. Finally, the user says, "Call to inquire about the situation." The target intent is to call the relevant personnel on the real estate transaction platform. The Bluetooth phone app uses its atomic call function to call the relevant personnel on the real estate transaction platform.

[0143] As an example, a user speaks a voice command related to the music app on the target vehicle. The target model determines the target intent related to the music app on the target vehicle and invokes the corresponding atomic service function of the music app on the target vehicle to realize the target intent.

[0144] Referring to Figure 7, it shows a schematic diagram of the effect of calling the corresponding atomic service function of the music APP on the target vehicle to achieve the target intent.

[0145] When the user interacts with the target vehicle's interaction system for the t-th time, the user says the interactive voice prompt "Which song is the most famous for '…'?" The target intent is to search for the name of the most famous song for "…". A dialogue service flow is initiated, calling the atomic service function of the music app on the target vehicle to search for the name of the most famous song for "…" from the knowledge base. When the user interacts with the target vehicle's interaction system for the t+1-th time, the user says the interactive voice prompt "What is the background of its creation?" The target intent is to search for the background of the most famous song for "…". A dialogue service flow is initiated, calling the atomic service function of the music app on the target vehicle to search for background music knowledge to search for the background of the most famous song for "…" from the knowledge base. When the user interacts with the target vehicle's interaction system for the t+2-th time, the user says the interactive voice prompt "In which album?" The target intent is to search for the album to which the song belongs. A dialogue service flow is initiated, calling the atomic service function of the music app on the target vehicle to search for the album to which the song belongs to search for the name of the album to which the most famous song for "…" belongs from the knowledge base. When the user inside the target vehicle interacts with the vehicle's interactive system for the (t+2)th time, the user says, "What other songs are in this album?" The intended purpose is to find the names of all songs in the album except for the hit song "...". A dialogue service flow is initiated, calling the atomic service function of the music app on the target vehicle to search for the names of all songs in the album except for the hit song "...". When the user inside the target vehicle interacts with the vehicle's interactive system for the (t+3)th time, the user says, "Add these songs to the playlist." The intended purpose is to add the other songs to the playlist.

[0146] Initiate a dialogue service flow and call the atomic service function of the music app on the target vehicle to add other songs to the playlist.

[0147] Referring to Figure 8, it shows a schematic diagram of the effect of calling the corresponding atomic service function of the music APP on the target vehicle to achieve the target intent.

[0148] The objective is to recommend a second song after the first song in each of the currently recommended songs has finished playing. A dialogue service flow is initiated, invoking the atomic service function of the music app on the target vehicle to search for the recommended second song. This second song is different from the first song. The dialogue service flow interface displays "The recommended songs have finished playing, and some new songs have been recommended for you." The "recommended songs" in this message refer to the first song. A new song identifier is displayed on the dialogue service flow interface. This new song refers to the second song.

[0149] As an example, a user speaks a voice command related to the navigation app on the target vehicle. The target model determines the target intent related to the navigation app on the target vehicle and invokes the corresponding atomic service function of the navigation app on the target vehicle to realize the target intent.

[0150] Referring to Figure 9, it shows a schematic diagram of the effect of calling the corresponding atomic service function of the navigation APP on the target vehicle to achieve the target intent.

[0151] When the user interacts with the target vehicle's interaction system for the tth time, the user says the interactive voice command "Go to... subway station to pick someone up, then go to..., find a place to eat," thus determining two target intentions: generating a navigation route through... subway station and the "..." in "then go to...", and searching for multiple restaurants near the "..." in "then go to...". A dialogue service flow is initiated, calling the atomic service function of the navigation app on the target vehicle to generate the navigation route through... subway station and the "..." in "then go to...". The atomic service function of the search app on the target vehicle to search for POIs is also called to search for multiple restaurants near the "..." in "then go to...". When the user interacts with the target vehicle's interaction system for the (t+1)th time, the user says, "Are there any hot pot restaurants?" The user's intention is to find a hot pot restaurant from multiple restaurants. This initiates a dialogue service flow, calling the atomic service function of the search app on the target vehicle for searching Points of Interest (POIs). During this (t+1)th interaction, the dialogue service flow interface displays, "These 5 hot pot restaurants are all pretty good." When the user interacts with the target vehicle's interaction system for the (t+2)th time, the user says, "Choose one with good parking." This initiates a dialogue service flow, calling the atomic service function of the search app on the target vehicle for searching POIs. Based on the surrounding environment information of the target vehicle, the atomic service function of the search app finds a hot pot restaurant with good parking from the 5 restaurants. When the user interacts with the target vehicle's interaction system for the t+3rd time, the user says the interactive voice "Just select...". The target intent is to navigate to the "..." in "Just select...". The dialogue service flow is started, and the atomic service function of the navigation APP on the target vehicle is called to generate a navigation route from the target vehicle's current location to the "..." in "Just select...".

[0152] Referring to Figure 10, it shows a schematic diagram of the effect of calling the corresponding atomic service function of the navigation APP on the target vehicle to achieve the target intent.

[0153] The user says, "Navigate to the barbecue restaurant I visited last week." The target intent is to generate a navigation route from the target vehicle's current location to the barbecue restaurant the user visited last week. This initiates a dialogue service flow, calling the atomic service function of the search app on the target vehicle for searching POIs, searching for the barbecue restaurants the user visited last week, and then calling the atomic service function used to guide the user's speech to display the information "I visited two barbecue restaurants last week, which one should I go to?" The user says, "Go to the closest one." This calls the atomic service function of the navigation app on the target vehicle to generate a navigation route from the target vehicle's current location to the barbecue restaurant closest to the target vehicle of the two barbecue restaurants visited last week, and displays this navigation route in the dialogue service flow interface. Additionally, the dialogue service flow interface also displays "Okay,...the closest one, navigating to...".

[0154] In one possible implementation, invoking the atomic service function for implementing the target intent includes: for each target intent in at least one target intent, sending an instruction corresponding to the target intent to the APP to which the atomic service function for implementing the target intent belongs in at least one application corresponding to at least one target intent, so that the APP to which the atomic service function for implementing the target intent belongs invokes the atomic service function for implementing the target intent according to the instruction corresponding to the target intent.

[0155] Wherein, at least one application corresponding to the at least one target intent is used to implement the at least one target intent.

[0156] For a given target intent, the instructions corresponding to that target intent may include: the ID of the atomic service function used to implement the target intent. The instructions corresponding to that target intent may also include: the data required to implement the target intent.

[0157] In one possible implementation, the method further includes: determining the recommended atomic service function corresponding to the atomic service function used to achieve the target intent; and providing the user with information to guide user interaction, the information including: the call result corresponding to the atomic service function used to achieve the target intent, and information related to the recommended atomic service function corresponding to the atomic service function used to achieve the target intent.

[0158] As an example, the target intent is to recommend songs. The atomic service function used to implement this intent is the song search service function of the music app on the target vehicle. A dialog service flow is initiated, calling the song search service function of the music app on the target vehicle to search for recommended songs. The corresponding recommendation atomic service function used to implement the target intent is the music app's atomic service function for setting sound effects for the recommended songs.

[0159] To determine the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent, the relationships between atomic service functions can be pre-defined. Other atomic service functions related to the atomic service function used to achieve the target intent are then identified as the recommended atomic service functions corresponding to the atomic service function used to achieve the target intent. Alternatively, the target model can predict the recommended atomic service functions corresponding to the atomic service function used to achieve the target intent based on the relationships between atomic service functions.

[0160] In one possible implementation, determining the recommended atomic service function corresponding to the atomic service function used to achieve the target intent includes: determining the recommended atomic service function corresponding to the atomic service function used to achieve the target intent based on information indicating the user's usage habits of atomic service functions.

[0161] Information indicating user usage habits for atomic service functions: atomic service functions invoked due to user-related target intents. Using the target model, based on information indicating user usage habits for atomic service functions, recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent can be determined.

[0162] In one possible implementation, providing information to the user to guide user interaction includes: displaying information to guide user interaction in the dialogue service flow interface or playing voice prompts for information items in the information to guide user interaction.

[0163] Among them, the call result corresponding to the atomic service function used to achieve the target intent is used as an information item in the information used to guide user interaction, and the information related to each recommended atomic service function corresponding to the atomic service function used to achieve the target intent is used as an information item in the information used to guide user interaction.

[0164] When information guiding user interaction is displayed in the dialogue service flow interface, the interface also displays the call results corresponding to the atomic service functions used to achieve the target intent, and the user interface (UI) components of the recommended atomic service functions used to achieve the target intent.

[0165] The user interface component for the recommended atomic service function corresponding to the atomic service function used to achieve the target intent may include: the name of the recommended atomic service function corresponding to the atomic service function used to achieve the target intent, and parameters related to the recommended atomic service function corresponding to the atomic service function used to achieve the target intent.

[0166] When playing voice prompts to guide user interaction, the system also plays voice prompts to indicate the call results corresponding to the atomic service function that achieves the target intent, and voice prompts related to the recommended atomic service function that corresponds to the atomic service function that achieves the target intent.

[0167] In one possible implementation, the dialogue service flow further includes: after providing the user with information to guide user interaction, in response to the user's operation on the corresponding information item in the information to guide user interaction, performing a feedback operation corresponding to the user's operation on the corresponding information item in the information to guide user interaction.

[0168] In one possible implementation, the user's action could be clicking on a corresponding information item in the information used to guide user interaction or speaking a voice command for the corresponding information item in the information used to guide user interaction.

[0169] In one possible implementation, if a user's operation on a corresponding information item in the information used to guide user interaction is an operation on information related to the recommended atomic service function corresponding to the atomic service function used to achieve the target intent, the feedback operation corresponding to the corresponding information item is to call the recommended atomic service function corresponding to the atomic service function used to achieve the target intent.

[0170] As an example, the atomic service function used to achieve the target intent is the atomic service function for activating the seat massage function in a seat massage app. The recommended atomic service function corresponding to the atomic service function for activating the seat massage function is used to set the parameter values ​​for the seat massage function's operating parameters. The corresponding information items in the information used to guide user interaction include: the parameter values ​​for the seat massage function's operating parameters. When the user speaks the parameter values ​​for the seat massage function's operating parameters, the recommended atomic service function corresponding to the atomic service function for activating the seat massage function is activated.

[0171] Referring to Figure 11, it shows a flowchart of an example of the interaction between the target vehicle and the user.

[0172] Multimodal information related to the target vehicle is input into the target model, which outputs the target intent. The skill distribution function of the target vehicle's voice app sends the corresponding instruction to each of at least one target intent to the app containing the atomic service function used to implement the target intent in the vehicle center app, navigation app, music app, etc. The app containing the atomic service function used to implement the target intent calls the atomic service function used to implement the target intent according to the instruction corresponding to the target intent, and obtains the call result corresponding to the atomic service function used to implement the target intent. When providing information to guide user interaction, in this example, the UI component of the call result corresponding to the atomic service function used to implement the target intent and the recommended atomic service function corresponding to the atomic service function used to implement the target intent is displayed in the dialogue service flow interface. In this example, the user's operation on the corresponding information item in the information used to guide user interaction is the user's operation on the information related to the recommended atomic service function corresponding to the atomic service function used to implement the target intent in the information used to guide user interaction. In this example, the feedback operation corresponding to the relevant information item is to call the recommended atomic service function corresponding to the atomic service function used to achieve the target intent.

[0173] Referring to Figure 12, which shows a schematic diagram of an example of recommended songs and sound effects.

[0174] The user says, "Recommend a song." This initiates a dialogue service flow, invoking the atomic service function of the music app on the target vehicle to search for recommended songs. The recommended song is then displayed on the vehicle's infotainment screen.

[0175] The atomic service function used to achieve the target intent corresponds to the atomic service function for setting the sound effects of recommended songs in the music app on the target vehicle. Figure 12 shows the UI components of the atomic service function for setting the sound effects of recommended songs displayed in the dialogue service flow interface. The UI components for setting the sound effects of recommended songs include multiple sound effect names such as HIFI Original Sound, Powerful Sound Effect, Immersive Surround, Concert Hall, Live House, and Cinema.

[0176] The user interface component for the recommended atomic service function corresponding to the atomic service function used to achieve the target intent allows users to invoke the recommended atomic service function corresponding to the atomic service function used to achieve the target intent by clicking on the user interface component for the recommended atomic service function used to achieve the target intent, or by speaking instructions to invoke the recommended atomic service function used to achieve the target intent. This facilitates continued interaction between the user and the vehicle and improves the convenience of the interaction between the vehicle and the user.

[0177] In one possible implementation, the recommended atomic service function corresponding to the atomic service function used to achieve the target intent is used to set the parameter values ​​of the working parameters of the vehicle function corresponding to the atomic service function used to achieve the target intent.

[0178] Among them, vehicle functions are the functions provided by the components of the target vehicle.

[0179] If the recommended atomic service function corresponding to the atomic service function used to achieve the target intent is used to set the parameter values ​​of the working parameters of the vehicle function corresponding to the atomic service function used to achieve the target intent, the UI component of the recommended atomic service function corresponding to the atomic service function used to achieve the target intent includes: multiple candidate values ​​of the working parameters of the vehicle function used to achieve the atomic service function used to achieve the target intent. Users can trigger the invocation of the recommended atomic service function corresponding to the atomic service function used to achieve the target intent by clicking on a candidate value of the working parameter or by speaking an instruction to select a candidate value of the working parameter. This sets the parameter value of the working parameter to the candidate value selected by the user, so that the target atomic service function works according to the set parameter value.

[0180] Referring to Figure 13, it shows a schematic diagram of a UI component that displays recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent.

[0181] When a user speaks the interactive voice command "Turn on seat massage," the atomic service function used to achieve this intention is the seat massage app's atomic service function for activating seat massage. This initiates a dialogue service flow, calling the seat massage app's atomic service function to activate the vehicle's seat massage function. Figure 13 shows the result of this call in the dialogue service flow interface: "Okay, seat massage is turned on." The recommended atomic service function for activating seat massage is used to set the parameter values ​​for the seat massage function's operating parameters. Figure 13 shows the UI components of this recommended atomic service function for activating seat massage in the dialogue service flow interface. This recommended atomic service function is used to set the parameter values ​​for the seat massage function's operating parameters. The UI components of this recommended atomic service function include operating parameters such as massage duration and massage intensity. Figure 13 shows candidate values ​​for the massage intensity parameter, such as "Level 1," "Level 2," and "Level 3." Figure 13 shows multiple parameter values ​​for the massage duration parameter, such as "15 minutes", "20 minutes", and "30 minutes". If the user says "Level 1" and "30 minutes" in succession, the seat massage function will work according to "Level 1" and "30 minutes", that is, it will work at "Level 1 massage intensity" for 30 minutes.

[0182] In one possible implementation, the atomic service function for achieving the target intent is used to activate the vehicle function for responding to the target situation, and the recommended atomic service function corresponding to the atomic service function for achieving the target intent is used to activate the alternative vehicle function for responding to the target situation. The UI component of the recommended atomic service function for achieving the target intent includes the name of the alternative vehicle function for responding to the target situation. The user can activate the alternative vehicle function by clicking the UI component of the recommended atomic service function for achieving the target intent or by speaking a command. The UI component of the recommended atomic service function for achieving the target intent triggers the invocation of the recommended atomic service function to activate the alternative vehicle function for responding to the target situation.

[0183] For each of the multiple situations encountered by the target vehicle, vehicle functions for dealing with that situation and alternative vehicle functions for dealing with that situation can be pre-set.

[0184] Referring to Figure 14, it shows a schematic diagram illustrating the effect of displaying the activated vehicle functions and alternative vehicle functions.

[0185] As an example, when a user says the interactive voice command "It's snowing heavily, what should I do?", the target situation the vehicle encounters is "snowing." The atomic service function used to achieve this target intent is the atomic service function used to activate vehicle functions designed to handle "snowing." This includes: activating the atomic service function to activate the rearview mirror heating, the steering wheel heating, the automatic windshield wipers, and the driver's seat heating. Thus, the rearview mirror heating, steering wheel heating, automatic windshield wipers, and driver's seat heating functions are activated. The alternative vehicle function for handling "snowing" is defrosting. The recommended atomic service function is the atomic service function used to activate defrosting, an alternative vehicle function for handling "snowing." Figure 14 shows the UI components for the recommended atomic service function, which includes the name of this alternative vehicle function for dealing with "snow," namely defrost.

[0186] In this embodiment, a scenario is considered where the user is unsure which vehicle function to activate when the vehicle encounters a target situation. When the target vehicle encounters a target situation, the appropriate vehicle function can be automatically activated, along with a UI component displaying the atomic service function corresponding to the alternative vehicle function for that situation. This allows the system to address the target vehicle's situation. Simultaneously, the user is informed that alternative vehicle functions can be activated. Users can activate these alternative functions by clicking the recommended atomic service function corresponding to the atomic service function for the target intention or by speaking a command to activate them, thus improving the convenience of user interaction.

[0187] In one possible implementation, invoking the atomic service function used to implement the target intent includes: when there are multiple target intents, invoking the atomic service function corresponding to each target intent.

[0188] For a target intent among multiple target intents, the corresponding atomic service function is used to implement that target intent.

[0189] As an example, the user says the interactive voice command "Play…, Navigate to…". Two target intentions are identified: the music app on the target vehicle plays the "…" in "Play…"; the navigation app on the target vehicle generates a navigation route from the target vehicle's location to the "…" in "Navigate to…".

[0190] Referring to Figure 15, which illustrates an example of simultaneously activating the atomic service functions corresponding to each of multiple intents. In this example, the user utters the interactive voice command "Turn on the seat massage and air conditioning." Two target intents are identified: turn on the seat massage and turn on the air conditioning. Figure 15 shows the result of the call displayed in the dialogue service flow interface: "Both the seat massage and air conditioning are now on."

[0191] In this embodiment, the user can speak commands indicating multiple intentions simultaneously, eliminating the need for the user to speak the commands corresponding to each intention individually. This simplifies user operations and enhances the user experience.

[0192] In one possible implementation, when a user-defined scene trigger condition is detected, an atomic service function is invoked to implement the intent corresponding to the scene trigger condition.

[0193] As an example, the user-defined scenario trigger condition is "a female passenger is sitting in the front passenger seat", and the corresponding intent of the scenario trigger condition is "turn on the fragrance". The atomic service function of the vehicle center APP of the target vehicle is called to turn on the fragrance function.

[0194] User-defined scene trigger conditions and corresponding intents can be determined by the user's voice commands.

[0195] Referring to Figure 16, it shows a schematic diagram of an example of determining scene trigger conditions and the intent corresponding to the scene trigger conditions.

[0196] The user's instruction for the custom interactive scene operation and the corresponding interactive scene operation trigger condition's voice is "...After getting on the bus, announce 'Happy Birthday...'". The confirmed scene trigger condition is: "...Get on the bus". The corresponding intent for the scene trigger condition is to announce "Happy Birthday...".

[0197] Referring to Figure 17, it shows a schematic diagram of the effect of invoking an atomic service function to implement an intent corresponding to a scene trigger condition.

[0198] A user-defined scenario trigger condition is: User A gets into the vehicle. After User A enters the target vehicle, the target model determines that the scenario trigger condition of User A getting into the vehicle has been met based on User A's identity information and the action information indicating User A's action of getting into the vehicle. It then calls the voice playback atomic service function of the voice broadcasting APP on the target vehicle to broadcast "Happy Birthday to...".

[0199] In one possible implementation, in response to detecting an operation on the dialogue service flow interface indicating a zoom-in or zoom-out of an image frame, the ratio of the adjusted image frame height indicated by the operation on the dialogue service flow interface to the height of the target vehicle's display screen is calculated; based on the ratio of the adjusted image frame height indicated by the operation on the dialogue service flow interface to the height of the target vehicle's display screen, the original image frame including information to be displayed on the dialogue service flow interface is zoomed in or out to obtain a target image frame including information to be displayed on the dialogue service flow interface, and the target image frame is displayed on the dialogue service flow interface.

[0200] If the ratio of the height of the adjusted image frame to the height of the target vehicle's display screen in the operation instruction indicating to reduce or enlarge the size of the image frame is greater than the ratio of the height of the original image frame to the height of the target vehicle's display screen, then the original image frame, including the information that needs to be displayed in the dialog service flow interface, is enlarged.

[0201] If the ratio of the height of the adjusted image frame to the height of the target vehicle's display screen in the operation instruction indicating to reduce or enlarge the image frame size is less than the ratio of the height of the original image frame to the height of the target vehicle's display screen, then the original image frame, including the information that needs to be displayed in the dialog service flow interface, is reduced.

[0202] The height of the target image frame, including the information to be displayed in the dialogue service flow interface, has an adjusted image frame height. The ratio of the width of the target image frame, including the information to be displayed in the dialogue service flow interface, to the width of the target vehicle's display screen can be: the ratio of the adjusted image frame height to the height of the target vehicle's display screen, indicating the operation of zooming in or out of the image frame.

[0203] When scaling down or up the original image frame, which includes the information to be displayed in the dialogue service flow interface, according to this ratio, to obtain the target image frame containing the information to be displayed in the dialogue service flow interface, a system-level component for scaling image frames can be used to scale down or up the original image frame, which includes the information to be displayed in the dialogue service flow interface, according to this ratio, thus obtaining the target image frame containing the information to be displayed in the dialogue service flow interface. This eliminates the need for adaptation with individual apps on the target vehicle. As an example, if the target vehicle's infotainment system runs on Android, the system-level DecorCaptionView component can be used to scale down or up the original image frame, which includes the information to be displayed in the dialogue service flow interface, according to this ratio, to obtain the target image frame containing the information to be displayed in the dialogue service flow interface.

[0204] In this embodiment, the original image frame containing target information is scaled down or enlarged according to the ratio of the adjusted image frame height to the height of the target vehicle's display screen, based on the operation instruction. This yields a target image frame containing the information to be displayed in the dialogue service flow interface. Displaying the target image frame in the dialogue service flow interface avoids the following situation in related technologies: when adjusting the size of the image frame to be displayed, the module of the application layer used to display the image frame notifies the kernel module of the operating system used to adjust the image frame. The kernel module of the operating system then modifies the image frame size configuration item from the original size to the adjusted size. Adjusting the size of the image frame to be displayed incurs significant overhead. This notification and modification cause the page to refresh during scaling, resulting in page jumping and the corresponding lifecycle methods being called back due to the page refresh, which adversely affects the performance of the vehicle system.

[0205] This application also provides an interactive device for implementing the above-described method embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "unit" can refer to a combination of software and / or hardware that performs a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated. The devices in this application are presented in the form of functional units, where a functional unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above-described functions.

[0206] An interactive device, the interactive device comprising:

[0207] The intent determination unit is used to determine the target intent related to the user based on multimodal information related to the target vehicle;

[0208] The startup unit is used to initiate a dialogue service flow, which includes invoking atomic service functions used to achieve the target intent.

[0209] Optionally, the dialogue service flow also includes:

[0210] Determine the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent;

[0211] Provide users with information to guide user interaction. This information includes: the call results corresponding to the atomic service functions used to achieve the target intent, and information related to the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent.

[0212] Optionally, providing users with information to guide user interaction includes:

[0213] Display information that guides user interaction or play voice prompts for information items in the dialogue service flow interface.

[0214] Optionally, the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent include:

[0215] Based on information about users' usage habits of atomic service functions, determine the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent.

[0216] Optionally, the dialogue service flow also includes:

[0217] After providing the user with information to guide user interaction, in response to the user's operation on the corresponding information item in the information to guide user interaction, a feedback operation corresponding to the corresponding information item is executed.

[0218] Optionally, the recommended atomic service function is used to set the parameter values ​​of the working parameters of the vehicle function corresponding to the atomic service function used to achieve the target intent.

[0219] Optionally, the atomic service function for achieving the target intent is used to enable the vehicle function for responding to the target situation, and the recommended atomic service function is used to enable the alternative vehicle function for responding to the target situation.

[0220] Optionally, invoking atomic service functions to achieve the target intent includes:

[0221] When there are multiple target intents, the atomic service function corresponding to each target intent is invoked, wherein the atomic service function corresponding to the target intent is used to implement the target intent.

[0222] Optionally, invoking atomic service functions to achieve the target intent includes:

[0223] For each of the at least one target intent, an instruction corresponding to the target intent is sent to the application to which the atomic service function for implementing the target intent belongs in at least one application corresponding to the at least one target intent, so that the application invokes the atomic service function for implementing the target intent according to the instruction.

[0224] Optionally, the intent determination unit is used to determine the target intent based on multimodal information related to the target vehicle using a target model on a cloud server.

[0225] Optionally, the interactive device also includes:

[0226] The scene triggering unit is used to invoke atomic service functions that implement the intent corresponding to the scene triggering conditions when a user-defined scene triggering condition is detected.

[0227] Optionally, the interactive device also includes:

[0228] A scaling unit is configured to, in response to detecting an operation on the dialogue service flow interface indicating to zoom in or out of an image frame, calculate the ratio of the adjusted image frame height indicated by the operation to the height of the target vehicle's display screen.

[0229] Based on the stated ratio, the original image frame, including the information to be displayed in the dialog service flow interface, is reduced or enlarged to obtain the target image frame, and the target image frame is displayed in the dialog service flow interface.

[0230] Referring to Figure 18, which is a schematic diagram of the hardware structure of a computer device according to an embodiment of this application, the computer device includes one or more processors 10, a memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components communicate with each other using different buses and can be installed on a common motherboard or otherwise as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple devices can be connected, each providing some necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). The processor 10 can be a central processing unit, a network processor, or a combination thereof. The processor 10 may also optionally include a hardware chip. The hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device can be a complex programmable logic device (CLP), a field-programmable gate array (FPGA), a general-purpose array logic (GPRS), or any combination thereof. The memory 20 stores instructions executable by at least one processor 10 to perform the methods shown in the above embodiments. The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on vehicle usage, etc. Furthermore, the memory 20 may include high-speed random access memory and non-transient memory, such as at least one disk storage device, flash memory device, or other non-transient solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, which can be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof. The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk, or solid-state drive; the memory 20 may also include combinations of the above types of memory. The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 may be connected via a bus or other means. The input device 30 can receive input digital or character information, and generate key signal inputs related to user settings and function control of the computer device, such as touch screen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc.The output device 40 may include a display device, an auxiliary lighting device (e.g., an LED), and a haptic feedback device (e.g., a vibration motor). The display device includes, but is not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0231] This application also provides a computer-readable storage medium. The methods described in this application can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code downloaded over a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; optionally, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0232] A portion of the embodiments of this application can be applied as a computer program product, such as computer program instructions. When executed by a computer, these instructions, through the operation of the computer, can invoke or provide the methods and / or technical solutions according to this application. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, and installation package files. Accordingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions; the computer compiling the instructions and then executing the corresponding compiled program; the computer reading and executing the instructions; or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0233] The above embodiments are merely preferred embodiments provided to fully illustrate this application, and the scope of protection of this application is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on this application are all within the scope of protection of this application.

Claims

1. An interaction method, characterized by: The method includes: Based on multimodal information related to the target vehicle, determine the target intent related to the user; Initiate a dialogue service flow, which includes invoking atomic service functions used to achieve the target intent.

2. The method of claim 1, wherein: The dialogue service flow also includes: Determine the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent; Provide users with information to guide user interaction. This information includes: the call results corresponding to the atomic service functions used to achieve the target intent, and information related to the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent.

3. The method of claim 2, wherein: Providing users with information to guide user interaction includes: Display information that guides user interaction or play voice prompts for information items in the dialogue service flow interface.

4. The method of claim 2, wherein: Determine the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent; Based on information about users' usage habits of atomic service functions, determine the recommended atomic service functions corresponding to the atomic service functions used to achieve the target intent.

5. The method of claim 2, wherein: The dialogue service flow also includes: After providing the user with information to guide user interaction, in response to the user's operation on the corresponding information item in the information to guide user interaction, a feedback operation corresponding to the corresponding information item is executed.

6. The method of claim 2, wherein: The recommended atomic service function is used to set the parameter values ​​of the working parameters of the vehicle function corresponding to the atomic service function used to achieve the target intent.

7. The method of claim 2, wherein: The atomic service function for achieving the target intent is used to enable the vehicle function for responding to the target situation, and the recommended atomic service function is used to enable the alternative vehicle function for responding to the target situation.

8. The method of claim 1, wherein: Invoking atomic service functions used to achieve the target intent includes: When there are multiple target intents, the atomic service function corresponding to each target intent is invoked, wherein the atomic service function corresponding to the target intent is used to implement the target intent.

9. The method of claim 1, wherein: Invoking atomic service functions used to achieve the target intent includes: For each of the at least one target intent, an instruction corresponding to the target intent is sent to the application to which the atomic service function for implementing the target intent belongs in the at least one application corresponding to the at least one target intent, so that the application to which the atomic service function for implementing the target intent belongs invokes the atomic service function for implementing the target intent according to the instruction.

10. The method of claim 1, wherein: Based on multimodal information related to the target vehicle, the target intent is determined as follows: By utilizing the target model on the cloud server, the target intent is determined based on multimodal information related to the target vehicle.

11. The method of claim 1, wherein: The method further includes: When a user-defined scene trigger condition is detected to be met, the atomic service function used to implement the intent corresponding to the scene trigger condition is invoked.

12. The method of claim 1, wherein: The method further includes: In response to detecting an operation on the dialogue service flow interface that instructs the user to zoom in or out on the size of an image frame, the ratio of the adjusted image frame height indicated by the operation to the height of the target vehicle's display screen is calculated. Based on the stated ratio, the original image frame, including the information to be displayed in the dialogue service flow interface, is reduced or enlarged to obtain a target image frame including the information to be displayed in the dialogue service flow interface, and the target image frame is displayed in the dialogue service flow interface.

13. An interactive device, characterized by: The device includes: The intent determination unit is used to determine the target intent related to the user based on multimodal information related to the target vehicle; The startup unit is used to initiate a dialogue service flow, which includes invoking atomic service functions used to achieve the target intent.

14. A computer device, comprising: include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method of any one of claims 1 to 12.

15. A vehicle characterized by comprising: include: A memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, the processor executing the computer instructions to perform the method of any one of claims 1 to 12.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the method of any one of claims 1 to 12.

17. A computer program product, characterised in that, Includes computer instructions for causing a computer to perform the method of any one of claims 1 to 12.

Images