Vehicle interaction method and device, vehicle, electronic equipment and storage medium

Abstract

The invention discloses an interaction method and device of a vehicle, the vehicle, electronic equipment and a storage medium, and relates to the technical field of vehicles, receiving an interaction request of man-machine interaction, identifying fusion type information in the interaction request according to a pre-established preset processing model, obtaining a to-be-executed task corresponding to the interaction request, and sending the to-be-executed task to a server; and calling at least one target plug-in corresponding to the to-be-executed task in the preset plug-in system, performing parallel processing on the corresponding to-be-executed task by the at least one target plug-in, and outputting a processing result of the at least one target plug-in to complete human-computer interaction. According to the method, the interaction request of the user is identified as the executable to-be-executed task, the at least one target plug-in corresponding to the to-be-executed task is determined by calling the preset plug-in system, and the corresponding sub-task is executed in parallel by the at least one plug-in, so that from the perspective of the user, the interaction operation steps are simplified, the man-machine interaction operation with the vehicle machine only needs to be executed once, and the user experience is improved. Therefore, the vehicle use requirement of executing multiple tasks through one-time interaction can be met.

Description

Technical Field

[0001] The present disclosure relates to the technical field of vehicles, and in particular, to an interaction method and device for a vehicle, a vehicle, an electronic device, and a storage medium. Background Art

[0002] Vehicle human-machine interaction, simply speaking, is a process in which a person and a vehicle complete information exchange through a certain interaction method, which can directly affect the user's vehicle use experience. Its interaction method has gradually developed from the original single physical knob to the current intelligent interaction methods such as touch screen interaction and voice interaction.

[0003] Before driving a vehicle, if a user wants to take a long-distance trip, taking touch screen interaction as an example, it may be necessary to separately set navigation applications, cockpit temperature applications, player applications, etc. The specific setting process includes: after the user starts the navigation application, enters the destination, and then returns to the main screen; starts the cockpit temperature application, sets the air conditioning information, and then returns to the main screen; starts the player application, sets the audio / video, and then returns to the main screen to complete the human-machine interaction process.

[0004] From the above user's human-machine interaction process, it can be seen that since each application is independent of each other, when multiple applications in the vehicle are used simultaneously, the interaction operation steps are cumbersome, restricting the user's needs. Summary of the Invention

[0005] The present disclosure provides an interaction method, device, vehicle, electronic device, and storage medium for a vehicle. Its main purpose is to solve the problem that since each application is independent of each other, when multiple applications in the vehicle are used simultaneously, the interaction operation steps are cumbersome, restricting the user's needs.

[0006] According to a first aspect of the present disclosure, there is provided an interaction method for a vehicle, including:

[0007] Receiving an interaction request for human-machine interaction;

[0008] Identifying the fusion type information in the interaction request according to a preset processing model established in advance to obtain a to-be-executed task corresponding to the interaction request, where the fusion type information includes at least one interaction input information;

[0009] Invoking at least one target plugin corresponding to the to-be-executed task in a preset plugin system, and respectively performing parallel processing on the corresponding to-be-executed task through the at least one target plugin;

[0010] Outputting and displaying the task execution result of the to-be-executed task.

[0011] In some embodiments, the invoking at least one target plugin corresponding to the to-be-executed task in the preset plugin system includes:

[0012] Invoke a generative pre-trained model in the preset plug-in system to find the target scenario corresponding to the task to be executed; the preset plug-in system contains different preset scenarios and their corresponding plug-ins, and different scenarios correspond to the same or different plug-ins;

[0013] According to the corresponding relationship between the scenarios and plug-ins in the generative pre-trained model, find at least one target plug-in corresponding to the task to be executed.

[0014] In some embodiments, the finding at least one target plug-in corresponding to the task to be executed according to the corresponding relationship between the scenarios and plug-ins in the generative pre-trained model includes:

[0015] Respectively obtain the interface addresses of the at least one target plug-in from the corresponding relationship between the scenarios and plug-ins in the generative pre-trained model;

[0016] Find the corresponding target plug-ins according to the respective interface addresses.

[0017] In some embodiments, before finding the target plug-in corresponding to the task to be executed according to the corresponding relationship between the scenarios and plug-ins, the method includes:

[0018] In response to the request information for registering a plug-in, the preset plug-in system sends the request information for registering the plug-in to the generative pre-trained model according to the plug-in protocol, and the request information includes the life cycle of the registered plug-in and the corresponding belonging scenario;

[0019] The generative pre-trained model obtains the belonging scenario, and adds the registered plug-in and the belonging scenario to the corresponding relationship between the plug-ins and scenarios;

[0020] Monitor the registered plug-in according to the life cycle;

[0021] After the life cycle arrives, cancel the registration of the registered plug-in and delete the registered plug-in from the corresponding relationship between the plug-ins and scenarios.

[0022] In some embodiments, the output display of the task execution result of the task to be executed includes:

[0023] Obtain the input category of the fusion-type information, and the input category of the fusion-type information includes at least one of pictures, voices, gestures, videos, and gazes;

[0024] Display the processing results of the at least one target plug-in in a display style matching the input category.

[0025] In some embodiments, the output display of the task execution result of the to-be-executed task includes:

[0026] Parse the processing result of the at least one target plug-in to determine the marker information carried in the processing result, and the processing result carries marker information of directly controlling the vehicle state / not directly controlling the vehicle state;

[0027] If it is determined that the marker information is to directly control the vehicle state, output the execution result information corresponding to the controlled vehicle state;

[0028] If it is determined that the marker information is not to directly control the vehicle state, call the user interface container and construct a user interface view corresponding to the processing result based on the user interface container;

[0029] Output and display the user interface view.

[0030] In some embodiments, after outputting and displaying the user interface view, the method includes:

[0031] In response to an interaction instruction triggered by a user in the user interface view, determine the interaction instruction as an execution event and send it to the scheduling module, and the scheduling module responds to the interaction instruction to complete multi-round interaction, and the scheduling module performs interaction by calling the preset plug-in system.

[0032] According to a second aspect of the present disclosure, there is provided an interaction device for a vehicle, including:

[0033] A receiving unit for receiving an interaction request for human-machine interaction;

[0034] An identifying unit for identifying the fusion-type information in the interaction request according to a preset processing model established in advance to obtain a to-be-executed task corresponding to the interaction request, where the fusion-type information includes at least one interaction input information;

[0035] A calling unit for calling at least one target plug-in corresponding to the to-be-executed task in a preset plug-in system;

[0036] A processing unit for respectively performing parallel processing on the corresponding to-be-executed task through the at least one target plug-in;

[0037] An output unit for outputting and displaying the task execution result of the to-be-executed task.

[0038] In some embodiments, the calling unit includes:

[0039] A calling module, configured to call a generative pre-trained model in the preset plug-in system to find a target scenario corresponding to the to-be-executed task; the preset plug-in system includes preset different scenarios and their corresponding plug-ins, and different scenarios correspond to the same or different plug-ins;

[0040] A searching module, configured to search for at least one target plug-in corresponding to the to-be-executed task according to the corresponding relationship between scenarios and plug-ins in the generative pre-trained model.

[0041] In some embodiments, the searching module is further configured to:

[0042] Obtain the interface addresses of the at least one target plug-in respectively from the corresponding relationship between the scenarios and the plug-ins in the generative pre-trained model;

[0043] Search for the corresponding target plug-ins according to the respective interface addresses.

[0044] In some embodiments, the calling unit further includes:

[0045] A sending module, configured to, before the searching module searches for a target plug-in corresponding to the to-be-executed task according to the corresponding relationship between scenarios and plug-ins, in response to a request message for registering a plug-in, send the request message for registering the plug-in to the generative pre-trained model by the preset plug-in system according to the plug-in protocol, where the request message includes the life cycle of the registered plug-in and the corresponding belonging scenario;

[0046] An obtaining module, configured to obtain the belonging scenario by the generative pre-trained model;

[0047] An adding module, configured to add the registered plug-in and the belonging scenario to the corresponding relationship between the plug-ins and the scenarios;

[0048] A monitoring module, configured to monitor the registered plug-in according to the life cycle;

[0049] A processing module, configured to, after the life cycle arrives, cancel the registration of the registered plug-in and delete the registered plug-in from the corresponding relationship between the plug-ins and the scenarios.

[0050] In some embodiments, the output unit includes:

[0051] An obtaining module, configured to obtain the input category of the fusion-type information, where the input category of the fusion-type information includes at least one of picture, voice, gesture, video, and gaze;

[0052] A first display module, configured to display the processing results of the at least one target plug-in in a display style matching the input category.

[0053] In some embodiments, the output unit includes:

[0054] A parsing module, configured to parse the processing result of the at least one target plugin, determine the marker information carried in the processing result, and the processing result carries marker information of directly controlling the vehicle state / indirectly controlling the vehicle state;

[0055] An output module, configured to output execution result information corresponding to the controlled vehicle state if it is determined that the marker information is for directly controlling the vehicle state;

[0056] An invocation module, configured to invoke a user interface container if it is determined that the marker information is for indirectly controlling the vehicle state;

[0057] A construction module, configured to construct a user interface view corresponding to the processing result based on the user interface container;

[0058] A second display module, configured to output and display the user interface view.

[0059] In some embodiments, the device includes:

[0060] A processing unit, configured to, after outputting and displaying the user interface view, in response to an interaction instruction triggered by a user in the user interface view, determine the interaction instruction as an execution event and send it to a scheduling module, and the scheduling module responds to the interaction instruction to complete multi-round interaction, and the scheduling module performs interaction by invoking the preset plugin system.

[0061] According to a third aspect of the present disclosure, there is provided a vehicle, where the vehicle includes the vehicle interaction device as described in the second aspect.

[0062] According to a fourth aspect of the present disclosure, there is provided an electronic device, including:

[0063] At least one processor; and

[0064] A memory communicatively connected to the at least one processor; wherein,

[0065] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the foregoing first aspect.

[0066] According to a fifth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method described in the foregoing first aspect.

[0067] According to a sixth aspect of the present disclosure, there is provided a computer program product including a computer program which, when executed by a processor, implements the method described in the foregoing first aspect.

[0068] For the vehicle interaction method, device, vehicle, electronic device, and storage medium provided by the present disclosure, an interaction request of human-machine interaction is received, and the fusion information in the interaction request is identified according to a preset processing model established in advance to obtain a to-be-executed task corresponding to the interaction request. The fusion information includes at least one interaction input information. At least one target plugin corresponding to the to-be-executed task in a preset plugin system is called, and the corresponding to-be-executed task is processed in parallel through the at least one target plugin, and the task execution result of the to-be-executed task is output and displayed. Compared with the related art, in the embodiment of the present application, by identifying the user's interaction request as an executable to-be-executed task, at least one target plugin corresponding to the to-be-executed task is determined by calling the preset plugin system, and at least one plugin executes the corresponding subtasks in parallel. From the user's perspective, the interaction operation steps are simplified, and only one human-machine interaction operation with the vehicle machine needs to be performed to meet the vehicle use requirement of executing multiple tasks in one interaction.

[0069] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0071] Figure 1 is a schematic flowchart of a vehicle interaction method provided by an embodiment of the present disclosure;

[0072] Figure 2 is a schematic flowchart of a plugin registration method provided by an embodiment of the present application;

[0073] Figure 3 is a schematic structural diagram of a vehicle interaction device provided by an embodiment of the present disclosure;

[0074] Figure 4 is a schematic structural diagram of another vehicle interaction device provided by an embodiment of the present disclosure;

[0075] Figure 5 is a schematic block diagram of an exemplary electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0076] The exemplary embodiments of the present disclosure will be described below in conjunction with the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, descriptions of well-known functions and structures are omitted in the following description for clarity and conciseness.

[0077] The vehicle interaction method, device, vehicle, electronic device, and storage medium according to the embodiments of the present disclosure will be described below with reference to the accompanying drawings.

[0078] Figure 1 The flowchart of a vehicle interaction method provided by an embodiment of the present disclosure. The method is applied to a vehicle interaction system, such as Figure 1 As shown, the method includes the following steps:

[0079] Step 101, receiving an interaction request for human-machine interaction.

[0080] The system described in the embodiments of the present application takes natural language as input and provides the ability to interact with users in natural language conversations. The user inputs unstructured fusion information into the vehicle interaction system. The fusion information includes, but is not limited to, at least one of image information, acoustic information, touch interaction, geographical location information, vehicle status information, environmental information (such as light information), etc. The fusion information is used to trigger an interaction request during human-vehicle interaction.

[0081] Step 102, identifying the fusion information in the interaction request to obtain a to-be-executed task corresponding to the interaction request. The fusion information includes at least one interaction input information.

[0082] As an implementation manner of the embodiments of the present application, the fusion information in the interaction request is identified through a pre-established recognition model to obtain a to-be-executed task corresponding to the interaction request. The pre-established recognition model described in the embodiments of the present application is a diversified platform that can identify any type of input information among image information, acoustic information, touch interaction, geographical location information, vehicle status information, environmental information (such as light information), etc. That is, the user only needs to give the vehicle-mounted system an interaction, and the pre-established recognition model analyzes and processes it, understands the environment and task requirements according to the requirements corresponding to the interaction request, and converts the recognition result into an executable to-be-executed task of the vehicle-mounted system.

[0083] As another implementation manner of the embodiments of the present application, any recognition algorithm in the prior art may also be used to recognize the fusion-type information in the interaction request. For example, when the fusion-type information includes an image, any image recognition algorithm in the prior art is used as the recognition algorithm; when the fusion-type information includes voice, any acoustic recognition algorithm in the prior art is used as the recognition algorithm, and so on. The specific recognition algorithms in the prior art called by the embodiments of the present application will not be elaborated one by one.

[0084] Step 103: Invoke at least one target plug-in corresponding to the to-be-executed task in the preset plug-in system, and respectively perform parallel processing on the corresponding to-be-executed task through the at least one target plug-in.

[0085] The preset plug-in system described in the embodiments of the present application aims to enhance and customize the natural language processing ability of the vehicle interaction system. First, it can process and change the user input information and provide additional context for the system, so as to optimize the output result of the model. The plug-in can enable the system to access and obtain the latest information, connect to and use third-party services, improve the matching degree between actual application requirements, and improve the performance of the system in various scenarios. Second, the plug-in can also empower developers, enabling developers to better control the behavior of the system, enabling developers to optimize, customize and expand the capabilities of the system according to specific requirements and preferences, and upgrading the system into a powerful and diversified platform.

[0086] In the embodiments of the present application, the plug-in Plugin can be provided by an application, can be provided by the system, or can be provided by an independent service. Any module that provides atomic capabilities following the standard plug-in protocol can be regarded as a plug-in.

[0087] After the preset plug-in system obtains the executable to-be-executed task, it matches at least one target plug-in corresponding to the to-be-executed task in the preset plug-in system, and schedules the to-be-executed task to the corresponding target plug-in, and different target plug-ins execute the to-be-executed task in parallel.

[0088] Step 104: Output and display the task execution result of the to-be-executed task.

[0089] The execution results of each target plug-in can be presented in forms such as vibration, light, voice, text, etc. Specifically, the embodiments of the present application do not limit the presentation form of the execution result.

[0090] The vehicle interaction method provided by the present disclosure receives an interaction request for human-machine interaction, identifies the fusion information in the interaction request according to a preset processing model established in advance, obtains a to-be-executed task corresponding to the interaction request, where the fusion information includes at least one interaction input information, calls at least one target plugin corresponding to the to-be-executed task in a preset plugin system, respectively performs parallel processing on the corresponding to-be-executed task through the at least one target plugin, and outputs and displays the task execution result of the to-be-executed task. Compared with the related art, in the embodiment of the present application, the user's interaction request is identified as an executable to-be-executed task, at least one target plugin corresponding to the to-be-executed task is determined by calling a preset plugin system, and at least one plugin executes the corresponding subtasks in parallel. From the user's perspective, the interaction operation steps are simplified, and only one human-machine interaction operation with the in-vehicle computer needs to be performed to meet the vehicle use requirement of completing multiple tasks in one interaction.

[0091] As a refinement of the above embodiment, when step 103 executes to call at least one target plugin corresponding to the to-be-executed task in the preset plugin system, the following methods can be used but are not limited to: call a Generative Pre-trained Transformer (GPT) in the preset plugin system to find the target scenario corresponding to the to-be-executed task; the preset plugin system contains preset different scenarios and their corresponding plugins, and different scenarios correspond to the same or different plugins; according to the correspondence between the scenarios and plugins in the Generative Pre-trained Transformer, find at least one target plugin corresponding to the to-be-executed task. In the embodiment of the present application, one to-be-executed task corresponds to one target scenario, and one scenario corresponds to at least one target plugin. These target plugins jointly respond to a user's interaction request to meet the user's vehicle use requirement of completing multiple tasks in one interaction.

[0092] To enhance the accuracy of GPT search, separate training can be performed according to different scenarios and their corresponding at least one target plugin to improve the accuracy of GPT. The training process is not the focus of the embodiment of the present application, so it will not be elaborated one by one.

[0093] Before using the plugin, the registration of the plugin needs to be executed. When registering the plugin, the interface address of the plugin is carried. When searching for the target plugin subsequently, the corresponding target plugin can be directly found through the addressing method. The specific search process includes: obtaining the interface addresses of the at least one target plugin from the correspondence between the scenarios and the plugins in the generative pre-trained model, and searching for the corresponding target plugins according to the respective interface addresses. Each target plugin corresponds to a unique interface (Application Programming Interface, API) address, which enables users to use various plugins more conveniently.

[0094] The above embodiments have detailed the usage process of the plugin, such as Figure 2 shown Figure 2 is a schematic flowchart of a plugin registration method provided by an embodiment of the present application, including:

[0095] Step 201, in response to the request information for registering the plugin, the preset plugin system sends the request information for registering the plugin to the generative pre-trained model according to the plugin protocol, and the request information includes the life cycle of the registered plugin and the corresponding affiliated scenario.

[0096] The registration of the plugin includes but is not limited to the internal plugin registration of the system and the third-party plugin registration. The registration of the plugin needs to follow the standard plugin protocol. The plugin protocol includes but is not limited to: the Plugin interface, the API interface, and the Manifest list file. Among them, the Plugin interface: defines the system interface, and the plugin needs to implement the Plugin interface. The API interface: includes multiple functions and is responsible for defining the input and output of data in different scenarios. The Manifest list file: tells the system how to call the API through the natural language prompt prompt, enables the system to learn which API to call in different scenarios, what parameters need to be passed in when calling the API, and what the output parameters are, etc.

[0097] The preset plugin system has a set of standard registration mechanisms. When the registered plugin Plugin registers with the plugin system, it carries the (specific domain Domain) of the registered plugin. According to the specific domain Domain, the affiliated scenario can be divided. In the preset plugin system, the request information for the registered plugin is received, and the preset plugin system registers the request information with the GPT plugin interface.

[0098] Step 202, the generative pre-trained model obtains the affiliated scenario and adds the registered plugin and the affiliated scenario to the correspondence between the plugins and the scenarios.

[0099] Step 203, monitor the registered plugin according to the life cycle.

[0100] Step 204, after the lifecycle arrives, deactivate the registered plugin and delete the registered plugin from the correspondence between the plugin and the scenario.

[0101] Each in the preset plugin system includes key steps such as registration, creation, invocation, and deactivation. The plugin management service can manage the lifecycles of various plugins through the above steps. When a plugin is created or deactivated, it needs to be synchronously updated in the correspondence between the plugin and the scenario to avoid errors in GPT calls and thus affect the user's vehicle usage experience.

[0102] After the target plugin finishes executing the corresponding subtask, the processing result of the target plugin needs to be fed back to the user for viewing. The embodiments of this application adopt two ways for display:

[0103] Way 1: Obtain the input category of the fusion-type information, where the input category of the fusion-type information includes at least one of picture, voice, gesture, video, and gaze; display the processing results of the at least one target plugin in a display style matching the input category.

[0104] In the embodiments of this application, the input category and its matching display style are an editable correspondence, and the user can flexibly configure according to their own preferences.

[0105] For ease of understanding, when the input category of the fusion-type information is a gesture, the display styles matching the processing results of the target plugin may include but are not limited to voice and UI interface. Specifically, the embodiments of this application are not limited thereto.

[0106] Way 2: Analyze the processing results of the at least one target plugin to determine the marker information carried in the processing results, where the processing results carry marker information of directly controlling the vehicle state / indirectly controlling the vehicle state; if it is determined that the marker information is directly controlling the vehicle state, output the execution result information corresponding to controlling the vehicle state; if it is determined that the marker information is indirectly controlling the vehicle state, call the user interface container and construct a user interface view corresponding to the processing results based on the user interface container, and output and display the user interface view.

[0107] To facilitate the understanding of direct control of vehicle status / indirect control of vehicle status, the following is an example. When the target subtask is to open the window, the target plugin is the window drive plugin. Directly controlling the opening of the window can complete the execution of this target subtask. This window drive plugin directly controls the vehicle status without the need to display the task execution result. Another scenario is the one that requires the display of the task execution result (indirect control of vehicle status). For example, when the user executes music playback or navigates to a certain location, the music playback interface or the navigation interface needs to be displayed on the display screen. This target plugin indirectly controls the vehicle status and needs to display the processing result.

[0108] For the scenario where the marked information is indirect control of vehicle status, the processing result of the target plugin is converted into corresponding layout description information. The layout description information is used to describe the processing results and layout attribute information of each target plugin. The layout attribute information includes the identification information and display information of the target plugin. In practical applications, the layout description information can describe the display information of the interface layout through a specific layout language (DSL language). The display information includes, but is not limited to, the position information on the display screen or in the application, the size of the display window, the display style, the transparency, etc. Specifically, the embodiments of the present application do not limit this.

[0109] The system calls the user interface UI container to parse the obtained layout description information to obtain the identification information and display information of the target plugin. In the specific application process, the obtained identification information of the target plugin is regarded as the identification information of a card, that is, one target plugin corresponds to one card node. For example, target plugin 1 corresponds to card node 1, target plugin 2 corresponds to card node 2, and so on.

[0110] Load the target plugins corresponding to multiple target subtasks respectively according to the identification information of the target plugin, determine the layout information of the loaded target plugin on the display desktop or in the application according to the obtained display information, and generate and construct the user interface UI views corresponding to the multiple subtarget tasks respectively according to the layout information, and render and display the user interface UI views. It can be understood that the layout information is used as a node tree, and the rendering engine dynamically constructs the UI view according to the node tree and presents the UI view through the generative UI container.

[0111] Based on the displayed UI view, in response to the interaction instruction triggered by the user in the user interface UI view, determine the interaction instruction as an execution event and send it to the scheduling module. The scheduling module responds to the interaction instruction to complete multiple rounds of interaction. The scheduling module performs the interaction by calling the preset plugin system.

[0112] In response to a user interaction instruction, callback the operation behaviors such as user clicking, swiping, and dragging, process the interaction event through a scheduling module, and the scheduling module passes the interaction event to a preset plugin system for scheduling to complete the closed-loop of the interaction link.

[0113] Corresponding to the above vehicle interaction method, the present invention also proposes a vehicle interaction device. Since the device embodiment of the present invention corresponds to the above method embodiment, the details not disclosed in the device embodiment can be referred to the above method embodiment, and will not be elaborated in the present invention.

[0114] Figure 3 It is a schematic structural diagram of a vehicle interaction device provided by an embodiment of the present disclosure, as Figure 3 shown, including:

[0115] A receiving unit 31, configured to receive an interaction request of human-machine interaction;

[0116] An identifying unit 32, configured to identify the fusion information in the interaction request according to a preset processing model established in advance, and obtain a to-be-executed task corresponding to the interaction request, where the fusion information includes at least one interaction input information;

[0117] An invoking unit 33, configured to invoke at least one target plugin corresponding to the to-be-executed task in a preset plugin system;

[0118] A processing unit 34, configured to respectively perform parallel processing on the corresponding to-be-executed task through the at least one target plugin;

[0119] An output unit 35, configured to output and display the task execution result of the to-be-executed task.

[0120] The vehicle interaction device provided by the present disclosure receives an interaction request of human-machine interaction, identifies the fusion information in the interaction request according to a preset processing model established in advance, obtains a to-be-executed task corresponding to the interaction request, where the fusion information includes at least one interaction input information, invokes at least one target plugin corresponding to the to-be-executed task in a preset plugin system, respectively performs parallel processing on the corresponding to-be-executed task through the at least one target plugin, and outputs and displays the task execution result of the to-be-executed task. Compared with the related art, in the embodiment of the present application, by identifying the user's interaction request as an executable to-be-executed task, determining at least one target plugin corresponding to the to-be-executed task by invoking a preset plugin system, and having at least one plugin execute the corresponding subtasks in parallel, from the user's perspective, the interaction operation steps are simplified, and only one human-machine interaction operation with the in-vehicle computer needs to be performed to meet the vehicle use requirement of executing multiple tasks in one interaction.

[0121] Further, in a possible implementation manner of this embodiment, as Figure 4 shown, the calling unit 33 includes:

[0122] A calling module 331, configured to call a generative pre-trained model in the preset plug-in system to find a target scenario corresponding to the to-be-executed task; different scenarios and their corresponding plug-ins are preset in the preset plug-in system, and different scenarios correspond to the same or different plug-ins;

[0123] A searching module 332, configured to find at least one target plug-in corresponding to the to-be-executed task according to the corresponding relationship between scenarios and plug-ins in the generative pre-trained model.

[0124] Further, in a possible implementation manner of this embodiment, the searching module 332 is further configured to:

[0125] respectively obtain the interface addresses of the at least one target plug-in from the corresponding relationship between scenarios and plug-ins in the generative pre-trained model;

[0126] respectively find the corresponding target plug-ins according to the respective interface addresses.

[0127] Further, in a possible implementation manner of this embodiment, as Figure 4 shown, the calling unit 33 further includes:

[0128] A sending module 333, configured to, before the searching module finds a target plug-in corresponding to the to-be-executed task according to the corresponding relationship between scenarios and plug-ins, in response to a request message for registering a plug-in, send the request message for registering the plug-in to the generative pre-trained model by the preset plug-in system according to the plug-in protocol, where the request message includes the life cycle of the registered plug-in and the corresponding belonging scenario;

[0129] An obtaining module 334, configured to obtain the belonging scenario by the generative pre-trained model;

[0130] An adding module 335, configured to add the registered plug-in and the belonging scenario to the corresponding relationship between plug-ins and scenarios;

[0131] A monitoring module 336, configured to monitor the registered plug-in according to the life cycle;

[0132] A processing module 337, configured to, after the life cycle arrives, cancel the registration of the registered plug-in and delete the registered plug-in from the corresponding relationship between plug-ins and scenarios.

[0133] Further, in a possible implementation manner of this embodiment, as Figure 4As shown, the output unit 35 includes:

[0134] An acquisition module 351, configured to acquire the input category of the fusion-type information, where the input category of the fusion-type information includes at least one of pictures, voices, gestures, videos, and gazes;

[0135] A first display module 352, configured to display the processing results of the at least one target plug-in in a display style matching the input category.

[0136] Further, in a possible implementation manner of this embodiment, as Figure 4 shown, the output unit 35 includes:

[0137] An analysis module 353, configured to analyze the processing results of the at least one target plug-in to determine the marker information carried in the processing results, where the processing results carry marker information of directly controlling the vehicle state / indirectly controlling the vehicle state;

[0138] An output module 354, configured to output the execution result information corresponding to the vehicle state control if it is determined that the marker information is directly controlling the vehicle state;

[0139] An invocation module 355, configured to invoke the user interface container if it is determined that the marker information is indirectly controlling the vehicle state;

[0140] A construction module 356, configured to construct a user interface view corresponding to the processing results based on the user interface container;

[0141] A second display module 357, configured to output and display the user interface view.

[0142] Further, in a possible implementation manner of this embodiment, as Figure 4 shown, the device includes:

[0143] A processing unit 36, configured to, after outputting and displaying the user interface view, in response to an interaction instruction triggered by a user in the user interface view, determine the interaction instruction as an execution event and send it to a scheduling module, and the scheduling module responds to the interaction instruction to complete multi-round interaction, and the scheduling module performs interaction by invoking the preset plug-in system.

[0144] It should be noted that the foregoing explanations of the method embodiments also apply to the device in this embodiment, with the same principle, and are not limited in this embodiment.

[0145] According to the embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0146] Figure 5 FIG. 2 shows a schematic block diagram of an example electronic device 400 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, personal digital assistants, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementations of the present disclosure described and / or claimed herein.

[0147] As Figure 5 shown, the device 400 includes a computing unit 401 that can perform various appropriate actions and processes in accordance with a computer program stored in a ROM (Read-Only Memory) 402 or a computer program loaded from a storage unit 408 into a RAM (Random Access Memory) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An I / O (Input / Output) interface 405 is also connected to the bus 404.

[0148] A plurality of components in the device 400 are connected to the I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a magnetic disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the device 400 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0149] The computing unit 401 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any suitable processor, controller, microcontroller, etc. The computing unit 401 executes the various methods and processes described above, such as the interaction method of the vehicle. For example, in some embodiments, the interaction method of the vehicle can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 400 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the computing unit 401, one or more steps of the method described above can be executed. Alternatively, in other embodiments, the computing unit 401 can be configured to execute the aforementioned vehicle interaction method in any other suitable manner (e.g., by means of firmware).

[0150] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chip), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0151] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program codes can be executed entirely on the machine, partially on the machine, executed partially on the machine and partially on a remote machine as an independent software package, or executed entirely on a remote machine or server.

[0152] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a RAM, a ROM, an EPROM (Electrically Programmable Read-Only-Memory), or a flash memory, an optical fiber, a CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0153] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or an LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0154] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0155] A computer system can include a client and a server. The client and the server are generally far from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS" for short). The server can also be a server of a distributed system, or a server combined with blockchain.

[0156] It should be noted that artificial intelligence is a discipline that studies enabling a computer to simulate certain thinking processes and intelligent behaviors of humans (such as learning, reasoning, thinking, planning, etc.), and it has both hardware-level technologies and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include several major directions such as computer vision technology, speech recognition technology, natural language processing technology, and machine learning / deep learning, big data processing technology, and knowledge graph technology.

[0157] It should be understood that various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and no limitations are imposed herein.

[0158] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. An interaction method for a vehicle, characterized in that, Including: Receiving an interaction request for human-computer interaction; Identifying the fusion-type information in the interaction request to obtain a to-be-executed task corresponding to the interaction request, where the fusion-type information includes at least one interaction input information; Invoking at least one target plugin corresponding to the to-be-executed task in a preset plugin system, and respectively performing parallel processing on the corresponding to-be-executed task through the at least one target plugin; Outputting and displaying the task execution result of the to-be-executed task.

2. The method according to claim 1, characterized in that, The invoking at least one target plugin corresponding to the to-be-executed task in the preset plugin system includes: Invoking a generative pre-trained model in the preset plugin system to find a target scenario corresponding to the to-be-executed task; different scenarios and their corresponding plugins are preset in the preset plugin system, and different scenarios may correspond to the same or different plugins; According to the corresponding relationship between the scenarios and plugins in the generative pre-trained model, finding at least one target plugin corresponding to the to-be-executed task.

3. The method according to claim 2, characterized in that, The finding at least one target plugin corresponding to the to-be-executed task according to the corresponding relationship between the scenarios and plugins in the generative pre-trained model includes: Respectively obtaining the interface addresses of the at least one target plugin from the corresponding relationship between the scenarios and plugins in the generative pre-trained model; Respectively finding the corresponding target plugins according to the respective interface addresses.

4. The method according to claim 2, characterized in that, Before finding the target plugin corresponding to the to-be-executed task according to the corresponding relationship between the scenarios and plugins, the method includes: In response to a request message for registering a plugin, the preset plugin system sends the request message for registering the plugin to the generative pre-trained model according to the plugin protocol, and the request message includes the life cycle of the registered plugin and the corresponding belonging scenario; The generative pre-trained model obtains the belonging scenario and adds the registered plugin and the belonging scenario to the corresponding relationship between the plugins and the scenarios; Monitoring the registered plugin according to the life cycle; After the life cycle arrives, canceling the registration of the registered plugin and deleting the registered plugin from the corresponding relationship between the plugins and the scenarios.

5. The method according to claim 1, characterized in that, The outputting and displaying the task execution result of the to-be-executed task includes: Obtaining the input category of the fusion-type information, where the input category of the fusion-type information includes at least one of picture, voice, gesture, video, and gaze; Displaying the processing results of the at least one target plugin in a display style matching the input category.

6. The method according to claim 1, characterized in that, The outputting and displaying the task execution result of the to-be-executed task includes: Parsing the processing results of the at least one target plugin to determine the marker information carried in the processing results, and the processing results carry marker information of directly controlling the vehicle state / indirectly controlling the vehicle state; If it is determined that the marker information is directly controlling the vehicle state, outputting the execution result information corresponding to controlling the vehicle state; If it is determined that the marker information is indirectly controlling the vehicle state, invoking a user interface container and constructing a user interface view corresponding to the processing results based on the user interface container. Output and display the user interface view.

7. The method according to claim 6, characterized in that, After outputting and displaying the user interface view, the method includes: In response to an interaction instruction triggered by the user in the user interface view, determine the interaction instruction as an execution event and send it to a scheduling module. The scheduling module responds to the interaction instruction to complete multi-round interactions. The scheduling module performs interactions by invoking the preset plug-in system.

8. An interaction device for a vehicle, characterized in that, It includes: A receiving unit for receiving interaction requests for human-computer interaction; An identifying unit for identifying the fusion information in the interaction request according to a preset processing model established in advance to obtain the to-be-executed task corresponding to the interaction request. The fusion information includes at least one interaction input information; An invoking unit for invoking at least one target plug-in corresponding to the to-be-executed task in the preset plug-in system; A processing unit for respectively performing parallel processing on the corresponding to-be-executed tasks through the at least one target plug-in; An output unit for outputting and displaying the task execution result of the to-be-executed task.

9. A vehicle, characterized in that, The vehicle includes the interaction device of the vehicle as described in claim 8.

10. An electronic device, characterized in that, It includes: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to execute the method described in any one of claims 1-7.

11. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to execute the method described in any one of claims 1-7.

12. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images