Interaction methods, devices, equipment, media and vehicles

By acquiring query requests and integrating target tools and memory information to generate prompt words, the problem of low accuracy in response statements of large language models is solved, and more accurate and personalized response statements are generated.

CN123115299APending Publication Date: 2026-11-17BEIJING CO WHEELS TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202510639392.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-05-16
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing large-scale language models fail to adequately consider users' long-term interests and background information when generating response statements, resulting in low accuracy of responses.

Method used

By acquiring the query request, identifying the target tool and target memory, and integrating the query request, the results of the target tool's operation, and the target memory, prompt words are generated and input into the language model to generate response statements.

Benefits of technology

It improves the accuracy and completeness of response statements, ensures the integration of personalized features of input information, and enhances the response quality of the language big data model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123115299A_ABST
    Figure CN123115299A_ABST
Patent Text Reader

Abstract

This application discloses an interaction method, apparatus, device, medium, and vehicle. The method includes: acquiring a query request; acquiring the query request; determining a target tool corresponding to the query request and a target memory associated with the query request based on the query request; fusing the target memory, the query request, and the execution result of the target tool to obtain a prompt word; and generating a response statement corresponding to the prompt word based on a target model. According to embodiments of this application, the accuracy of response statements from a large language model can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application belongs to the field of language model technology, and in particular relates to an interaction method, device, equipment, medium and vehicle. Background Technology

[0002] During the dialogue between the user and the language model, after the user enters a query, the electronic device usually uses the query directly as input to the language model to generate a response to send to the user.

[0003] However, in this method of generating response statements, the language big model cannot fully consider the user's long-term interests and background information, nor can it obtain real data support. Therefore, it is easy to miss key information, affecting the accuracy and depth of the response, resulting in low accuracy of response statements generated by the language big model based on prompt words. Summary of the Invention

[0004] This application provides an interactive method, apparatus, device, medium, and vehicle that can solve the problem of low accuracy of response statements in existing large language models.

[0005] In a first aspect, embodiments of this application provide an interaction method, the method comprising:

[0006] Get the query request;

[0007] The target tool corresponding to the query request and the target memory associated with the query request are determined based on the query request.

[0008] The target memory, the query request, and the results of the target tool's operation are combined to obtain the prompt words;

[0009] Generate the corresponding response statements based on the target model.

[0010] In some embodiments, before fusing the target memory, the query request, and the execution result of the target tool to obtain the prompt words, the method further includes:

[0011] When the query request type is task-based, the query request is converted into structured information using a transformation model.

[0012] The target model is used to generate a calling function based on the structured information;

[0013] The target tool corresponding to the query request is executed by calling the function to obtain the execution result.

[0014] In some embodiments, before fusing the target memory, the query request, and the execution result of the target tool to obtain the prompt words, the method further includes:

[0015] If the query request type is question-and-answer type, identify the key information in the query request;

[0016] Based on the key information, determine the target tool corresponding to the query request, and generate the target parameters corresponding to the target tool;

[0017] The target tool is run according to the target parameters to obtain the running results.

[0018] In some embodiments, the interaction method is applied to an electronic device, and the target tool includes at least one of an aggregation search tool, an application of the electronic device, and an external tool embedded in the target model.

[0019] In some embodiments, determining the target memory associated with the query request based on the query request includes:

[0020] If a user identifier associated with the query request is detected, the memory information corresponding to the user identifier is determined as the target memory associated with the query request;

[0021] If no user identifier associated with the query request is detected, the general memory information is identified as the target memory associated with the query request.

[0022] In some embodiments, before determining the memory information corresponding to the user identifier as the target memory associated with the query request upon detecting the user identifier associated with the query request, the method further includes:

[0023] The interaction information between the user identified by the user identifier and the electronic device, the user's characteristic information, and the device information of the electronic device are obtained.

[0024] The interaction information, feature information, and device information are processed by the target model to generate memory information corresponding to the user identifier.

[0025] In some embodiments, fusing the target memory, the query request, and the execution result of the target tool to obtain prompt words includes:

[0026] Determine the business configuration corresponding to the business category of the query request;

[0027] Based on the characteristics of the business configuration, determine at least one instruction and instruction framework corresponding to the query request;

[0028] The query request, the target memory, the execution result, and at least one instruction are combined according to the instruction framework to obtain the prompt word.

[0029] Secondly, embodiments of this application provide an interaction method, the method comprising:

[0030] Get the query request;

[0031] The operation indicated by the query request is broken down to obtain at least one sub-task;

[0032] The execution module corresponding to at least one subtask is determined based on the characteristics of each subtask;

[0033] The execution module executes the at least one subtask to obtain the execution result of the at least one subtask;

[0034] A response statement for the query request is generated based on at least one of the execution results of the at least one subtask.

[0035] In some embodiments, the at least one subtask includes at least one of the following:

[0036] Determine the target tool corresponding to the query request based on the query request;

[0037] Run the target tool and obtain the results of its execution.

[0038] Retrieve the target memory associated with the query request;

[0039] The target memory, the query request, and the execution result are combined to obtain the prompt words;

[0040] The prompt words are input into the target model to obtain the response statement output by the large language model.

[0041] In some embodiments, executing the at least one subtask via the execution module includes:

[0042] Obtain the dependencies between the at least one subtask;

[0043] The execution order of the at least one subtask is determined based on the dependencies;

[0044] The execution module executes the at least one subtask in the execution order.

[0045] Thirdly, embodiments of this application provide an interactive device, the device comprising:

[0046] The first acquisition module is used to acquire query requests;

[0047] The first determining module is used to determine the target tool corresponding to the query request and the target memory associated with the query request based on the query request.

[0048] The fusion module is used to fuse the target memory, the query request, and the running results of the target tool to obtain prompt words;

[0049] The response module is used to generate response statements corresponding to the prompt words based on the target model.

[0050] Fourthly, embodiments of this application provide an interactive device, the device including: a processor and a memory storing computer program instructions;

[0051] The processor implements the above interactive method when executing computer program instructions.

[0052] Fifthly, embodiments of this application provide a computer storage medium storing computer program instructions, which, when executed by a processor, implement the above-described interaction method.

[0053] Sixthly, embodiments of this application provide a computer program product, the computer program product including computer program instructions, which, when executed by a processor, implement the above-described interaction method.

[0054] In this application, after obtaining the user's query request, the request type can be queried, and an appropriate tool can be invoked based on the query request to obtain the tool's execution results, while simultaneously retrieving relevant memory data. Then, this information is fused to generate prompt words, which are then input into a language model to generate a response statement. This solution, by fusing query requests, target memory, and execution results, ensures that the information input to the language model is complete, accurate, and incorporates the user's personalized characteristics, thereby improving the accuracy of the response statement. Attached Figure Description

[0055] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a flowchart illustrating an embodiment of the interaction method provided in this application;

[0057] Figure 2 This is a schematic diagram of the structure of an interactive device provided in an embodiment of this application;

[0058] Figure 3 This is a schematic diagram of the hardware structure of an interactive device provided in an embodiment of this application. Detailed Implementation

[0059] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples of this application.

[0060] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0061] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The embodiments will now be described in detail with reference to the accompanying drawings.

[0062] Specifically, in order to solve the problems of the prior art, embodiments of this application provide an interaction method, apparatus, device, medium, and vehicle. The interaction method provided by the embodiments of this application will be described first below.

[0063] Figure 1 A flowchart illustrating an embodiment of the interaction method provided in this application is shown. This interaction method can be used in vehicles, smartphones, or other smart devices with interactive functions. The method includes the following steps:

[0064] S110, retrieve query request.

[0065] In this embodiment, the query request is input by the user and can include information in different modalities to instruct the user to obtain information or perform an operation. Specifically, the query request can be text, voice, image, or video information input by the user. After the system receives the query request, it can input the query request into the target model. The target model can determine the user's intent by analyzing the query request, and thus determine what operation the user wants the system to perform or what information to provide. The target model can be a large language model.

[0066] For example, when the language model is a model in a vehicle, the query request could be "check today's weather", the voice command "open the car window", or an image.

[0067] S120, determine the target tool corresponding to the query request and the target memory associated with the query request based on the query request.

[0068] In this embodiment, after the system receives a query request from the user, it can classify the query request to obtain the request type. Then, based on the request type and the content of the query request, the operation indicated by the query request is planned, the target tool to be invoked is determined, and then the target tool is run to obtain the result of the target tool's operation.

[0069] Specifically, query requests can be categorized into three types: question-and-answer requests, casual conversation requests, and task-based requests. Question-and-answer requests involve users asking specific questions and expecting accurate answers from the system. Examples include "How's the weather today?" or "Which team will win the 2022 World Cup?" Casual conversation requests involve relaxed dialogue between the user and the system, not aimed at solving a problem. For example, "How are you feeling today?" Task-based requests involve users requesting the system to perform a specific task or operation. This input typically has a clear objective, requiring the system to understand the user's needs and execute a specific action. Examples include "Set an alarm for 7 AM every morning" or "Play my favorite music."

[0070] For example, if the query request is a casual chat request, there is no need to invoke the target tool; if the query request is a task request, the target tool for the query request can be an application in the electronic device, or an external tool embedded in the electronic device, such as a map application, music application, calculator, or calendar; if the query request is a question-and-answer request, the target tool can be an aggregated search database in the electronic device.

[0071] For example, if the query request is a casual chat-like request like "How are you feeling today?", then there is no need to invoke any target tool; if the query request is a task-like request like "Set an alarm for me at 7 a.m. every morning", then the external tool alarm clock can be identified as the target tool to be invoked, and a 7 a.m. reminder sound can be set in the alarm clock; if the query request is a question-and-answer type request like "Which team won the 2022 World Cup?", then the aggregated search database can be identified as the target tool to be invoked, and the search for the 2022 World Cup champion team can be performed by invoking the aggregated search database.

[0072] Furthermore, target memory can include long-term memory, short-term memory, and memory summaries. Electronic devices may include memory modules that store long-term memory. Long-term memory can include a user's historical behavior, interests and preferences, and past conversation records, etc.

[0073] After an electronic device receives a query from a user, its system can retrieve relevant long-term memory data associated with that query from its memory module. This long-term memory retrieval process can be performed using methods such as keyword matching and vector matching to ensure that the retrieved long-term memory data is highly relevant to the query.

[0074] For example, if a user's query is "Please retrieve my past food order information", the system can find specific content related to "food order" from the user's long-term memory through keyword matching or vector matching.

[0075] Short-term memory can be memory information that is close in time to the time of the query retrieval, or the context information of the query. For example, short-term memory can be memory information within 30 minutes before the retrieval time, or it can be the ten rounds of dialogue between the user and the interactive device before the query.

[0076] Memory summaries are the content generated by the language big model after semantic fusion and generalization of long-term and short-term memories.

[0077] S130, the target memory, the query request, and the running result of the target tool are fused together to obtain the prompt word.

[0078] In this embodiment, after determining the target tool corresponding to the query request, the target tool can be invoked, and corresponding parameters can be generated based on the content of the query request. The target tool can then be run according to the parameters to obtain the running result of the target tool.

[0079] After acquiring the target memory associated with the query statement, the query statement, target memory, and the results of the target tool's execution can be combined to generate a complete prompt. This prompt is then used as input to the target model. Specifically, the fusion process of target memory, query statement, and execution results can be described as integrating the user's target memory with the current query statement and execution results to form a complete prompt. The target model can be a large language model.

[0080] For example, if a user enters the query "Navigate me to the restaurant I visited last time," the system retrieves from its memory platform that the user previously visited restaurant A. The system can then determine that the target tool for this query request is a map application. Therefore, it can use the map application to locate the address of restaurant A and navigate to it. The resulting prompt could be: "The user is instructing navigation to their last restaurant. According to their history, their last restaurant was restaurant A, which they have already navigated to using the map application. Please generate a relevant response."

[0081] S140, Generate the response statement corresponding to the prompt word based on the target model.

[0082] In this embodiment, the target model can be a large language model, which is an artificial intelligence model capable of understanding the input text content and generating response statements. After fusing the target memory, query request, and the results of the target tool to obtain prompt words, the system can input the generated prompt words into the pre-trained large language model. The large language model can then generate response statements based on the information in the prompt words and the context. Here, the large language model refers to a generative model based on natural language processing that can generate coherent dialogue-like response statements based on the input text.

[0083] Before applying the large-scale language model, it needs to be trained. Specifically, this can begin by collecting a large amount of text data, which may include books, encyclopedias, news articles, articles, and user history conversations. This text data can then be cleaned, labeled, and formatted to obtain the training set for the large-scale language model.

[0084] Then, the untrained language model is initialized by setting its weight parameters to random values. Next, the training data from the training set is input into the language model in batches, yielding the prediction results for each batch of training data output by the language model.

[0085] For each batch of training data, the actual answers are obtained, and the actual answers are compared with the predicted results. The loss between the actual answers and the predicted results is calculated, and then the gradient of the model parameters is calculated based on the loss and propagated to each layer of the neural network. Finally, the model parameters of the large language model are adjusted through an optimization algorithm to minimize the loss.

[0086] The training data of all batches in the training set can be iteratively trained using the above method. When the loss value drops to within the allowable error range during the training process, the language model can be considered to have been trained and the training of the language model can be ended.

[0087] Furthermore, after obtaining the prompt words, the output of the response can be controlled to have different formats or styles. For example, specific tags can be inserted into the language model before it generates the response, and these tags are used to control the format of the output response. For instance, tag 1 can control the response to be in a general-to-specific structure, making the output response clearer and easier to read; tag 2 can indicate that the response style is more child-friendly, making the response more vivid and interesting.

[0088] In addition, after the system receives the reply statement, it can filter out inappropriate, illegal, or meaningless content in the reply statement and output the filtered reply statement to the user.

[0089] In this embodiment, after obtaining the user's query request, the request type can be queried, and an appropriate tool can be invoked based on the query request to obtain the tool's execution results, while simultaneously retrieving relevant memory data. Then, this information is fused to generate prompt words, which are then input into the language model to generate a response statement. This solution, by fusing query requests, target memory, and execution results, ensures that the information input to the language model is complete, accurate, and incorporates the user's personalized characteristics, thereby improving the accuracy of the response statement.

[0090] As an optional embodiment, before fusing the target memory, the query request, and the execution result of the target tool to obtain the prompt words, the method further includes:

[0091] When the query request type is task-based, the query request is converted into structured information using a transformation model.

[0092] The target model is used to generate a calling function based on the structured information;

[0093] The target tool corresponding to the query request is executed by calling a function, and the execution result is obtained.

[0094] In this embodiment, when the query request type is task-oriented, the user wants the system to perform a specific operation, such as "open the car window" or "set an alarm clock." Therefore, the system can first parse the user's query request into structured information, such as a DAS (Domain, Action, Slots) command; then, the system generates a calling function based on the structured information to invoke the target tool corresponding to the query request to perform the appropriate operation.

[0095] For example, after the system receives a user's query request, it can use a link routing model based on BERT (Bidirectional Encoder Transformer) to process the query request and determine whether it is a task-oriented, question-answering, or other type of request based on the processing results. If the query request is detected as task-oriented, the DAS prediction model is used to analyze the input query request and extract key information from it. Key information can include the domain, action, and slot of the query request. The domain refers to the category of the operation to be performed, the action refers to the specific operation requested by the user, and the slot refers to the additional information required to perform the operation. The additional information refers to some parameter details that must be specified to complete the action, such as object, time, location, and content. Structured information can be generated based on this extracted key information. Then, a language big data model can be used to generate a calling function based on the structured information. The calling function is used to call the target tool and run the target tool to obtain the result.

[0096] For example, if a user enters a task-oriented query request such as "open car window", then the domain of the query request can be extracted as "vehicle-related", the action as "open", and the additional information as "car window". The query request can then be converted into structured information, which is the DAS instruction (vehicle-related, open, car window). Subsequently, the DAS instruction can be converted into a callable function format, and the operation of opening the car window can be implemented through function calls.

[0097] For example, if a user enters a task-oriented query request, "Set my alarm for 8 AM tomorrow," then the domain of the query request can be extracted as "Schedule Management," the action as "Set Reminder," and the additional information as "Specific Time." The query request can then be converted into structured information, which is the DAS instruction (Schedule Management, Set Reminder, 8 AM). Subsequently, the DAS instruction can be converted into a callable function format, and the reminder setting operation can be implemented through function calls.

[0098] In this way, by converting task-oriented queries into structured information and calling the corresponding functions, the appropriate tools can be automatically selected and executed, thereby improving the efficiency of the system and the accuracy of task execution.

[0099] As an optional embodiment, before fusing the target memory, the query request, and the execution result of the target tool to obtain the prompt words, the method further includes:

[0100] If the query request type is question-and-answer type, identify the key information in the query request;

[0101] Based on the key information, determine the target tool corresponding to the query request, and generate the target parameters corresponding to the target tool;

[0102] The target tool is run according to the target parameters to obtain the running results.

[0103] In this embodiment, when the query request type is question-and-answer type, the user's goal is to obtain an answer to a question. The system needs to query relevant information based on the core content of the question and return the answer. For example, a question-and-answer type query request could be "How is the weather today?". In this case, the system can identify the key information in the query request, select the appropriate target tool based on the key information, and generate the target parameters required to execute the target tool. Subsequently, the target tool and target parameters can be standardized into a unified API (Application Programming Interface) format, thereby calling the target tool based on the target parameters to obtain the execution result.

[0104] For example, when a user enters a question-and-answer type query request, "What's the weather like today?", the target tool can be identified as a weather query API by the key information in the query request. The target parameters include location and date, etc. Subsequently, the system sends a query request to the target tool through the API and receives the query result returned by the target tool. This query result is the result of the target tool's operation.

[0105] In this way, when the query request is question-and-answer type, the appropriate tool can be automatically selected based on the query request, and the corresponding parameters can be generated to execute the task. This standardized process can improve the efficiency of query request processing.

[0106] As an optional embodiment, the interaction method is applied to an electronic device, and the target tool includes at least one of an aggregation search tool, an application of the electronic device, and an external tool embedded in the target model.

[0107] In this embodiment, the aggregated search tool is a search tool that combines multiple information sources. The aggregated search tool can perform searches through external search engines or within a self-built knowledge base on the electronic device. This self-built knowledge base can include reading comprehension search and matching search. External tools embedded in the language model refer to external tools integrated into the language model to enhance its functionality and capabilities. External tools are independent applications or services integrated into the language model to enhance its functionality and reliability; these can include weather forecast tools, calculators, and code interpreters. Applications on the electronic device can include map applications, music applications, and video playback applications.

[0108] In this way, targeting aggregated search tools, electronic device applications, and embedded external tools can enhance the system's flexibility and expand the functionality of the large language model.

[0109] As an optional embodiment, determining the target memory associated with the query request based on the query request includes:

[0110] If a user identifier associated with the query request is detected, the memory information corresponding to the user identifier is determined as the target memory associated with the query request;

[0111] If no user identifier associated with the query request is detected, the general memory information is identified as the target memory associated with the query request.

[0112] In this embodiment, the user identifier is used to identify the user's identity information. The user identifier may include a username, user ID, and the user's mobile phone number, etc. If the system can detect the user identifier associated with the query request, the system can search for the personalized memory information corresponding to that user identifier and determine the personalized memory information as the target memory associated with the query request. The memory information corresponding to the user identifier is based on the user's historical interactions, preferences, and needs recorded with the electronic device.

[0113] If the system cannot detect the user identifier associated with the query request, it cannot determine which user issued the query request, i.e., it cannot identify the user who sent the query request. In this case, the system determines the target memory associated with the query request as general memory information. This general memory information is generated based on the collective behavior, common preferences, and shared knowledge of all users.

[0114] By using the above method, the system can decide whether to use personalized memory or general memory based on whether a user identifier is detected. This ensures that when the language model interacts with a user, it can provide customized responses based on the known user identity, and can still provide standardized services when the user identity cannot be identified, thereby improving the flexibility of the system response.

[0115] As an optional embodiment, before determining the memory information corresponding to the user identifier as the target memory associated with the query request upon detecting the user identifier associated with the query request, the method further includes:

[0116] The interaction information between the user identified by the user identifier and the electronic device, the user's characteristic information, and the device information of the electronic device are obtained.

[0117] The interaction information, feature information, and device information are processed by the target model to generate memory information corresponding to the user identifier.

[0118] In this embodiment, the system can collect device information, user characteristic information, and user-electronic interaction information from various sensors and modules of the electronic device. Device information refers to the hardware or software attributes of the electronic device, such as its model, system version, or sensor status. Interaction information refers to the user's operational or input / output behaviors with the electronic device, such as clicking on the device or sending voice commands. Characteristic information refers to the user's attributes or behavioral characteristics, such as their name, voiceprint features, preferences, or historical behavior records.

[0119] For example, when the electronic device is a vehicle, the interactive information can include image information, voice information, text information, etc., received by the vehicle; the device information can include vehicle control signals and sensor signals, etc., which are vehicle-related signals. Among them, image information can include images captured by the vehicle's camera; voice information can be voice commands for user-vehicle interaction; text information can be text input by the user in a system such as an in-vehicle navigation system; vehicle control signals can be user signals such as steering, acceleration, and braking; and sensor signals can be signals collected by sensors in the vehicle, such as temperature, pressure, and GPS positioning.

[0120] Whenever an electronic device acquires interactive information, feature information, or device information, it can perform standardized processing on this information in real time according to preset specifications. Standardization processing includes cleaning, normalizing, and format alignment of these signals to ensure data consistency and usability. For example, images captured by in-vehicle cameras are processed into a uniform format, and output data from in-vehicle sensors are calibrated or aligned to the timeline.

[0121] Next, the electronic device incrementally processes the standardized information, continuously updating and integrating existing information based on newly acquired data. This incrementally processed information is categorized according to different dimensions, such as time, event type, or user behavior habits. Through incremental processing, user behavior sequence data is formed. This data records the user's actions and habits over a short period, allowing the system to extract personalized memory information for that user.

[0122] In the above solution, by acquiring user and electronic device interaction information, user characteristic information, and electronic device device information, and standardizing these data, followed by incremental processing and classification, personalized memory information is formed for the user. This memory information corresponds to the user's user identifier. In this way, by combining personalized memory information with the user's query statements, the electronic device can provide a more accurate and intelligent interactive experience based on the user's historical behavior and the electronic device's historical records.

[0123] As an optional embodiment, the step of fusing the target memory, the query request, and the execution result of the target tool to obtain prompt words includes:

[0124] Determine the business configuration corresponding to the business category of the query request;

[0125] Based on the characteristics of the business configuration, determine at least one instruction and instruction framework corresponding to the query request;

[0126] The query request, the target memory, the execution result, and at least one instruction are combined according to the instruction framework to obtain the prompt word.

[0127] In this embodiment, after obtaining the target memory and the running results of the target tool, the query request, the target memory and the running results can be combined to obtain the prompt words, and the prompt words can be input into the language big model to obtain the response statement of the language big model.

[0128] Specifically, after an electronic device receives a query request from a user, it can identify the business category to which the query request belongs, and then determine at least one instruction and instruction framework corresponding to that business category. An instruction refers to a predefined command in the device, while an instruction framework refers to a template or structure that guides how to combine the input query request, target memory, execution result, and instruction to form a complete prompt.

[0129] Furthermore, instructions can include dynamic instructions and static instructions. Static instructions are predefined, fixed, and unchanging. These instructions are determined during system design and do not change based on input statements or the environment; they are used to guide large models to complete specific understanding or generation tasks. Dynamic instructions are generated based on real-time input, environment, or contextual information. After determining at least one instruction and instruction framework corresponding to the business category of the query request, the query request, at least one instruction, target memory, and execution results are combined according to this instruction framework to obtain a complete prompt.

[0130] After generating the prompt words, they can be input into a pre-trained language model. The language model will then generate a response based on the prompt words that is relevant to the user's input. The language model is a natural language processing model used to understand and generate natural language.

[0131] In this way, in the embodiments of this application, the appropriate instruction and instruction framework can be automatically selected according to the business category of the query request entered by the user. The instruction, query request, target memory and the running results of the target tool are combined according to the instruction framework to quickly generate accurate prompt words. This greatly improves the accuracy of the prompt words and helps the language model output more accurate response statements.

[0132] Furthermore, one embodiment of this application provides an interaction method. This interaction method can be used in vehicles, smartphones, or other smart devices with interactive functions. The method includes the following steps:

[0133] Get the query request;

[0134] The operation indicated by the query request is broken down to obtain at least one sub-task;

[0135] The execution module corresponding to at least one subtask is determined based on the characteristics of each subtask;

[0136] The execution module executes the at least one subtask to obtain the execution result of the at least one subtask;

[0137] A response statement for the query request is generated based on at least one of the execution results of the at least one subtask.

[0138] In this embodiment, a query request can be a question, an instruction, or a statement with an operational intent. After the system receives a query request input by the user, it can perform semantic parsing on the query request to identify the intent of the query request, determine the operation indicated by the query request based on the intent, then break down the operation to obtain at least one subtask, and determine the execution order of at least one subtask.

[0139] After determining at least one subtask and the execution order of each subtask, predefined expert modules or other modules can be called as execution modules for each subtask based on the goal and content of each subtask. Each execution module can independently complete the subtask it is responsible for, and return the execution result of the subtask after it is completed.

[0140] At least one subtask may include: determining the target tool corresponding to the query request based on the query request; running the target tool and obtaining the running result of the target tool; obtaining the target memory associated with the query request; fusing the target memory, the query request, and the running result to obtain the prompt words; and inputting the prompt words into the target model to obtain the response statement output by the language big model.

[0141] Because at least one subtask has upstream and downstream dependencies, the system can construct a response path to the query request by executing these subtasks sequentially: First, the target tool to be invoked is determined to provide external data support for the query request; then, the target memory associated with the query request is obtained to supplement user historical preferences or contextual information; next, the target tool is run to obtain its results; finally, the query request itself, the target tool's results, and the target memory are semantically fused to generate structured prompts, which are then input into the language model to drive the model to generate a complete, semantically accurate, and user-intent-aligned response. Throughout this process, the execution results of the subtasks are progressive and interdependent, forming an ordered processing chain from input understanding to response generation, ultimately yielding the response to the query request.

[0142] For example, after acquiring at least one subtask, a gating network in the MOE (Mixture of Experts) can be used to select a suitable expert module or other module as the execution module for each subtask. The Mixture of Experts is a dynamically routed neural network architecture composed of multiple expert modules, and a gating network determines which expert modules should handle each subtask. Each expert module is a neural network model used to process a specific task.

[0143] For example, a hybrid expert module may include a memory expert module and a tool management expert module, while an electronic device may also include an engineering module. A gating network can choose to invoke the memory expert module to acquire the target memory, choose to invoke the tool management expert module to invoke the external target tool, and choose to use the engineering module to complete the fusion of prompt words.

[0144] Furthermore, since each expert module in the hybrid expert module is responsible for executing different subtasks, after determining at least one subtask, different subtasks can be executed by different expert modules. Then, the weights of the execution results of the outputs of each expert module are calculated using the gating network in the hybrid expert module, and the execution results of the subtasks output by different expert modules are then weighted and fused based on the weights.

[0145] For example, in the process of prompt word fusion, the target memory, query request, and the output of the target tool are all execution results of different subtasks. These execution results can be weighted and fused to ensure that the constructed prompt words can accurately guide the language model to generate the optimal response. For example, if the confidence level of the execution result returned by the target tool is high, it is assigned a higher weight; if the target memory is frequently used memory information, it is assigned a higher weight. For instance, the weight of the target memory can be set to 0.3, the weight of the query request to 0.5, and the weight of the execution result to 0.2. According to the above weights, the target memory, query request, and execution result can be weighted and fused to obtain the final prompt words.

[0146] In this way, the operation indicated by the query request can be broken down into at least one subtask, and each subtask can be handled by the most appropriate expert module, thereby improving the system's response speed and the accuracy of the response statement.

[0147] As an optional embodiment, the at least one subtask includes at least one of the following:

[0148] Determine the target tool corresponding to the query request based on the query request;

[0149] Run the target tool and obtain the results of its execution.

[0150] Retrieve the target memory associated with the query request;

[0151] The target memory, the query request, and the execution result are combined to obtain the prompt words;

[0152] The prompt words are input into the target model to obtain the response statement output by the large language model.

[0153] In this embodiment, the electronic device may include an environment module, an engineering module, a hybrid expert module, and an expert router. The environment module receives multimodal input from users in the environment. The engineering module manages the prompts in the large language model. The hybrid expert module may include a memory expert module, a planning expert module, a multimodal expert module, a general language expert module, and a tool management expert module. The memory expert module manages the memory information in the electronic device. The planning expert module breaks down the operation indicated by the query request to obtain at least one sub-task and determines the execution sequence of at least one sub-task. The multimodal expert module processes non-verbal modal information such as speech and images, and uniformly generates a semantic representation in text form. The general language expert module understands the prompts based on the large language model and generates response statements. The tool management expert module manages tools. The expert router selects the most suitable execution module for each sub-task from the above modules.

[0154] Specifically, after the user enters a query request, the query request can be obtained through the environment module; then the planning expert module breaks down the operation indicated by the query request to obtain at least one sub-task.

[0155] The expert router determines the execution module corresponding to each subtask; then, multiple execution modules execute their respective subtasks, ultimately generating the response statement corresponding to the prompt word.

[0156] In this embodiment, the multiple execution modules may include a tool management expert module, a memory expert module, an engineering module, and a general language expert module:

[0157] The tool management module can determine the target tool corresponding to the query request and run the target tool to obtain the running result; the memory expert module can obtain the target memory associated with the query request; the engineering module can integrate the target memory, query request and the running result of the target tool to obtain the prompt word; the general language expert module can input the prompt word into the target model to obtain the response statement output by the target model, which is a large language model.

[0158] In this way, by decomposing and coordinating query requests across multiple expert modules, the entire process of tool execution, memory retrieval, and semantic fusion can be automated, significantly improving the efficiency of prompt word generation and thus improving the efficiency of response statement generation.

[0159] Furthermore, as an optional embodiment, the execution of the at least one subtask via the execution module includes:

[0160] Obtain the dependencies between the at least one subtask;

[0161] The execution order of the at least one subtask is determined based on the dependencies;

[0162] The execution module executes the at least one subtask in the execution order.

[0163] In this embodiment, after the system receives a query request input by the user, the operation indicated by the query request can be broken down by a planning expert to obtain at least one subtask, and the dependencies between the at least one subtask can be determined. Then, the execution order between the at least one subtask can be set according to the dependencies between the at least one subtask.

[0164] For example, a subtask may include determining the target tool corresponding to the query request, running the target tool to obtain the result, acquiring the target memory associated with the query request, and fusing the target memory, the query request, and the result to obtain a prompt word. Running the target tool depends on determining the target tool corresponding to the query request, while running the target tool and acquiring the target memory are parallel processes, and the fusion of the prompt word depends on running the target tool and acquiring the target memory. Therefore, the planning expert module can construct a processing chain for the subtasks based on the dependencies between them, and execute at least one subtask sequentially according to the execution order of the processing chain, ultimately obtaining the prompt word.

[0165] Specifically, the planning expert module can be set to first execute the target tool corresponding to the query request, then simultaneously execute the execution of the target tool and the acquisition of target memory, and only execute the fusion of prompt words if both the execution status of the target tool and the acquisition of target memory are successful.

[0166] In this way, by splitting tasks through query requests and arranging the execution order of subtasks reasonably based on dependencies, the execution efficiency and accuracy of tasks can be optimized.

[0167] In addition, as an optional embodiment, the system can reflect after each subtask is completed, so that the system can automatically summarize the lessons learned in the execution process after each subtask is executed, and dynamically adjust the planning strategy for subsequent tasks in combination with the planning update function, thereby continuously improving the flexibility of interaction.

[0168] Specifically, after each subtask is executed, the system can reflect on the execution process based on the subtask's results, user feedback, and pre-defined evaluation metrics. This helps identify shortcomings and automatically adjust subsequent subtask allocation, prompt generation logic, or expert activation strategies, enabling continuous self-optimization. The subtask's execution results can include whether the tool call was successful, whether the execution result is complete, and whether the large language model's response meets expectations. User feedback can include the user's emotions and textual content expressed through interaction.

[0169] In addition, as an optional embodiment, the response can be adjusted based on the user's mood.

[0170] Specifically, after receiving a query request from a user, the multimodal expert module can be used to analyze the features of the query request, such as tone of voice, word choice, and emotional tendency, to extract emotional cues from the query request, determine the user's emotional state, and adjust the response statement based on the emotional state.

[0171] For example, if a user's query includes the emotional word "a little angry," it can be determined that the user's emotional state is disappointment. Then, the reply can be adjusted to respond to the user with a warmer, more encouraging, or comforting language style.

[0172] Based on the interaction method provided in the above embodiments, this application also provides specific implementations of the interaction device. Please refer to the following embodiments.

[0173] First see Figure 2 The interactive device 200 provided in this application embodiment includes the following modules:

[0174] The first acquisition module 201 is used to acquire query requests;

[0175] The first determining module 202 is used to determine the target tool corresponding to the query request and the target memory associated with the query request based on the query request.

[0176] The fusion module 203 is used to fuse the target memory, the query request, and the running results of the target tool to obtain prompt words;

[0177] The response module 204 is used to generate response statements corresponding to the prompt words based on the target model.

[0178] In this embodiment, after obtaining the user's query request, the request type can be queried, and an appropriate tool can be invoked based on the query request to obtain the tool's execution results, while simultaneously retrieving relevant memory data. Then, this information is fused to generate prompt words, which are then input into the language model to generate a response statement. This solution, by fusing query requests, target memory, and execution results, ensures that the information input to the language model is complete, accurate, and incorporates the user's personalized characteristics, thereby improving the accuracy of the response statement.

[0179] As one implementation of this application, the interactive device 200 may further include:

[0180] The conversion module is used to convert the query request into structured information through a conversion model when the query request type is task-type.

[0181] The first generation module is used to generate calling functions based on the structured information using the language big model;

[0182] The first execution module is used to run the target tool corresponding to the query request by calling a function, and obtain the execution result.

[0183] As one implementation of this application, the interactive device 200 may further include:

[0184] The identification unit is used to identify key information in the query request when the query request type is question-and-answer type.

[0185] The second generation module is used to determine the target tool corresponding to the query request based on the key information, and to generate the target parameters corresponding to the target tool;

[0186] The second running module is used to run the target tool according to the target parameters and obtain the running results.

[0187] As one implementation of this application, the first determining module 202 may further include:

[0188] The first determining unit is configured to determine the memory information corresponding to the user identifier as the target memory associated with the query request when the user identifier associated with the query request is detected.

[0189] The second determining unit is used to determine general memory information as the target memory associated with the query request when no user identifier associated with the query request is detected.

[0190] As one implementation of this application, the interactive device 200 may further include:

[0191] The second acquisition module is used to acquire the interaction information between the user identified by the user identifier and the electronic device, the user's feature information, and the device information of the electronic device;

[0192] The processing module is used to process the interaction information, feature information, and device information through the target model to generate memory information corresponding to the user identifier.

[0193] As one implementation of this application, the fusion module 203 may further include:

[0194] The first determining unit is used to determine the business configuration corresponding to the business category of the query request;

[0195] The second determining unit is used to determine at least one instruction and instruction framework corresponding to the query request based on the characteristics of the service configuration.

[0196] The combination unit is used to combine the query request, the target memory, the running result, and the at least one instruction according to the instruction framework to obtain the prompt word.

[0197] In addition, another interactive device of this application includes the following modules:

[0198] The third acquisition module is used to acquire query requests;

[0199] The splitting module is used to split the operation indicated by the query request into at least one subtask;

[0200] The second determining module is used to determine the execution module corresponding to the at least one subtask based on the characteristics of each subtask.

[0201] An execution module is used to execute the at least one subtask and obtain the execution result of the at least one subtask.

[0202] A generation module is used to generate a response statement for the query request based on at least one of the execution results of the at least one subtask.

[0203] As one implementation of this application, the above-mentioned execution module may further include:

[0204] The acquisition unit is used to acquire the dependency relationships between the at least one subtask;

[0205] A sorting unit is used to determine the execution order of the at least one subtask based on the dependency relationship;

[0206] An execution unit is configured to execute the at least one subtask in the execution order via the execution module.

[0207] The interactive device provided in this embodiment of the invention can implement the steps in the above method embodiments, and will not be repeated here to avoid repetition.

[0208] Figure 3 A schematic diagram of the hardware structure of the interactive device provided in an embodiment of this application is shown.

[0209] The interactive device may include a processor 1001 and a memory 1002 storing computer program instructions.

[0210] Specifically, the processor 1001 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0211] Memory 1002 may include mass storage for data or instructions. For example, and not limitingly, memory 1002 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 1002 may include removable or non-removable (or fixed) media. Where appropriate, memory 1002 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 1002 is non-volatile solid-state memory.

[0212] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to one aspect of this disclosure.

[0213] The processor 1001 implements any of the interaction methods described in the above embodiments by reading and executing computer program instructions stored in the memory 1002.

[0214] In one example, the interactive device may further include a communication interface 1003 and a bus 1010. For example, Figure 3 As shown, the processor 1001, memory 1002, and communication interface 1003 are connected through bus 1010 and complete communication with each other.

[0215] The communication interface 1003 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0216] Bus 1010 includes hardware, software, or both, that couples components of an interactive device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 1010 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0217] The interactive device can be based on the above embodiments to realize the combination of the above-described interactive methods and apparatus.

[0218] Furthermore, in conjunction with the interaction methods described in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the interaction methods described in the above embodiments and achieve the same technical effect. To avoid repetition, further details are omitted here. The aforementioned computer-readable storage medium may include non-transitory computer-readable storage media, such as read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, etc., and is not limited thereto.

[0219] In addition, this application also provides a computer program product, including computer program instructions, which, when executed by a processor, can implement the steps and corresponding content of the aforementioned method embodiments.

[0220] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0221] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0222] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0223] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus, and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0224] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. An interaction method, characterized in that, The method includes: Get the query request; The target tool corresponding to the query request and the target memory associated with the query request are determined based on the query request. The target memory, the query request, and the results of the target tool's operation are combined to obtain the prompt words; Generate the corresponding response statements based on the target model.

2. The interaction method according to claim 1, characterized in that, Before fusing the target memory, the query request, and the results of the target tool's operation to obtain the prompt words, the method further includes: When the query request type is task-based, the query request is converted into structured information using a transformation model. The target model is used to generate a calling function based on the structured information; The target tool corresponding to the query request is executed by calling the function to obtain the execution result.

3. The interaction method according to claim 1 or 2, characterized in that, Before fusing the target memory, the query request, and the results of the target tool's operation to obtain the prompt words, the method further includes: If the query request type is question-and-answer type, identify the key information in the query request; Based on the key information, determine the target tool corresponding to the query request, and generate the target parameters corresponding to the target tool; The target tool is run according to the target parameters to obtain the running results.

4. The interaction method according to any one of claims 1-3, characterized in that, The interaction method is applied to an electronic device, and the target tool includes at least one of an aggregation search tool, an application of the electronic device, and an external tool embedded in the target model.

5. The interaction method according to any one of claims 1-4, characterized in that, The step of determining the target memory associated with the query request based on the query request includes: If a user identifier associated with the query request is detected, the memory information corresponding to the user identifier is determined as the target memory associated with the query request; If no user identifier associated with the query request is detected, the general memory information is identified as the target memory associated with the query request.

6. The interaction method according to claim 5, characterized in that, Before determining the memory information corresponding to the user identifier as the target memory associated with the query request when the user identifier associated with the query request is detected, the method further includes: The interaction information between the user identified by the user identifier and the electronic device, the user's characteristic information, and the device information of the electronic device are obtained. The interaction information, feature information, and device information are processed by the target model to generate memory information corresponding to the user identifier.

7. The interaction method according to any one of claims 1-6, characterized in that, The step of fusing the target memory, the query request, and the execution result of the target tool to obtain prompt words includes: Determine the business configuration corresponding to the business category of the query request; Based on the characteristics of the business configuration, determine at least one instruction and instruction framework corresponding to the query request; The query request, the target memory, the execution result, and at least one instruction are combined according to the instruction framework to obtain the prompt word.

8. An interaction method, characterized in that, The method includes: Get the query request; The operation indicated by the query request is broken down to obtain at least one sub-task; The execution module corresponding to at least one subtask is determined based on the characteristics of each subtask; The execution module executes the at least one subtask to obtain the execution result of the at least one subtask; A response statement for the query request is generated based on at least one of the execution results of the at least one subtask.

9. The interaction method according to claim 8, characterized in that, The at least one subtask includes at least one of the following: Determine the target tool corresponding to the query request based on the query request; Run the target tool and obtain the results of its execution. Retrieve the target memory associated with the query request; The target memory, the query request, and the execution result are combined to obtain the prompt words; The prompt word is input into the target model to obtain the response statement output by the target model.

10. The interaction method according to claim 8 or 9, characterized in that, The execution of the at least one subtask through the execution module includes: Obtain the dependencies between the at least one subtask; The execution order of the at least one subtask is determined based on the dependencies; The execution module executes the at least one subtask in the execution order.

11. An interactive device, characterized in that, The device includes: The first acquisition module is used to acquire query requests; The first determining module is used to determine the target tool corresponding to the query request and the target memory associated with the query request based on the query request. The fusion module is used to fuse the target memory, the query request, and the running results of the target tool to obtain prompt words; The response module is used to generate response statements corresponding to the prompt words based on the target model.

12. An interactive device, characterized in that, The interactive device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the interaction method as described in any one of claims 1-10.

13. A computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed by a processor, implement the interaction method as described in any one of claims 1-10.

14. A vehicle, characterized in that, The vehicle includes at least one of the device as claimed in claim 11, the equipment as claimed in claim 12, and the medium as claimed in claim 13.