Human-machine interaction method, device and system
By combining large language models and plugin resources, executable instructions that match user interaction needs are generated, solving the problems of inaccurate machine responses and high code development costs in existing technologies, and achieving efficient and accurate human-computer interaction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- IFLYTEK CO LTD
- Filing Date
- 2023-05-30
- Publication Date
- 2026-07-24
AI Technical Summary
In existing human-computer interaction solutions, the accuracy of machine responses is not high enough, and multiple business call codes for intent adaptation need to be developed in advance, which increases the code development cost and workload.
By using a large language model to generate executable instructions that match user interaction needs, and by calling and parsing plugin resources, accurate response information is generated, reducing the development of business call code to adapt to different user needs.
It improves the accuracy of machine responses, reduces code development costs, meets the needs of various dialogue intents, and enhances the efficiency and adaptability of human-computer interaction.
Smart Images
Figure CN116483980B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of large language model technology, and more specifically, to a human-computer interaction method, device and system. Background Technology
[0002] Human-computer interaction (HCI) refers to the process of information exchange and interaction between humans and computer systems. The goal of HCI is to make computer systems more aligned with human cognitive habits and needs, thereby achieving more efficient and convenient information processing. Common HCI scenarios include task-based dialogues, knowledge-based question answering, and casual conversation.
[0003] Existing human-computer interaction (HCI) solutions primarily employ classification and slot extraction models. Through reinforcement training, they determine the user's intent and key slot information corresponding to the question. Then, customized code matching the intent is used to retrieve relevant business data from third-party business information source interfaces using the identified intent and key slot information to respond to the user. For example, when a user inquires about weather information, the model identifies the user's intent as "inquiring about weather information," with key slot information including time and location. Customized code adapted to a third-party weather application is then invoked to query specific weather information based on the key slot information, and the user is responded to based on the results from the third-party weather application. However, existing HCI solutions are limited by the model's capabilities, resulting in insufficient accuracy in intent recognition and key slot information extraction. Furthermore, the business data retrieved by the code is prone to errors. In addition, users may have diverse dialogue intents during HCI interactions, requiring pre-customized code for various intent adaptations, increasing development costs and workload. Summary of the Invention
[0004] In view of the above problems, this application is proposed to provide a human-computer interaction method, device, and system to solve the problems of insufficient accuracy of machine responses in existing technologies and the need to pre-develop various intent-adaptive business call codes, which increases code development costs and workload. The specific solution is as follows:
[0005] Firstly, a human-computer interaction method is provided, including:
[0006] Acquire historical information about human-computer interaction and the user's current input information;
[0007] The system invokes a pre-built large language model and generates executable instructions that match the user's interaction needs in the current round, based on the human-computer interaction history information and the current input information. The executable instructions include calls to the plugin resources that are relied upon to respond to the user's interaction needs in the current round.
[0008] The executable instructions are parsed and executed to invoke the plugin according to the instructions, and the execution result of the plugin is obtained.
[0009] The large language model is invoked, and based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin, a response information corresponding to the user's current input information is generated.
[0010] Preferably, a pre-set large language model is invoked, and based on the human-computer interaction history information and the current input information, executable instructions matching the user's interaction needs in this round are generated, including:
[0011] Obtain a pre-configured first prompt format template, which includes a history information slot and a current input information slot. The first prompt format template is used to instruct the large language model to combine the information in the history information slot to understand the user's interaction needs in the current round and provide matching executable instructions.
[0012] The human-computer interaction history information is filled into the history information slot, and the user's current input information is filled into the current input information slot to obtain the first prompt instruction after editing;
[0013] The edited first prompt instruction is input into the large language model to obtain the executable instruction output by the large language model.
[0014] Preferably, the executable instructions are executable Python code.
[0015] Preferably, the process of parsing and executing the executable instructions to invoke the plugin according to the instructions and obtain the execution result of the plugin includes:
[0016] The pre-configured plugin parser is invoked to parse and execute the executable instructions, so as to invoke the plugin according to the instructions of the executable instructions and obtain the execution result of the plugin.
[0017] Preferably, the large language model is invoked to generate response information corresponding to the user's current input information based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin, including:
[0018] Obtain a pre-configured second prompt format template, which includes a history information slot, a current input information slot, and a plugin information slot. The second prompt format template is used to instruct the large language model to combine the information in the history information slot and the query information slot to respond to the user's input information in the current round.
[0019] The human-computer interaction history information is filled into the history information slot, the user's current input information is filled into the current input information slot, and the execution result of the plugin is filled into the query information slot to obtain the second prompt instruction after editing;
[0020] The edited second prompt instruction is input into the large language model to obtain the response information output by the large language model.
[0021] Preferably, before invoking the large language model and generating response information corresponding to the user's current input information based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin, the method further includes:
[0022] The large language model is invoked, and the next task is planned based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin.
[0023] If the next task is to generate a response, then the step of calling the large language model and generating a response corresponding to the user's current input information based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin is executed.
[0024] Preferably, it further includes:
[0025] If the next task is a retry task, then return to the step of calling the preset large language model, and generating executable instructions that match the user's interaction needs in this round based on the human-computer interaction history information and the current input information;
[0026] And / or,
[0027] If the next task is a fallback task, then the execution result of the plugin is set to empty, and the large language model is invoked to generate the response information corresponding to the user's current input information based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin.
[0028] Preferably, the large language model is invoked, and based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin, the next task is planned, including:
[0029] Obtain a pre-configured third prompt format template, which includes a history information slot, a current input information slot, and a plugin information slot. The third prompt format template is used to instruct the large language model to combine the information in the history information slot, the current input information slot, and the plugin information slot to plan the next task.
[0030] The human-computer interaction history information is filled into the history information slot, the user's current input information is filled into the current input information slot, and the execution result of the plugin is filled into the plugin information slot to obtain the third prompt instruction after editing;
[0031] Input the edited third prompt command into the large language model to obtain the next task output by the large language model.
[0032] Preferably, the plugin resources include any one or more of the following:
[0033] Information source plugin, knowledge base plugin, personalized content recommendation plugin, toolset plugin.
[0034] Preferably, the human-computer interaction method is applied to any one of the following scenarios: task-based dialogue, knowledge-based question-and-answer, or chat dialogue.
[0035] Secondly, a human-computer interaction device is provided, comprising:
[0036] The information acquisition unit is used to acquire historical information of human-computer interaction and the user's current input information.
[0037] The first model invocation unit is used to invoke a pre-set large language model, and generate executable instructions that match the user's interaction needs in the current round based on the human-computer interaction history information and the current round input information; the executable instructions include invocation of plugin resources on which the user's interaction needs in the current round depend;
[0038] The instruction parsing and processing unit is used to parse and execute the executable instructions to call the plugin according to the instructions of the executable instructions and obtain the execution result of the plugin;
[0039] The second model invocation unit is used to invoke the large language model and generate response information corresponding to the user's current input information based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin.
[0040] Thirdly, a human-computer interaction system is provided, including:
[0041] The module includes a central control module, a large language model module, a plugin parser module, and a plugin resource module.
[0042] The scheduling control module is used to acquire human-computer interaction history information and the user's current input information; call the large language model module to generate an executable instruction matching the user's interaction needs based on the human-computer interaction history information and the current input information, wherein the executable instruction includes the invocation of plugin resources on which the user's interaction needs depend; call the plugin parser module to parse and execute the executable instruction to invoke the plugins provided by the plugin resource module according to the instructions of the executable instruction, and obtain the execution result of the plugins; and call the large language model module to generate response information corresponding to the user's current input information based on the human-computer interaction history information, the user's current input information, and the execution result of the plugins.
[0043] Fourthly, a human-computer interaction device is provided, including: a memory and a processor;
[0044] The memory is used to store programs;
[0045] The processor is used to execute the program to implement the various steps of the human-computer interaction method as described above.
[0046] Fifthly, a storage medium is provided on which a computer program is stored, which, when executed by a processor, implements the various steps of the human-computer interaction method as described above.
[0047] By employing the aforementioned technical solution, this application implements the human-computer interaction process based on a large language model and pre-configured plugin resources. It fully leverages the powerful semantic understanding and multi-turn dialogue capabilities of the large language model, as well as the internal and external resources provided by the plugins, such as information sources, knowledge bases, content recommendation resources, and toolsets. Specifically, after obtaining the historical information of human-computer interaction and the user's current input information, the large language model is invoked. Based on this historical information and current input information, an executable instruction matching the user's interaction needs for the current turn is generated. This executable instruction includes calls to the plugin resources upon which the response to the user's interaction needs depends. Thus, this application can automatically generate executable instructions matching the user's interaction needs for the current turn by utilizing the semantic understanding and code generation capabilities of the large language model. Based on this, the generated executable instructions can be parsed and executed to invoke plugins according to the instructions, obtaining the plugin execution results. Furthermore, the large language model can be invoked again to generate response information corresponding to the user's current input information based on the historical information of human-computer interaction, the user's current input information, and the plugin execution results. Compared to existing classification and slotting models, this application uses a large language model to fully understand the user's real needs, generate executable instructions that match the user's needs, call third-party plugin resources, and generate response information based on plugin execution results, human-computer interaction history information, and current input information. Compared to the response information generated by existing technologies, the response information is more adapted to the user's dialogue needs. Furthermore, by using a large language model to generate executable instructions that call plugin resources, there is no need to pre-customize business call code to adapt to different user needs, saving code development costs and reducing workload. Attached Figure Description
[0048] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:
[0049] Figure 1 This application provides a schematic flowchart of a human-computer interaction method.
[0050] Figure 2 This is a schematic diagram of a human-computer interaction device structure provided in an embodiment of this application;
[0051] Figure 3 A schematic diagram of a human-computer interaction system structure is provided in an embodiment of this application;
[0052] Figure 4 This is a schematic diagram of the structure of a human-computer interaction device provided in an embodiment of this application. Detailed Implementation
[0053] Before introducing the proposed solution, let's first explain the English terms used in this document:
[0054] Prompt: Instructions. When engaging in dialogue with AI (such as large language models), you need to send instructions to the AI. These instructions can be a text description, such as "Please recommend a popular song for me" when you talk to the AI, or they can be parameter descriptions in a certain format, such as describing the relevant drawing parameters to ask the AI to draw a picture in a certain format.
[0055] Large language model (LLM) refers to a generative deep neural network model based on the Transformer structure. It has powerful semantic understanding and multi-turn dialogue capabilities. Common large language models include GT4 and other large language models developed by companies.
[0056] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0057] This application provides a human-computer interaction solution that can be applied to various forms of human-computer interaction scenarios, such as: task-based dialogue (dialogue conducted in a specific context, aimed at completing a specific task or achieving a common goal), knowledge-based question and answer, and casual conversation.
[0058] The proposed solution can be implemented based on a terminal with data processing capabilities, such as a mobile phone, computer, learning machine, or intelligent robot.
[0059] Next, combined Figure 1 The human-computer interaction method of this application may include the following steps:
[0060] Step S100: Obtain historical information on human-computer interaction and the user's input information in this round.
[0061] Specifically, to enhance the semantic understanding and multi-turn dialogue capabilities of large language models, in human-computer dialogue scenarios, in addition to acquiring the user's current input information, it is also possible to further acquire historical human-computer interaction information. This historical information refers to the input and output information from multiple rounds of human-computer interaction prior to the user's current input, and these historical interaction rounds can be all or some of the rounds.
[0062] This application can manage the input and output information of the interactive system when each round of interaction is completed. For example, the system's input and output can be managed by pushing them onto a history stack. Thus, when a new round of interaction is initiated, the historical information of human-computer interaction can be obtained based on the history stack and used as part of the input of the large language model.
[0063] Referring to Table 1 below, which uses a weather query as an example, the results of the historical data stack are illustrated:
[0064] Table 1
[0065]
[0066]
[0067] Furthermore, users can use various input methods, such as voice and text, for each round of input. When a user uses voice input, this application can recognize the input voice and obtain the recognized text as the user's input information for that round. When a user uses text input, the user's input text can be directly used as the input information for that round.
[0068] Step S110: Invoke the preset large language model, and generate executable instructions that match the user's interaction needs in this round based on the human-computer interaction history information and the current input information.
[0069] The executable instructions include calls to plugin resources that are required to respond to the user's interaction requests in this round.
[0070] This application allows for the pre-configuration of plugin resources, which can include various types of plugins, such as information source plugins, knowledge base plugins, personalized content recommendation plugins, and toolset plugins.
[0071] Understandably, in the process of human-computer interaction, to achieve machine responses, it is necessary to understand the interaction needs corresponding to the user's input information in each round. For example, in the process of knowledge-based question-and-answer interaction, it is necessary to understand the user's intention in asking questions in each round; in the process of casual conversation interaction, it is necessary to determine the interaction needs corresponding to the user's input information in each round. For example, if a user says "I'm so bored lately" in a certain round of conversation, the machine can analyze the user's input information to determine that the interaction need for this round is "recommend a movie", and so on.
[0072] After determining the interaction requirements, this step leverages the code generation capabilities of a large language model. Based on historical human-computer interaction information and the user's current input, it generates executable instructions that match the user's interaction requirements for this round. These executable instructions can be in code form, and the language is not limited; examples include Python code or executable code in other languages. These executable instructions, matching the user's interaction requirements, are used to retrieve resources that match those requirements after execution. Taking the determined interaction requirement "recommend movies" as an example, an executable instruction can be generated to call a "personalized content recommendation plugin." After executing this instruction, the "personalized content recommendation plugin" is invoked, and the content resources obtained after the plugin's execution are retrieved.
[0073] Step S120: Parse and execute the executable instructions to call the plugin according to the instructions of the executable instructions and obtain the execution result of the plugin.
[0074] Specifically, after the large language model generates executable instructions, these instructions can be parsed and executed to call the plugin according to the instructions and obtain the plugin's execution result.
[0075] Optionally, this application can pre-configure a plugin parser for parsing and executing executable instructions. In this step, the pre-configured plugin parser can be invoked to parse and execute the executable instructions, thereby calling plugins according to the instructions and obtaining the plugin execution results. The plugin parser can also standardize the execution results of each plugin into a uniform format for subsequent processing.
[0076] Step S130: Invoke the large language model and generate response information corresponding to the user's current input information based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin.
[0077] It should be noted that the execution result of the plugin obtained after executing the executable command in the previous step is data in a fixed format. In order to better respond to the user, this step can call the large language model again to generate response information corresponding to the user's current input information based on the human-computer interaction history information, the user's current input information, and the plugin's execution result. By making full use of the multi-turn dialogue capability of the large language model, more human-like and more accurate response information can be generated.
[0078] This application's embodiments utilize a large language model and pre-configured plugin resources to implement the human-computer interaction process. It fully leverages the large language model's powerful semantic understanding and multi-turn dialogue capabilities, as well as the internal and external resources provided by the plugins, such as information sources, knowledge bases, content recommendation resources, and toolsets. Specifically, after obtaining the human-computer interaction history information and the user's current input information, the large language model is invoked. Based on this history and current input information, an executable instruction matching the user's interaction needs for that round is generated. This executable instruction includes calls to the plugin resources relied upon to respond to the user's interaction needs. Thus, this application can automatically generate executable instructions matching the user's interaction needs for that round by utilizing the large language model's semantic understanding and code generation capabilities. Based on this, the generated executable instructions can be parsed and executed to invoke plugins according to their instructions, obtaining the plugin's execution result. Then, the large language model can be invoked again to generate response information corresponding to the user's current input information based on the human-computer interaction history information, the user's current input information, and the plugin's execution result. Compared to existing classification and slotting models, this application uses a large language model to fully understand the user's real needs, generate executable instructions that match the user's needs, call third-party plugin resources, and generate response information based on plugin execution results, human-computer interaction history information, and current input information. Compared to the response information generated by existing technologies, the response information is more adapted to the user's dialogue needs. Furthermore, by using a large language model to generate executable instructions that call plugin resources, there is no need to pre-customize business call code to adapt to different user needs, saving code development costs and reducing workload.
[0079] In some embodiments of this application, the process of calling a preset large language model to generate executable instructions that match the user's interaction needs in the current round based on the human-computer interaction history information and the current round input information in step S110 is described.
[0080] In this embodiment, a first prompt format template can be pre-configured, which includes a history information slot and a current input information slot.
[0081] The first prompt template is used to instruct the large language model to understand the user's interaction needs in the current round by combining the information in the historical information slots and to provide matching executable instructions. For example, the first prompt template could be: "Please understand the user's interaction needs in the current round by combining the human-computer interaction history information and provide executable plugin code. <historical information slots>, <current round input information slots>."
[0082] Furthermore, the human-computer interaction history information can be filled into the history information slot, and the user's current input information can be filled into the current input information slot to obtain the first prompt instruction after editing, which is then input into the large language model to obtain the executable instruction output by the large language model.
[0083] Taking the first three rounds of interaction in the example in Table 1 above as the human-computer interaction history information, and the user's input in the fourth round as the input information for this round, the first prompt after editing could be:
[0084] Please understand the user's interaction needs in this round based on the historical information of human-computer interaction, and provide executable plugin code. Q1: Is Huangshan fun? A1: Huangshan is a famous tourist destination in China…, worth a visit. Q2: What's the weather like there? A2: The weather in Huangshan City today is cloudy…, still in a good mood. Q3: What about tomorrow? A3: The weather forecast for Huangshan City tomorrow is cloudy turning to light rain…, so as to stay comfortable during the trip. Q: What clothes should I wear?
[0085] After inputting the above prompt into the large language model, the executable code output by the large language model (taking Python-like code as an example) is shown below:
[0086]
[0087] As shown above, the executable code output by the large language model includes a call to the third-party weather plugin "xinyuan". By executing this code, the corresponding weather plugin can be called to obtain the clothing index for Huangshan tomorrow.
[0088] By parsing and executing the executable code generated above, the execution result of the plugin can be obtained: "Tomorrow in Huangshan City will be cloudy, with temperatures ranging from 16 to 27 degrees Celsius. The daytime temperature will be moderate, but the mornings and evenings will be cool. A portable jacket that is easy to put on and take off is very practical."
[0089] Further optionally, in order to enable the large language model to generate executable instructions that match the user's current interaction needs more accurately, the large language model can be pre-trained in this embodiment. During the training phase, human-computer interaction history information in various domain scenarios, user input information in the current round, and manually annotated executable instructions that match the user's current interaction needs can be used as training data, thereby improving the code generation capability of the large language model in various domain scenarios.
[0090] In some embodiments of this application, the process of calling the large language model and generating response information corresponding to the user's current input information based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin is described.
[0091] In this embodiment, a second prompt format template can be pre-configured, which includes a history information slot, a current input information slot, and a plugin information slot.
[0092] The second prompt template is used to instruct the large language model to respond to the user's input in the current round, combining information from the historical information slots and the query information slots. For example, the second prompt template could be: "Please respond to the user's input in this round, combining historical human-computer interaction information and plugin execution results. <Historical Information Slot>, <Current Round Input Information Slot>, <Plugin Information Slot>."
[0093] Furthermore, the human-computer interaction history information can be filled into the history information slot, the user's current input information can be filled into the current input information slot, and the execution result of the plugin can be filled into the query information slot to obtain the edited second prompt instruction, which is then input into the large language model to obtain the response information output by the large language model.
[0094] Let's take the previous example as an illustration:
[0095] The second prompt command after editing can be:
[0096] Please respond to the user's input in this round, based on the human-computer interaction history and plugin execution results. Q1: Is Huangshan fun? A1: Huangshan is a famous tourist destination in China…, worth a visit. Q2: What's the weather like there? A2: The weather in Huangshan City today is cloudy…, still in a good mood. Q3: What about tomorrow? A3: The weather forecast for Huangshan City tomorrow is cloudy turning to light rain…, so you can stay comfortable during your trip. Q: What clothes should I wear? Plugin execution result: Tomorrow in Huangshan City will be cloudy, 16-27°C. Daytime temperatures are moderate, but it will be cool in the mornings and evenings. A portable jacket that is easy to put on and take off is very practical.
[0097] After inputting the above prompt into the large language model, the response information output by the large language model is as follows:
[0098] The weather in Huangshan tomorrow will be cloudy, with temperatures ranging from 16℃ to 27℃, which is on the warmer side. We recommend wearing summer clothing such as T-shirts and short-sleeved shirts. There is a significant temperature difference between day and night, so please adjust your clothing accordingly to stay comfortable.
[0099] Further optionally, in order to enable the large language model to generate response information more accurately and human-like, the large language model can be pre-trained in this embodiment. During the training phase, the training data can be the human-computer interaction history information in various domain scenarios, the user's current input information, the plugin execution results, and the manually compiled response information that matches the user's current input information, thereby improving the large language model's ability to generate human-computer dialogue in various domain scenarios.
[0100] In some embodiments of this application, a dialogue control management process can be further added to the above-described human-computer interaction scheme. Considering step S120, where the executable instructions are parsed and executed to obtain the plugin's execution result, the obtained plugin execution result may fall into the following categories:
[0101] 1. The plugin returned execution result data that failed (including but not limited to failure to obtain the plugin execution result, missing execution result data, errors, and other abnormal data);
[0102] 2. The execution result data returned by the plugin is not suitable for system output (including but not limited to: the execution result data contains prohibited words or other security issues);
[0103] 3. The execution result data returned by the plugin is incomplete and needs to be supplemented and merged.
[0104] 4. If the execution result data returned by the plugin does not fall under any of the above 1-3 situations, then the returned execution result data is considered successful.
[0105] In this embodiment, different processing methods can be adopted to control and manage the dialogue process based on different plugin execution results. For example, when the plugin execution result is the fourth case mentioned above, the next task can be determined as the response generation task; when the plugin execution result is the first case mentioned above, the next task can be determined as the retry task; when the plugin execution result is the second case mentioned above, the next task can be determined as the fallback task; and when the plugin execution result is the third case mentioned above, the next task can be determined as the planning of a new task.
[0106] Therefore, the first step is to determine which of the above-mentioned situations the plugin's execution result is currently in, or in other words, to plan the next task based on the current plugin execution result. In this embodiment, the planning of the next task can be achieved based on a large language model.
[0107] Specifically, the following processing steps may be included before the aforementioned step S130:
[0108] The large language model is invoked, and the next task is planned based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin.
[0109] If the next task is to generate a response, then step S130 is executed, which involves calling the large language model to generate a response corresponding to the user's current input based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin.
[0110] If the next task is a retry task, then return to re-execute step S110, call the preset large language model, generate executable instructions that match the user's interaction needs in this round based on the human-computer interaction history information and the current round input information, regenerate the executable instructions, and parse and execute the regenerated executable instructions to obtain a new plugin execution result.
[0111] If the next task is a fallback task, it means that the execution result of the plugin is unusable. Therefore, the execution result of the plugin can be set to none, and step S130 can be executed, which calls the large language model to generate the response information corresponding to the user's current input information based on the human-computer interaction history information, the user's current input information and the execution result of the plugin.
[0112] If the next task is to plan a new task, it means that the plugin's execution result is incomplete. Therefore, step S110 can be executed again to generate a new executable instruction, and the new executable instruction can be parsed and executed to obtain a new plugin execution result. The new plugin execution result is then merged with the obtained plugin execution result, and then step S130 is executed.
[0113] The process of calling the large language model described above, based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin, to plan the next task may specifically include:
[0114] Obtain a pre-configured third prompt format template, which includes a history information slot, a current input information slot, and a plugin information slot.
[0115] The third prompt template is used to instruct the large language model to plan the next task by combining information from the history information slot, the current input information slot, and the plugin information slot. For example, the third prompt template could be: "Please plan the next task by combining the human-computer interaction history information, the original user input information, and the plugin execution results. <History Information Slot>, <Current Input Information Slot>, <Plugin Information Slot>."
[0116] Furthermore, the human-computer interaction history information can be filled into the history information slot, the user's current input information can be filled into the current input information slot, and the execution result of the plugin can be filled into the plugin information slot to obtain the third prompt instruction after editing, which is then input into the large language model to obtain the next task output by the large language model.
[0117] Let's take the previous example as an illustration:
[0118] The edited third prompt command can be:
[0119] Please plan the next task based on the human-computer interaction history, the original user input, and the plugin execution results. Q1: Is Huangshan fun? A1: Huangshan is a famous tourist destination in China…, worth a visit. Q2: What's the weather like there? A2: The weather in Huangshan City today is cloudy…, still in a good mood. Q3: What about tomorrow? A3: The weather forecast for Huangshan City tomorrow is cloudy turning to light rain…, so as to stay comfortable during the trip. Q: What clothes should I wear? Plugin execution results: Huangshan City will be cloudy the day after tomorrow, 16-27°C. The daytime temperature is moderate, but it's cool in the morning and evening. A portable jacket that's easy to put on and take off is very practical.
[0120] After inputting the above prompt into the large language model, the response information output by the large language model is as follows: Response generation task. Based on this, we can proceed to step S130.
[0121] The human-computer interaction device provided in the embodiments of this application is described below. The human-computer interaction device described below can be referred to in correspondence with the human-computer interaction method described above.
[0122] See Figure 2 , Figure 2 This is a schematic diagram of the structure of a human-computer interaction device disclosed in an embodiment of this application.
[0123] like Figure 2 As shown, the device may include:
[0124] Information acquisition unit 11 is used to acquire human-computer interaction history information and the user's current input information;
[0125] The first model invocation unit 12 is used to invoke a preset large language model, and generate an executable instruction that matches the user's interaction needs in the current round based on the human-computer interaction history information and the current round input information; the executable instruction includes the invocation of the plug-in resources on which the user's interaction needs in the current round depend;
[0126] The instruction parsing and processing unit 13 is used to parse and execute the executable instruction to call the plugin according to the instruction and obtain the execution result of the plugin;
[0127] The second model invocation unit 14 is used to invoke the large language model to generate response information corresponding to the user's current input information based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin.
[0128] Optionally, the process by which the first model invocation unit invokes a preset large language model to generate executable instructions matching the user's interaction needs in the current round, based on the human-computer interaction history information and the current input information, may include:
[0129] Obtain a pre-configured first prompt format template, which includes a history information slot and a current input information slot. The first prompt format template is used to instruct the large language model to combine the information in the history information slot to understand the user's interaction needs in the current round and provide matching executable instructions.
[0130] The human-computer interaction history information is filled into the history information slot, and the user's current input information is filled into the current input information slot to obtain the first prompt instruction after editing;
[0131] The edited first prompt instruction is input into the large language model to obtain the executable instruction output by the large language model.
[0132] Optionally, the process by which the instruction parsing and processing unit parses and executes the executable instructions to call the plugin according to the instructions of the executable instructions and obtain the execution result of the plugin may include:
[0133] The pre-configured plugin parser is invoked to parse and execute the executable instructions, so as to invoke the plugin according to the instructions of the executable instructions and obtain the execution result of the plugin.
[0134] Optionally, the process by which the second model invocation unit calls the large language model to generate response information corresponding to the user's current input information based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin may include:
[0135] Obtain a pre-configured second prompt format template, which includes a history information slot, a current input information slot, and a plugin information slot. The second prompt format template is used to instruct the large language model to combine the information in the history information slot and the query information slot to respond to the user's input information in the current round.
[0136] The human-computer interaction history information is filled into the history information slot, the user's current input information is filled into the current input information slot, and the execution result of the plugin is filled into the query information slot to obtain the second prompt instruction after editing;
[0137] The edited second prompt instruction is input into the large language model to obtain the response information output by the large language model.
[0138] Optionally, the apparatus of this application may further include:
[0139] The third model invocation unit is used to invoke the large language model before the second model invocation unit is executed, and to plan the next task based on the human-computer interaction history information, the user's current input information and the execution result of the plugin. If the next task is a response generation task, then the steps of the second model invocation unit are executed.
[0140] Optionally, if the next task obtained by the third model calling unit is a retry task, then return to the step of executing the first model calling unit;
[0141] If the next task obtained by the third model calling unit is a fallback task, then the execution result of the plugin is set to empty, and the steps of the second model calling unit are executed.
[0142] Optionally, the process by which the third model invocation unit invokes the large language model and plans the next task based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin may include:
[0143] Obtain a pre-configured third prompt format template, which includes a history information slot, a current input information slot, and a plugin information slot. The third prompt format template is used to instruct the large language model to combine the information in the history information slot, the current input information slot, and the plugin information slot to plan the next task.
[0144] The human-computer interaction history information is filled into the history information slot, the user's current input information is filled into the current input information slot, and the execution result of the plugin is filled into the plugin information slot to obtain the third prompt instruction after editing;
[0145] Input the edited third prompt command into the large language model to obtain the next task output by the large language model.
[0146] In some embodiments of this application, a human-computer interaction system is further provided, referring to... Figure 3 As shown, a human-computer interaction system may include:
[0147] The system includes a scheduling control module 100, a large language model module 110, a plug-in parser module 120, and a plug-in resource module 130.
[0148] The scheduling control module 100 is used to acquire human-computer interaction history information and the user's current input information, call the large language model module 110, and generate an executable instruction matching the user's interaction needs in the current round based on the human-computer interaction history information and the current input information. The executable instruction includes the invocation of plugin resources on which the response to the user's interaction needs in the current round depends. The module then calls the plugin parser module 120 to parse and execute the executable instruction, so as to invoke the plugin provided by the plugin resource module 130 according to the instructions of the executable instruction, and obtain the execution result of the plugin. Finally, the module calls the large language model module 110 to generate and output the response information corresponding to the user's current input information based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin.
[0149] Further optional, such as Figure 3 As shown, the human-computer interaction system may also include:
[0150] The history management module 140 is used to manage the historical inputs and outputs of the human-computer interaction system and store them in the history stack. The scheduling control module 100 can then obtain historical human-computer interaction information through the history management module 140.
[0151] The detailed and extended functions of the aforementioned scheduling control module 100 can be found in the human-computer interaction method described above.
[0152] The human-computer interaction device provided in this application embodiment can be applied to human-computer interaction devices, such as mobile phones, computers, learning machines, and intelligent robots. Optionally, Figure 4 The hardware structure block diagram of the human-computer interaction device is shown, with reference to Figure 4 The hardware structure of a human-computer interaction device may include: at least one processor 1, at least one communication interface 2, at least one memory 3, and at least one communication bus 4;
[0153] In this embodiment of the application, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4;
[0154] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.
[0155] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;
[0156] The memory stores a program, which the processor can call. The program is used for:
[0157] Acquire historical information about human-computer interaction and the user's current input information;
[0158] The system invokes a pre-built large language model and generates executable instructions that match the user's interaction needs in the current round, based on the human-computer interaction history information and the current input information. The executable instructions include calls to the plugin resources that are relied upon to respond to the user's interaction needs in the current round.
[0159] The executable instructions are parsed and executed to invoke the plugin according to the instructions, and the execution result of the plugin is obtained.
[0160] The large language model is invoked, and based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin, a response information corresponding to the user's current input information is generated.
[0161] Optionally, the refined and extended functions of the program can be found in the description above.
[0162] This application embodiment also provides a storage medium that can store a program suitable for execution by a processor, the program being used for:
[0163] Acquire historical information about human-computer interaction and the user's current input information;
[0164] The system invokes a pre-built large language model and generates executable instructions that match the user's interaction needs in the current round, based on the human-computer interaction history information and the current input information. The executable instructions include calls to the plugin resources that are relied upon to respond to the user's interaction needs in the current round.
[0165] The executable instructions are parsed and executed to invoke the plugin according to the instructions, and the execution result of the plugin is obtained.
[0166] The large language model is invoked, and based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin, a response information corresponding to the user's current input information is generated.
[0167] Optionally, the refined and extended functions of the program can be found in the description above.
[0168] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0169] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.
[0170] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A human-computer interaction method, characterized in that, include: Acquire historical information about human-computer interaction and the user's current input information; Obtain a pre-configured first prompt format template, which includes a history information slot and a current input information slot. The first prompt format template is used to instruct the large language model to understand the user's interaction needs in the current round by combining the information in the history information slot and to provide a matching executable instruction. Fill the history information slot with the human-computer interaction history information and fill the current input information slot with the user's current input information to obtain the edited first prompt instruction. The edited first prompt instruction is input into the large language model to obtain the executable instruction output by the large language model; the executable instruction includes the call to the plugin resources on which the user's interaction needs in this round depend; The executable instructions are parsed and executed to invoke the plugin according to the instructions, and the execution result of the plugin is obtained. Obtain a pre-configured second prompt format template, which includes a history information slot, a current input information slot, and a plugin information slot. The second prompt format template is used to instruct the large language model to respond to the user's input information in the current round by combining the information in the history information slot and the query information slot. Fill the history information of the human-computer interaction into the history information slot, fill the current input information of the user into the current input information slot, and fill the query information slot with the execution result of the plugin to obtain the edited second prompt instruction. The second prompt instruction after editing includes: responding to the user's input in this round by combining human-computer interaction history information and plugin execution results; The edited second prompt instruction is input into the large language model to obtain the response information output by the large language model.
2. The method according to claim 1, characterized in that, The executable instructions are executable Python code.
3. The method according to claim 1, characterized in that, The process of parsing and executing the executable instructions to invoke the plugin according to the instructions and obtain the execution result of the plugin includes: The pre-configured plugin parser is invoked to parse and execute the executable instructions, so as to invoke the plugin according to the instructions of the executable instructions and obtain the execution result of the plugin.
4. The method according to claim 1, characterized in that, The process involves obtaining a pre-configured second prompt format template; filling the history information of human-computer interaction into the history information slot; filling the current input information of the user into the current input information slot; and filling the execution result of the plugin into the query information slot to obtain the edited second prompt instruction. Before inputting the edited second prompt instruction into the large language model and obtaining the response information output by the large language model, the process also includes: The large language model is invoked, and the next task is planned based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin. If the next task is to generate a response, then the step of calling the large language model and generating a response corresponding to the user's current input information based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin is executed.
5. The method according to claim 4, characterized in that, Also includes: If the next task is a retry task, then return to the step of calling the preset large language model, and generating executable instructions that match the user's interaction needs in this round based on the human-computer interaction history information and the current input information; And / or, If the next task is a fallback task, then the execution result of the plugin is set to empty, and the large language model is invoked to generate the response information corresponding to the user's current input information based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin.
6. The method according to claim 4, characterized in that, The large language model is invoked, and based on the human-computer interaction history information, the user's current input information, and the execution result of the plugin, the next task is planned, including: Obtain a pre-configured third prompt format template, which includes a history information slot, a current input information slot, and a plugin information slot. The third prompt format template is used to instruct the large language model to combine the information in the history information slot, the current input information slot, and the plugin information slot to plan the next task. The human-computer interaction history information is filled into the history information slot, the user's current input information is filled into the current input information slot, and the execution result of the plugin is filled into the plugin information slot to obtain the third prompt instruction after editing; Input the edited third prompt command into the large language model to obtain the next task output by the large language model.
7. The method according to any one of claims 1-6, characterized in that, The plugin resources include any one or more of the following: Information source plugin, knowledge base plugin, personalized content recommendation plugin, toolset plugin.
8. The method according to any one of claims 1-6, characterized in that, The human-computer interaction method can be applied to any one of the following scenarios: task-based dialogue, knowledge-based question-and-answer, or chat dialogue.
9. A human-computer interaction device, characterized in that, include: The information acquisition unit is used to acquire historical information of human-computer interaction and the user's current input information. The first model invocation unit is used to obtain a pre-configured first prompt format template, which includes a history information slot and a current input information slot. The first prompt format template is used to instruct the large language model to understand the user's interaction needs in the current round by combining the information in the history information slot and to give a matching executable instruction; fill the history information of human-computer interaction into the history information slot, and fill the current input information of the user into the current input information slot to obtain the edited first prompt instruction. The edited first prompt instruction is input into the large language model to obtain the executable instruction output by the large language model; the executable instruction includes the call to the plugin resources on which the user's interaction needs in this round depend; The instruction parsing and processing unit is used to parse and execute the executable instructions to call the plugin according to the instructions of the executable instructions and obtain the execution result of the plugin; The second model invocation unit is used to obtain a pre-configured second prompt format template. The second prompt format template includes a history information slot, a current input information slot, and a plugin information slot. The second prompt format template is used to instruct the large language model to combine the information in the history information slot and the query information slot to respond to the user's input information in the current round; fill the history information of the human-computer interaction into the history information slot, fill the current input information of the user into the current input information slot, and fill the execution result of the plugin into the query information slot to obtain the edited second prompt instruction. The second prompt instruction after editing includes: responding to the user's input in this round by combining human-computer interaction history information and plugin execution results; The edited second prompt instruction is input into the large language model to obtain the response information output by the large language model.
10. A human-computer interaction system, characterized in that, include: The module includes a central control module, a large language model module, a plugin parser module, and a plugin resource module. The scheduling control module is used to acquire historical information of human-computer interaction and the user's current input information; Obtain a pre-configured first prompt format template, which includes a history information slot and a current input information slot. The first prompt format template is used to instruct the large language model to understand the user's interaction needs in the current round by combining the information in the history information slot and to provide a matching executable instruction. Fill the history information slot with the human-computer interaction history information and fill the current input information slot with the user's current input information to obtain the edited first prompt instruction. The edited first prompt instruction is input into the large language model module to obtain the executable instruction output by the large language model module; the executable instruction includes the call to the plugin resources on which the user's interaction needs in this round depend; The plugin parser module is invoked to parse and execute the executable instructions, so as to invoke the plugins provided by the plugin resource module according to the instructions of the executable instructions, and obtain the execution results of the plugins; Obtain a pre-configured second prompt format template, which includes a history information slot, a current input information slot, and a plugin information slot. The second prompt format template is used to instruct the large language model to respond to the user's input information in the current round by combining the information in the history information slot and the query information slot. Fill the history information of the human-computer interaction into the history information slot, fill the current input information of the user into the current input information slot, and fill the query information slot with the execution result of the plugin to obtain the edited second prompt instruction. The second prompt instruction after editing includes: responding to the user's input in this round by combining human-computer interaction history information and plugin execution results; The edited second prompt instruction is input into the large language model module to obtain the response information output by the large language model module.