Man-machine interaction method, device and equipment based on large language model and storage medium

Abstract

The invention discloses a man-machine interaction method and device based on a large language model, electronic equipment and a storage medium, and the method comprises the steps: obtaining an input user instruction, and carrying out the intention recognition and task splitting of the user instruction through a large language model, and obtaining a user intention and a subtask corresponding to the user instruction; determining an execution mode for executing the user instruction according to the user intention and the subtask; tool information corresponding to the subtasks is determined, and the tool information comprises tool types and tool parameters; and processing the subtask by using a tool corresponding to the tool information according to the execution mode, and generating an interaction result corresponding to the user instruction according to a result obtained by processing. And the convenience and the accuracy corresponding to the complex interaction instruction are improved.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction technology, and in particular to a human-computer interaction method, device, electronic device and storage medium based on a large language model. Background Art

[0002] With the emergence of Large Language Model (LLM) technology, such as ChatGPT, artificial intelligence technology has taken a big step towards artificial general intelligence (AGI), leading a new direction for scientific and technological innovation.

[0003] LLM contains more than 100 billion parameters. After being trained with a large amount of data of all kinds, it has the "common sense" of human society, can well understand and summarize human natural language, and then generate high-quality output in combination with different needs. A large number of studies and practical applications have shown that after inputting detailed instructions and examples, the large language model can give answers comparable to or even better than humans. When LLM technology is combined with user instructions, it can better and more comprehensively understand users, and assist users to better complete instructions or achieve their expected goals.

[0004] However, in actual applications, complex user instructions cannot be accurately and quickly processed, which reduces the user's interactive experience. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a human-computer interaction method, device, electronic device and storage medium based on a large language model to solve the technical problem in the related art that complex user needs cannot be quickly and accurately identified and processed.

[0006] In a first aspect, an embodiment of the present application provides a human-computer interaction method based on a large language model, comprising:

[0007] Obtain input user instructions, and use a large language model to perform intent recognition and task splitting on the user instructions to obtain user intent and subtasks corresponding to the user instructions;

[0008] Determining, according to the user intention and the subtask, a method for executing the user instruction;

[0009] Determine tool information corresponding to the subtask, wherein the tool information includes tool category and tool parameters;

[0010] The subtask is processed using the tool corresponding to the tool information according to the execution mode, and an interaction result corresponding to the user instruction is generated according to the result obtained by the processing.

[0011] In a second aspect, an embodiment of the present application provides a human-computer interaction device based on a large language model, comprising:

[0012] An instruction receiving module is used to obtain an input user instruction, and use a large language model to perform intent recognition and task splitting on the user instruction to obtain the user intent and subtasks corresponding to the user instruction;

[0013] A method determination module, used to determine the execution method of executing the user instruction according to the user intention and the subtask;

[0014] A tool determination module, used to determine tool information corresponding to the subtask, wherein the tool information includes tool category and tool parameters;

[0015] The instruction execution module is used to process the subtask according to the execution mode using the tool corresponding to the tool information, and generate an interaction result corresponding to the user instruction according to the result obtained by the processing.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the steps in the human-computer interaction method based on a large language model when executing the computer program.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in any one of the above-mentioned human-computer interaction methods based on a large language model are implemented.

[0018] The embodiment of the present application provides a human-computer interaction method, device, electronic device and storage medium based on a large language model. When performing human-computer interaction, the device / equipment / system for human-computer interaction receives the user instruction issued by the user, and uses the large language model to perform intent recognition and task splitting on the user instruction, so as to split the user instruction input by the user into subtasks, and then determines the execution mode corresponding to the execution of each subtask according to the user intention recognized by the intention and the split subtasks, and at the same time determines the tool information corresponding to each subtask, and the tool information is used to realize the processing and execution of the subtask, and then performs task processing on the split subtask according to the determined execution mode and tool information, and then generates an interaction result corresponding to the user instruction according to the obtained processing result when the processing is completed, so as to realize human-computer interaction. When performing human-computer interaction, the user instruction input by the user is used to perform intent recognition and task splitting on the user instruction using the large language model, obtain the user intention and subtasks, simplify the complex tasks, and at the same time determine the execution mode of executing the subtasks, so as to execute the subtasks in combination with the determined execution mode, so that the task execution is faster and more accurate, and the convenience and accuracy of task execution are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of a human-computer interaction method based on a large language model provided in an embodiment of the present application;

[0020] Figure 2 This is a flowchart of the steps of determining the execution method provided in the embodiment of the present application;

[0021] Figure 3 This is a schematic diagram of a scene block diagram for human-computer interaction provided in an embodiment of the present application;

[0022] Figure 4 This is another schematic diagram of a scene block diagram for human-computer interaction provided in an embodiment of the present application;

[0023] Figure 5 This is a schematic diagram of another scenario block diagram for human-computer interaction provided in an embodiment of the present application;

[0024] Figure 6 It is a block diagram of a process of human-computer interaction based on a large language model provided in an embodiment of the present application;

[0025] Figure 7 It is a structural schematic diagram of a human-computer interaction device based on a large language model provided in an embodiment of the present application;

[0026] Figure 8 is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0027] Fig. 9This is another structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0029] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0030] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0031] In order to solve the technical problems existing in the related art, the present application embodiment provides a human-computer interaction method based on a large language model. Figure 1 , Figure 1 It is a flow chart of a human-computer interaction method based on a large language model provided in an embodiment of the present application, and the method includes steps 101 to 104.

[0032] Step 101, obtain input user instructions, and use a large language model to perform intent recognition and task splitting on the user instructions to obtain user intent and subtasks corresponding to the user instructions.

[0033] In one embodiment, during the human-computer interaction process, in order to achieve interaction with relevant equipment / devices / systems, the user will input language, text or video according to his or her own needs, so that the equipment / device / system performing the human-computer interaction completes the interaction with the user according to the received relevant input information. Specifically, during the interaction process, the input user instructions are received, and then the user instructions are recognized and task-splitting is performed using a large language model to obtain the user intention and subtasks corresponding to the user instructions.

[0034] Exemplarily, when processing user instructions, a pre-trained large language model is used to perform relevant processing on the user instructions, wherein the user instructions may be in the form of voice, text, pictures, audio and video, etc. When the large language model receives the user instructions, it performs corresponding processing according to the form of the user instructions to complete the acquisition of the information contained in the user instructions. For example, user instructions in any form can be converted into corresponding texts, and then the text is subjected to intent recognition and task splitting to obtain the user intent and subtasks corresponding to the user instructions. Of course, when completing the task splitting, the number of subtasks obtained can be one or more. That is, for simple user instructions, human-computer interaction can be achieved based on the method described in this application. For complex user instructions, human-computer interaction can also be achieved after being processed based on the method described in this application.

[0035] Among them, when using a large language model to process user instructions to obtain user intentions and subtasks, it includes: using a large language model to perform intent recognition on user instructions to obtain user intentions corresponding to the user instructions; performing task recognition on user instructions to determine the task domain corresponding to the user instructions; and performing task splitting on user instructions according to user intentions and task domains to obtain subtasks corresponding to the user instructions.

[0036] Specifically, for different text descriptions, when performing intent recognition, errors may occur in task splitting due to the different fields of the content described in the text. For example, "apple" can belong to the fruit field or the electronic product field. Therefore, in order to improve the accuracy of task splitting, when identifying and splitting tasks, it is necessary to refer to the current corresponding field information, and then complete the splitting of tasks contained in user instructions based on the field information.

[0037] Exemplarily, when processing user instructions, a large language model is used to perform intent recognition on the instructions to obtain the user intent corresponding to the user instructions. At the same time, task recognition is performed on the user instructions to determine the task domain corresponding to the user instructions. The user instructions are then split into tasks based on the obtained user intent and task domain, and the subtasks contained in the user instructions are determined, so that more accurate processing results can be obtained when the subtasks are subsequently processed.

[0038] Step 102: Determine the execution method of executing the user instruction according to the user intention and the subtask.

[0039] In one embodiment, after completing the task splitting of the user instruction and obtaining the subtasks, before processing the subtasks, it is necessary to determine the method for processing the subtasks. In fact, based on the different processing methods of the subtasks, the methods for executing the subtasks can be divided into three methods, including the first method, the second method and the third method.

[0040] For example, the first method is to execute non-instantaneously according to the timeline, the second method is to execute instantly based on interaction, and the third method is to execute instantly. For example, through analysis and processing, the execution method corresponding to "I have an important temporary meeting with Mr. Zhang on Friday afternoon" is the first method, the execution method corresponding to "I have three symbols on my desk, how do I get them out if they disappear" is the second method, and the execution method corresponding to "I will go to withdraw some money immediately, I have ICBC and ABC cards, which one is suitable" is the third method. Of course, when determining the execution method corresponding to the user instruction, it is necessary to process and determine it according to the actual user instruction.

[0041] Specifically, when determining the execution mode corresponding to the user instruction, it is determined by analyzing and processing the user intention and the subtask. Figure 2 , Figure 2 It is a flowchart of the steps for determining the execution method provided in an embodiment of the present application, wherein the steps include steps 201 to 203.

[0042] Step 201, determining whether there is task continuity between subtasks according to user intention;

[0043] Step 202, when it is determined that there is task continuity between the subtasks, determining the execution mode to be the first mode;

[0044] Step 203, when it is determined that there is no task continuity between the subtasks, it is determined whether there is a fuzzy task in the subtasks, and the execution mode is determined according to whether there is a fuzzy task.

[0045] As can be seen from the above description, when processing, since there are three execution modes that can be used, when determining which execution mode to use to execute the user instruction or subtask, it is necessary to determine it based on the actual user intention and the relevant characteristics of the subtask. Specifically, when making the determination, it is determined whether there is task continuity between the subtasks based on the user intention, and when it is determined that there is task continuity, the execution mode is determined to be the first mode, and when it is determined that there is no task continuity, further judgment is required.

[0046] It should be noted that task continuity refers to whether there is a hierarchical relationship between tasks, that is, whether two adjacent tasks require the completion of the previous task before the next task can be processed. If so, it means that there is task continuity, otherwise it is determined that there is no task continuity.

[0047] Furthermore, when it is determined that there is no task continuity between subtasks, the current execution method cannot be directly determined. In actual applications, when conducting human-computer interaction, there may be ambiguous and inaccurate information. In this case, the large language model can but cannot completely automatically and accurately process the information. It usually requires certain interactions with the user to complete the information supplement. Therefore, when it is determined that there is no task continuity, it is necessary to determine whether there is an ambiguous task and then choose an appropriate method to handle it.

[0048] Among them, when it is determined that there is no task continuity between subtasks, the process of determining the execution method includes: when it is determined that there is an ambiguous task in the subtask, determining the execution method to be the second method; when it is determined that there is no ambiguous task in the subtask, determining the execution method to be the third method.

[0049] That is, when it is determined that there is an ambiguous task, the second method is used for processing, and when it is determined that there is no ambiguous task, the third method is used for processing.

[0050] For example, for the user instruction described above, "There are three symbols on my desktop, how do I get them out if they disappear?", there is a fuzzy task. Due to the fuzziness of "desktop", it is impossible to accurately determine the relevant parameters for processing during processing. Therefore, it is necessary to interact with the user to determine the accuracy of the "desktop", such as whether the "desktop" is a mobile phone desktop or a specific table desktop.

[0051] In addition, in a special case, when the number of obtained subtasks is one, the third method can be used to process when determining the currently used execution method.

[0052] Step 103: Determine the tool information corresponding to the subtask, wherein the tool information includes tool category and tool parameters.

[0053] In actual applications, when performing interactive processing, you can combine relevant tools for thinking, plan and process complex user intentions, and obtain subtasks. When executing and processing subtasks, select appropriate tools and parameters for processing to obtain the processing results corresponding to each subtask.

[0054] Among them, tools refer to device system capabilities, external applications, integrated services, various network API interfaces, etc., and specific classifications and examples can be seen in Table 1. For the sake of simplicity and consistency in description, unless otherwise specified, "tools" are uniformly used in various embodiments to refer to system capabilities / applications / APIs / services, etc. in Table 1.

[0055] Table 1

[0056]

[0057]

[0058] Referring to Table 1 , tools can be divided into different levels. For example, the first level of tools are: system, application, API and service, while the second level of tools are included in the examples in Table 1 , such as: search, calculator, flashlight, etc.

[0059] It should be noted that although the large language model has common sense in human society, it is limited by the distribution and timeliness of training data. The model cannot give a 100% correct answer to a specific event or choose the right tool. In some cases, the model may generate seemingly correct erroneous information (called hallucination in the industry). Therefore, the tool understanding module is used to let the large language model master the existing tools and corresponding parameters.

[0060] In actual application, two different methods can be selected according to the size of the tool set to make the large language model understand the tools. When the tool set is small, all tool names, descriptions, and parameter descriptions can be put into prompts, so that the large language model can understand all available tools in real time when using it. In other words, there is no need to fine-tune the parameters of the large language model at this time. When the number of tools is large, due to the length limit of the large language model input window, all tool information cannot be put in. In this case, the tool use case dataset is constructed and the large language model is fine-tuned to let the model master all available tools. In other words, the large language model needs to be fine-tuned at this time.

[0061] When fine-tuning a large language model, a use case dataset is constructed. Each piece of data contains a user instruction and the corresponding plan and tools. The number of tools can be one or more, as long as it is reasonable to meet the user instructions. The entire dataset needs to involve all tools and ensure that each tool is used a certain number of times.

[0062] When determining the tool information of each subtask in the subtask, it is determined by analyzing and processing the subtasks, including: identifying the tools of each subtask in the subtask, and determining the tool categories corresponding to each subtask in the subtask; determining the task intent of each subtask in the subtask, and determining the tool parameters corresponding to the tool category according to the task intent.

[0063] Specifically, by identifying the tool for each subtask, the tool type corresponding to each subtask is determined, and then the corresponding tool parameters are determined according to the task intent actually corresponding to each subtask. Taking the user instruction "buy a plane ticket to Beijing" as an example, when splitting the task, the subtasks obtained are: the user wants to buy a plane ticket to Beijing, so when determining the tool at this time, the tools required include: positioning, search and air ticket software, and the tool parameter determined at this time is: Beijing, and then when processing, by opening the corresponding air ticket software, turning on the positioning service, searching for air ticket information from the current location to Beijing, and then displaying the obtained air ticket information to the user. In other words, tool parameters are used to ensure the accurate operation of the tool.

[0064] Step 104: Use the tool corresponding to the tool information to process the subtask according to the execution mode, and generate an interactive result corresponding to the user instruction according to the processing result.

[0065] In one embodiment, after determining the execution method used by the current user instruction and the tool information used when executing the subtask, the subtask will be processed based on the execution method and tool information, and then the interaction result corresponding to the user instruction will be obtained according to the result of the processing.

[0066] Exemplarily, when processing according to the execution method, it includes: determining the tool corresponding to the tool information, and using the tool to process each subtask in the subtask according to the execution method to obtain the processing result corresponding to the subtask; integrating information and converting text into the processing result to generate an interactive result corresponding to the user instruction.

[0067] Specifically, when processing, the execution method determines the way to handle all subtasks. When processing one of the subtasks, according to the tool parameters corresponding to the subtask, appropriate tools are used to perform processing according to the corresponding parameters, such as searching, positioning, navigating, etc., to obtain the processing results corresponding to each subtask, such as the search results obtained, the current location information obtained by positioning, and the route obtained by navigation.

[0068] When processing the user instructions as a whole, due to the differences in the execution methods, the human-computer interaction process is different when processing based on different execution methods, specifically including: when the execution method is the first method, the subtask is executed according to the first method, the processing result is displayed, and the first feedback information input based on the processing result is received; when the execution method is the second method, the first subtask is determined among the subtasks, and an information supplement instruction is generated according to the first subtask, and the second feedback information input in response to the information supplement instruction is received; when the execution method is the third method, each subtask in the subtask is executed according to the third method to obtain the processing result.

[0069] Due to the differences among the first method, the second method and the third method, different execution methods have different processes when processing subtasks.

[0070] When the execution mode is the first mode, during the processing, it is necessary to interact with the user to obtain a more accurate interaction result. At this time, during the processing, all subtasks can be planned and displayed, but since some subtasks require further confirmation, real-time interaction with the user is required during the planning process to obtain a more accurate interaction result.

[0071] Take the user instruction "There is an important temporary meeting with Mr. Zhang on Friday afternoon" as an example. At this time, when processing, the processing process and block diagram can be as follows: Figure 3 As shown, Figure 3 is a schematic diagram of a scene block diagram for human-computer interaction provided in an embodiment of the present application. Figure 3 It can be seen that during processing, the model has obtained an output based on the processing of user instructions, but during the output, there is information that requires further confirmation from the user. Therefore, real-time interaction with the user can be carried out at this time, so that the model can perform further planning and processing based on the real-time interaction information, thereby obtaining more accurate interaction results.

[0072] When the execution mode is the second mode, during processing, due to the information ambiguity of the first subtask, it is necessary to interact with the user in real time before processing the subtask to obtain relevant information for supplementing the information of the first subtask, so as to supplement the information of the first subtask in a timely manner, and after completing the information supplement, the task is planned and processed to obtain the interaction result.

[0073] Take the example of a user instruction "Three symbols on my desktop are missing, how do I get them out?". At this time, when processing, the processing process and block diagram can be as follows: Figure 4 As shown, Figure 4It is another schematic diagram of a scene block diagram for human-computer interaction provided in an embodiment of the present application. Figure 4 It can be seen that during processing, since the "desktop" in the user's command is ambiguous information, further confirmation is required. At this time, when confirming, an interactive information will be generated based on the ambiguous information to determine the specific information of the "desktop". The user inputs relevant information to supplement the ambiguous information, and then all subtasks are processed and planned again after the supplement is completed. Similarly, in the further processing and planning process, it can be determined whether information supplementation is needed based on actual conditions. After confirming that information supplementation is not necessary, the corresponding interactive results can be obtained through the final planning processing.

[0074] When the execution mode is the third mode, immediate execution is performed directly during processing, that is, no further interaction with the user is required to complete and supplement the information. At this time, the subtasks are directly planned and processed to obtain corresponding interaction results.

[0075] For example, if the user instruction is "I need a research report on the adjustment of domestic agricultural policies within one year, about 5,000 words", you can refer to it when processing Figure 5 , Figure 5 This is another scenario block diagram schematic diagram for human-computer interaction provided in an embodiment of the present application. At this time, the subtasks and corresponding tool information are determined by processing using a large language model, and the subtasks are directly planned and processed to output a report file that meets user needs.

[0076] For example, if the user's instruction is "OK, I'll go get some money right away. I have ICBC and ABC cards. I'll walk there. Which one is suitable?" Figure 5 At this time, when processing, the corresponding subtasks and related tool information are also analyzed and processed, and all subtasks are immediately planned and processed to obtain interactive results: OK, based on your current location, XXX business point is recommended for you, and walking navigation is enabled for you, so you can set off at any time.

[0077] As can be seen from the above description, the subject interacting with the user can be a mobile phone, tablet, computer, VR glasses, etc. Through interaction with the device or terminal, the device or terminal can give the user appropriate feedback according to the actual needs of the user, and realize the functions such as meeting reservation, reason search, file search, route planning and navigation described above.

[0078] Furthermore, in the actual processing process, there may be a situation where there is no interaction result or the interaction result is wrong. There are many reasons for the situation where there is no interaction result or the interaction result is wrong. Based on the entire interaction process, there are two reasons for the situation where there is no interaction result feedback or the interaction result is wrong. One is the tool error and the other is the task splitting error. Therefore, in the interaction process, a tool parameter checking function can be set to detect the determined tool information to ensure the accuracy of the tool information. At the same time, an adjustment function can also be set. When it is determined that a subtask has no corresponding processing result and the tool information is accurate, self-reflection is required, that is, the subtask is re-split and processed.

[0079] In one embodiment, based on the processing results and user intentions, the accuracy of the tools of each subtask in the subtask is judged to determine whether there is an erroneous tool; if it is determined that there is an erroneous tool, the step of determining the tool information corresponding to each subtask in the subtask is executed.

[0080] That is, after obtaining the processing results, the accuracy of the tool for the subtask can be judged based on the processing results of each subtask and the user intention of the user instruction to determine the correctness of the tool. If an error is determined, it is necessary to re-determine the tool and re-plan the subtask.

[0081] In addition, when planning and processing subtasks according to the determined execution method, it includes: when executing subtasks according to the execution method, if it is determined that there is a second subtask in the subtask that does not have a corresponding processing result, then execute the step: determine the tool information corresponding to each subtask in the subtask.

[0082] That is, when planning and processing subtasks, if there is a second subtask without a corresponding processing result, it means that there is an error in the splitting of the second subtask. At this time, it is necessary to split the user instructions again and then plan and process the re-split subtasks.

[0083] Further, refer to Figure 6 , Figure 6 It is a block diagram of the process of human-computer interaction based on a large language model provided in an embodiment of the present application.

[0084] Before use, the large language model (referred to as the large model) needs to have the ability to recognize intent and confirm tools, and needs to be configured in advance, such as Figure 6 Tool understanding and prompt management in the process. When the big model has the ability to understand intent, configure the big model with corresponding functions, and then configure the big model in task planning.

[0085] When planning tasks, the large model is first used to process the user instructions (such as voice or text) input by the user, complete intention recognition and task splitting, and determine the tools corresponding to each subtask. At the same time, the tool parameters corresponding to each subtask are determined. After determining the execution method for performing the task, the determined execution method can be used to perform the task using the determined tool.

[0086] At the same time, after obtaining the tool, you can also add a tool parameter checking module to further ensure the execution success rate of the large model planning. In addition, in the process of executing each subtask, check the rationality of the obtained results and whether they can be used by subsequent subtasks. If they cannot be used normally, let the model reflect and re-plan from the current step.

[0087] Among them, when correcting the tool, automatic and passive adjustments can be performed, such as adjustments based on user interactive input, and interactive management based on actual execution results, to achieve correction of the tool. When correction and adjustment are not required, each subtask can be gradually planned and processed according to the determined execution method to obtain the final interactive result for output.

[0088] In summary, the present application discloses a human-computer interaction method based on a large language model. When performing human-computer interaction, the device / equipment / system for human-computer interaction receives the user instruction issued by the user, and uses the large language model to perform intent recognition and task splitting on the user instruction, so as to split the user instruction input by the user into subtasks, and then determine the execution mode corresponding to each subtask according to the user intention recognized by the intention and the split subtasks, and at the same time determine the tool information corresponding to each subtask, and the tool information is used to realize the processing and execution of the subtask, and then perform task processing on the split subtask according to the determined execution mode and tool information, and then generate the interaction result corresponding to the user instruction according to the obtained processing result when the processing is completed, so as to realize human-computer interaction. When performing human-computer interaction, the user instruction input by the user is used to perform intent recognition and task splitting on the user instruction using the large language model, obtain the user intention and subtasks, simplify the complex tasks, and at the same time determine the execution mode of executing the subtasks, so as to execute the subtasks in combination with the determined execution mode, so that the task execution is faster and more accurate, and the convenience and accuracy of task execution are improved.

[0089] According to the method described in the above embodiment, this embodiment will be further described from the perspective of a human-computer interaction device based on a large language model. The human-computer interaction device based on a large language model can be implemented as an independent entity or integrated into an electronic device, such as a terminal. The terminal may include a mobile phone, a tablet computer, etc.

[0090] See also Figure 7 , Figure 7 is a structural diagram of a human-computer interaction device based on a large language model provided in an embodiment of the present application, such as Figure 7 As shown, the human-computer interaction device 700 based on the large language model provided in the embodiment of the present application includes:

[0091] The instruction receiving module 701 is used to obtain the input user instruction, and use the large language model to perform intent recognition and task splitting on the user instruction to obtain the user intent and subtask corresponding to the user instruction;

[0092] A method determination module 702 is used to determine the execution method of executing the user instruction according to the user intention and the subtask;

[0093] A tool determination module 703 is used to determine the tool information corresponding to the subtask, wherein the tool information includes the tool category and tool parameters;

[0094] The instruction execution module 704 is used to process the subtask according to the execution mode using the tool corresponding to the tool information, and generate an interaction result corresponding to the user instruction according to the processing result.

[0095] In one embodiment, the instruction receiving module 701 is further used to:

[0096] Use a large language model to identify the intent of user commands and obtain the user intent corresponding to the user commands;

[0097] Perform task identification on user instructions and determine the task field corresponding to the user instructions;

[0098] According to user intention and task domain, user instructions are split into tasks to obtain subtasks corresponding to user instructions.

[0099] In one embodiment, the mode determination module 702 is further configured to:

[0100] Determine whether there is task continuity between subtasks based on user intent;

[0101] When it is determined that there is task continuity between the subtasks, determining the execution mode to be the first mode;

[0102] When it is determined that there is no task continuity between the subtasks, it is determined whether there is a fuzzy task in the subtasks, and an execution mode is determined according to whether there is the fuzzy task.

[0103] In one embodiment, the mode determination module 702 is further configured to:

[0104] In the case where it is determined that there is an ambiguous task in the subtask, determining the execution mode to be the second mode;

[0105] When it is determined that there is no ambiguous task in the subtasks, the execution mode is determined to be the third mode.

[0106] In one embodiment, the tool determination module 703 is further configured to:

[0107] Perform tool identification on each subtask in the subtask to determine the tool category corresponding to each subtask in the subtask;

[0108] Determine the task intent of each subtask in the subtask, and determine the tool parameters corresponding to the tool category according to the task intent.

[0109] In one embodiment, the instruction execution module 704 is further configured to:

[0110] Determine the tool corresponding to the tool information, and use the tool to process each subtask in the subtask according to the execution mode to obtain the processing result corresponding to the subtask;

[0111] The processing results are integrated and converted into text to generate interactive results corresponding to user instructions.

[0112] In one embodiment, the instruction execution module 704 is further configured to:

[0113] When the execution mode is the first mode, the subtask is executed according to the first mode, a processing result is obtained for display, and first feedback information input based on the processing result is received;

[0114] In the case where the execution mode is the second mode, determining a first subtask from among the plurality of subtasks, generating an information supplement instruction according to the first subtask, and receiving second feedback information input in response to the information supplement instruction;

[0115] When the execution mode is the third mode, each subtask in the subtask is executed according to the third mode to obtain a processing result.

[0116] In one embodiment, the human-computer interaction device based on the large language model further includes a result adjustment module, which is used to:

[0117] When the execution mode is the first mode or the second mode, the processing result is displayed, and adjustment information input based on the processing result is received;

[0118] Performing semantic recognition on the adjustment information to obtain adjustment semantics corresponding to the adjustment information;

[0119] A subtask to be adjusted corresponding to the adjustment semantics is determined in the subtasks, and a processing result corresponding to the subtask to be adjusted by the adjustment semantics is adjusted.

[0120] In one embodiment, the human-computer interaction device based on the large language model further includes a tool correction module, which is used to:

[0121] Based on the processing results and user intentions, the accuracy of the tools in each subtask is judged to determine whether there are wrong tools;

[0122] When it is determined that there is an erroneous tool, the step of determining tool information corresponding to the subtask is performed.

[0123] In one embodiment, the human-computer interaction device based on the large language model further includes a tool correction module, which is used to:

[0124] In the case of executing the subtask according to the execution mode, if it is determined that there is a second subtask in the subtask but there is no corresponding processing result, the step of determining tool information corresponding to the subtask is executed.

[0125] Also, see Figure 8 , Figure 8 is a schematic diagram of a structure of an electronic device provided in an embodiment of the present application, and the electronic device may be a mobile terminal such as a smart phone, a tablet computer, or the like. Figure 8 As shown, the electronic device 800 includes a processor 801 and a memory 802. The processor 801 is electrically connected to the memory 802.

[0126] The processor 801 is the control center of the electronic device 800. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device 800 and processes data by running or loading applications stored in the memory 802 and calling data stored in the memory 802, thereby monitoring the electronic device 800 as a whole.

[0127] In this embodiment, the processor 801 in the electronic device 800 will load the instructions corresponding to the processes of one or more applications into the memory 802 according to the following steps, and the processor 801 will run the application stored in the memory 802, thereby implementing any step in the human-computer interaction method based on the large language model provided in the above embodiment.

[0128] The electronic device 800 can implement the steps in any embodiment of the human-computer interaction method based on a large language model provided in the embodiments of the present application. Therefore, it can achieve the beneficial effects that can be achieved by any human-computer interaction method based on a large language model provided in the embodiments of the present application. For details, please refer to the previous embodiments and will not be repeated here.

[0129] See also Fig. 9 , Fig. 9 is another structural schematic diagram of an electronic device provided in an embodiment of the present application, such as Fig. 9As shown, Fig. 9 The specific structural block diagram of the electronic device provided in the embodiment of the present application is shown, and the electronic device can be used to implement the human-computer interaction method based on the large language model provided in the above embodiment. The electronic device K00 can be a mobile terminal such as a smart phone or a laptop computer.

[0130] The RF circuit 910 is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, and thus communicate with a communication network or other devices. The RF circuit 910 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity module (SIM) cards, memories, etc. The RF circuit 910 can communicate with various networks such as the Internet, corporate intranets, wireless networks, or communicate with other devices through wireless networks. The above-mentioned wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. The above-mentioned wireless networks can use various communication standards, protocols and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as Institute of Electrical and Electronics Engineers standards IEEE 802.11a, IEEE 802.11b, IEEE802.11g and / or IEEE802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging and short messages, and any other suitable communication protocols, even those that have not yet been developed.

[0131] The memory 920 can be used to store software programs and modules, such as the program instructions / modules corresponding to the human-computer interaction method based on the large language model in the above-mentioned embodiment. The processor 980 executes various functional applications and the human-computer interaction method based on the large language model by running the software programs and modules stored in the memory 920.

[0132] The memory 920 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 920 may further include a memory remotely arranged relative to the processor 980, and these remote memories may be connected to the electronic device 900 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0133] The input unit 930 can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control. Specifically, the input unit 930 may include a touch-sensitive surface 931 and other input devices 932. The touch-sensitive surface 931, also known as a touch display screen or a touchpad, can collect user touch operations on or near it (such as operations performed by users using fingers, styluses, or any other suitable objects or accessories on or near the touch-sensitive surface 931), and drive corresponding connection devices according to a pre-set program. Optionally, the touch-sensitive surface 931 may include a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into touch point coordinates, and then sends it to the processor 980, and can receive and execute commands sent by the processor 980. In addition, the touch-sensitive surface 931 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch-sensitive surface 931, the input unit 930 may further include other input devices 932. Specifically, the other input devices 932 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0134] The display unit 940 can be used to display information input by the user or information provided to the user and various graphical user interfaces of the electronic device 900, which can be composed of graphics, text, icons, videos and any combination thereof. The display unit 940 may include a display panel 941, and optionally, the display panel 941 may be configured in the form of LCD (Li6uid Crystal Display), OLED (Organic Light-Emitting Diode), etc. Further, the touch-sensitive surface 931 may cover the display panel 941, and when the touch-sensitive surface 931 detects a touch operation on or near it, it is transmitted to the processor 980 to determine the type of the touch event, and then the processor 980 provides corresponding visual output on the display panel 941 according to the type of the touch event. Although in the figure, the touch-sensitive surface 931 and the display panel 941 are implemented as two independent components to implement input and output functions, in some embodiments, the touch-sensitive surface 931 and the display panel 941 can be integrated to implement input and output functions.

[0135] The electronic device 900 may also include at least one sensor 950, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 941 according to the brightness of the ambient light, and the proximity sensor may generate an interrupt when the flip cover is closed or closed. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that can also be configured in the electronic device 900, they will not be repeated here.

[0136] The audio circuit 960, the speaker 961, and the microphone 962 can provide an audio interface between the user and the electronic device 900. The audio circuit 960 can transmit the electrical signal converted from the received audio data to the speaker 961, which is converted into a sound signal for output; on the other hand, the microphone 962 converts the collected sound signal into an electrical signal, which is received by the audio circuit 960 and converted into audio data, and then the audio data is output to the processor 980 for processing, and then sent to another terminal through the RF circuit 910, or the audio data is output to the memory 920 for further processing. The audio circuit 960 may also include an earplug jack to provide communication between an external headset and the electronic device 900.

[0137] The electronic device 900 can help users receive requests, send information, etc. through the transmission module 970 (such as a Wi-Fi module), which provides users with wireless broadband Internet access. Although the transmission module 970 is shown in the figure, it is understandable that it is not a necessary component of the electronic device 900 and can be omitted as needed without changing the essence of the invention.

[0138] The processor 980 is the control center of the electronic device 900. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 920, and calling data stored in the memory 920, it executes various functions of the electronic device 900 and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 980 may include one or more processing cores; in some embodiments, the processor 980 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communications. It can be understood that the above-mentioned modem processor may not be integrated into the processor 980.

[0139] The electronic device 900 also includes a power supply 990 (such as a battery) for supplying power to various components. In some embodiments, the power supply can be logically connected to the processor 980 through a power management system, so that the power management system can manage charging, discharging, and power consumption management. The power supply 990 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0140] Although not shown, the electronic device 900 also includes a camera (such as a front camera, a rear camera), a Bluetooth module, etc., which will not be described in detail here. Specifically in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory, and one or more programs, wherein one or more programs are stored in the memory, and are configured to be executed by one or more processors to implement any step of the human-computer interaction method based on a large language model provided in the above embodiment.

[0141] In specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same entity or multiple entities. The specific implementation of the above modules can be found in the previous method embodiments, which will not be repeated here.

[0142] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling related hardware through instructions, and the instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, an embodiment of the present application provides a storage medium, which stores multiple instructions, and when the instructions are executed by the processor, any step in the human-computer interaction method based on a large language model provided in the above embodiments can be implemented.

[0143] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0144] Since the instructions stored in the storage medium can execute the steps in any embodiment of the human-computer interaction method based on a large language model provided in the embodiments of the present application, the beneficial effects that can be achieved by any human-computer interaction method based on a large language model provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0145] The above is a detailed introduction to a human-computer interaction method, device, electronic device and storage medium based on a large language model provided by the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and its core idea of ​​the present application; at the same time, for those skilled in the art, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application. Moreover, for those of ordinary skill in the art, improvements and modifications can be made without departing from the principles of the present application, and these improvements and modifications are also regarded as the scope of protection of the present application.

Claims

1. A human-computer interaction method based on a large language model, characterized in that: include: Obtain input user instructions, and use a large language model to perform intent recognition and task splitting on the user instructions to obtain user intent and subtasks corresponding to the user instructions; Determining, according to the user intention and the subtask, a method for executing the user instruction; Determine tool information corresponding to the subtask, wherein the tool information includes tool category and tool parameters; The subtask is processed using the tool corresponding to the tool information according to the execution mode, and an interaction result corresponding to the user instruction is generated according to the result obtained by the processing.

2. The human-computer interaction method based on a large language model as claimed in claim 1, characterized in that: The using of the large language model to perform intent recognition and task splitting on the user instruction to obtain the user intent and subtasks corresponding to the user instruction includes: Using a large language model to perform intent recognition on the user instruction to obtain a user intent corresponding to the user instruction; Performing task identification on the user instruction to determine the task field corresponding to the user instruction; According to the user intention and the task domain, the user instruction is split into tasks to obtain subtasks corresponding to the user instruction.

3. The human-computer interaction method based on a large language model as claimed in claim 1, characterized in that: The determining, according to the user intention and the subtask, an execution mode of executing the user instruction includes: Determining whether there is task continuity between the subtasks according to the user intention; In the case where it is determined that there is task continuity between the subtasks, determining the execution mode to be the first mode; In the case that it is determined that there is no task continuity between the subtasks, it is determined whether there is a fuzzy task in the subtasks, and the execution mode is determined according to whether there is a fuzzy task.

4. The human-computer interaction method based on a large language model as claimed in claim 3, characterized in that: The determining whether there is a fuzzy task in the subtasks, and determining the execution mode according to whether there is a fuzzy task, includes: In the case where it is determined that there is an ambiguous task in the subtask, determining that the execution mode is the second mode; When it is determined that there is no fuzzy task in the subtasks, the execution mode is determined to be the third mode.

5. The human-computer interaction method based on a large language model as claimed in claim 1, characterized in that: The tool information corresponding to the subtask is determined, including: Performing tool identification on each of the subtasks to determine the tool category corresponding to each of the subtasks; The task intention of each subtask in the subtask is determined, and the tool parameters corresponding to the tool category are determined according to the task intention.

6. The human-computer interaction method based on a large language model as claimed in claim 4, characterized in that: The tool corresponding to the tool information processes the subtask according to the execution mode, and generates an interaction result corresponding to the user instruction according to the result obtained by the processing, including: Determine a tool corresponding to the tool information, and use the tool to process each subtask in the subtask according to the execution mode to obtain a processing result corresponding to the subtask; The processing results are subjected to information integration and text conversion to generate interactive results corresponding to the user instructions.

7. The human-computer interaction method based on a large language model as claimed in claim 6, characterized in that: The using the tool to process each of the subtasks in the subtasks according to the execution mode to obtain a processing result corresponding to the subtask includes: In the case where the execution mode is the first mode, executing the subtask according to the first mode, obtaining a processing result for display, and receiving first feedback information input based on the processing result; In the case where the execution mode is the second mode, determining a first subtask in the subtasks, generating an information supplement instruction according to the first subtask, and receiving second feedback information input in response to the information supplement instruction; When the execution mode is the third mode, each subtask in the subtask is executed according to the third mode to obtain a processing result.

8. The human-computer interaction method based on a large language model as claimed in claim 7, characterized in that: After the tool is used to process each of the subtasks in the execution mode to obtain the processing result corresponding to the subtask, the method further includes: When the execution mode is the first mode or the second mode, displaying the processing result and receiving adjustment information input based on the processing result; Performing semantic recognition on the adjustment information to obtain adjustment semantics corresponding to the adjustment information; A subtask to be adjusted corresponding to the adjustment semantics is determined in the subtasks, and a processing result corresponding to the adjustment semantics on the subtask to be adjusted is adjusted.

9. The human-computer interaction method based on a large language model as claimed in claim 1, characterized in that: After the tool is used to process each of the subtasks in the execution mode to obtain the processing result corresponding to the subtask, the method further includes: According to the processing result and the user intention, the accuracy of the tool of each subtask in the subtask is judged to determine whether there is an erroneous tool; In the case where it is determined that an erroneous tool exists, the step of determining tool information corresponding to the subtask is performed.

10. The human-computer interaction method based on a large language model according to claim 1, characterized in that: The method further comprises: When the subtask is executed according to the execution mode, if it is determined that there is a second subtask in the subtask but no corresponding processing result, the step of determining tool information corresponding to the subtask is executed.

11. A human-computer interaction device based on a large language model, characterized in that: include: An instruction receiving module is used to obtain an input user instruction, and use a large language model to perform intent recognition and task splitting on the user instruction to obtain the user intent and subtasks corresponding to the user instruction; A method determination module, used to determine the execution method of the user instruction according to the user intention and the subtask; A tool determination module, used to determine tool information corresponding to the subtask, wherein the tool information includes tool category and tool parameters; The instruction execution module is used to process the subtask according to the execution mode using the tool corresponding to the tool information, and generate an interaction result corresponding to the user instruction according to the result obtained by the processing.

12. An electronic device, characterized in that: The electronic device comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the method according to any one of claims 1 to 10 when executing the computer program.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 10 are implemented.

Images