A method, medium, device and product for calling tools in task-based dialogue

Abstract

This application relates to the field of intelligent interaction technology, specifically providing a method, medium, device, and product for invoking tools in task-based conversations. The method, applied to smart glasses, includes: obtaining the content of multiple rounds of conversations between a user and a pre-trained large language model; wherein the pre-trained large language model contains multiple different application scenarios; marking tool invocation prompt words based on the multiple rounds of conversation content; using the pre-trained large language model, outputting and invoking a target tool in the scenario corresponding to the tool invocation prompt word; the target tool includes: a tool name, a tool parameter name, and a tool parameter value; the target tool includes application software corresponding to the multiple different application scenarios, as well as a function control tool for the application software. Some embodiments of this application can implement task-based multi-round conversation interactions to meet the user's conversation needs.

Description

Technical Field

[0001] The present application relates to the field of intelligent interaction technology, and more specifically, to a method, medium, device, and product for invoking a tool in a task-based conversation. Background Art

[0002] Smart glasses are a general term for wearable eyewear devices that have independent operating systems, similar to smartphones, and can be installed with software to perform various functions. Currently, there are MR glasses (Mixed Reality Glasses), VR glasses, and AR glasses on the market. MR glasses are the hardware carrier of mixed reality technology, capable of deeply integrating virtual objects with the real world and enabling interaction. Their technical features lie somewhere between VR (virtual reality) and AR (augmented reality), expanding the dimension of human-computer interaction through the integration of virtual and real. To enhance the user experience, smart glasses can interact with users through voice to meet their daily needs.

[0003] Currently, users can only interact with smart glasses through voice, controlling them to perform corresponding operations in corresponding scenarios through voice commands, but they cannot actively trigger the functional tools that users want to use.

[0004] Therefore, how to provide a technical solution for an efficient method of calling tools in task-based dialogues has become a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The purpose of some embodiments of the present application is to provide a method, medium, device and product for tool calling in task-based dialogues. Through the technical solutions of the embodiments of the present application, active calling of relevant tools in the process of task-based interaction with users can be achieved based on a large language model. The calling efficiency is high and accurate, which improves the user experience.

[0006] In a first aspect, some embodiments of the present application provide a method for tool calling in a task-based conversation, which is applied to smart glasses, including: obtaining the content of multiple rounds of conversations between a user and a pre-trained large language model; wherein the pre-trained large language model contains multiple different application scenarios; based on the content of the multiple rounds of conversations, marking tool call prompt words; the tool call prompt words represent user intentions; using the pre-trained large language model, outputting and calling a target tool in the scene corresponding to the tool call prompt words; the target tool includes: a tool name, a tool parameter name, and a tool parameter value; the target tool includes application software corresponding to the multiple different application scenarios, and a function control tool of the application software.

[0007] Some embodiments of this application identify tool invocation prompts by marking multiple rounds of conversations between a user and a pre-trained large language model. The pre-trained large language model is then used to output and invoke the corresponding target tool. These embodiments can proactively and efficiently invoke the target tool based on the large language model and user conversations, making them suitable for task-based conversational interaction scenarios and enhancing the user interaction experience.

[0008] In some embodiments, the use of the pre-trained large language model to output and call the target tool in the scene corresponding to the tool call prompt word includes: determining the scene in which the user is located based on the content of the multiple rounds of conversations; and using the pre-trained large language model to select the target tool from the tool library corresponding to the scene.

[0009] Some embodiments of the present application can achieve accurate calling of tools by determining the scene in which the user is located and selecting the target tool from the corresponding tool library.

[0010] In some embodiments, before obtaining the content of multiple rounds of conversations between the user and the pre-trained large language model, the method further includes: obtaining a training data set, wherein the training data set includes multiple rounds of conversation samples and tool call results; and using the training data set to train an initial large language model to obtain the pre-trained large language model.

[0011] Some embodiments of the present application train an initial large language model using a training dataset to obtain a pre-trained large language model, providing a model basis for tool calls.

[0012] In some embodiments, obtaining a training data set includes: inputting a conversation target into a conversation generation large language model to obtain a conversation output result; wherein the conversation output result includes conversation content, conversation state, and action data; inputting a historical conversation sample corresponding to the action data in the conversation output result into a tool call model to obtain the tool call result; using the conversation generation large language model to extract the conversation state for each round of conversation in the conversation output result to obtain extracted data; and comparing the tool call result with the conversation state, the extracted data, and the action data respectively to determine the multi-round conversation samples.

[0013] Some embodiments of the present application select multi-round dialogue samples based on the content output by the dialogue-generated large language model and the tool-calling model to ensure the quality and accuracy of the samples.

[0014] In some embodiments, comparing the tool call result with the dialogue state, the extracted data, and the action data to determine the multi-round dialogue sample includes: confirming that the tool call result is consistent with at least two of the dialogue state, the extracted data, and the action data.Figure 1 If the dialogue output result is the same, the dialogue output result is used as a multi-round dialogue sample.

[0015] Some embodiments of the present application compare tool call results with other conversation information to determine multi-round conversation samples, thereby ensuring the accuracy of the samples.

[0016] In some embodiments, the using the training data set to train the initial large language model to obtain the pre-trained large language model includes: using the text content in the multi-round dialogue samples as the input of the initial large language model, and the tool call result as the output of the initial large language model to perform model training to obtain the pre-trained large language model.

[0017] Some embodiments of the present application use the text content of multiple rounds of dialogue samples and tool call results as the input and output of the training model to achieve effective training of the model.

[0018] In some embodiments, the database of the pre-trained large language model stores historical multi-round dialogue data in different scenarios; when the pre-trained large language model switches to a historical scenario again, the historical multi-round dialogue data corresponding to the historical scenario is called.

[0019] Some embodiments of the present application can provide a reference basis for subsequent conversations by storing historical multi-round conversation data in different scenarios, thereby achieving accurate services for users.

[0020] In a second aspect, some embodiments of the present application provide a device for calling tools in task-based conversations, which is applied to smart glasses and includes: a conversation acquisition module for acquiring the content of multiple rounds of conversations between a user and a pre-trained large language model; wherein the pre-trained large language model contains multiple different application scenarios; a marking module for marking tool call prompt words based on the content of the multiple rounds of conversations; the tool call prompt words represent user intentions; a tool calling module for using the pre-trained large language model to output and call a target tool in the scene corresponding to the tool call prompt word; the target tool includes: a tool name, a tool parameter name, and a tool parameter value; the target tool includes application software corresponding to the multiple different application scenarios, and a function control tool of the application software.

[0021] In a third aspect, some embodiments of the present application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the method described in any embodiment of the first aspect.

[0022] In a fourth aspect, some embodiments of the present application provide an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor can implement a method as described in any embodiment of the first aspect when executing the program.

[0023] In a fifth aspect, some embodiments of the present application provide a computer program product, comprising a computer program, wherein the computer program, when executed by a processor, can implement the method described in any embodiment of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of some embodiments of the present application, the following is a brief introduction to the drawings required for use in some embodiments of the present application. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0025] Figure 1 A system diagram of tool calls in task-based conversations provided for some embodiments of the present application;

[0026] Figure 2 A flowchart of a method for obtaining a pre-trained large language model provided in some embodiments of the present application;

[0027] Figure 3 One of the flow charts of the method for calling a tool in a task-based dialogue provided in some embodiments of the present application;

[0028] Figure 4 A second flowchart of a method for calling a tool in a task-based conversation provided in some embodiments of the present application;

[0029] Figure 5 A block diagram of the apparatus for invoking a tool in a task-based conversation provided in some embodiments of the present application;

[0030] Figure 6 A schematic diagram of an electronic device is provided for some embodiments of the present application. DETAILED DESCRIPTION

[0031] The technical solutions in some embodiments of the present application will be described below in conjunction with the drawings in some embodiments of the present application.

[0032] It should be noted that similar reference numerals and letters represent similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and should not be understood as indicating or implying relative importance.

[0033] Task-based dialogue systems help users complete specific tasks through multi-round conversations. They are primarily used in vertical fields such as appointment booking, inquiries, and payments. Their core focus is understanding user intent, tracking conversation status, and generating effective responses. Currently, smart glasses can only complete tasks based on user instructions, without the ability to intelligently invoke tools within task-based conversations.

[0034] In view of this, some embodiments of the present application provide a method for tool calling in task-based conversations, which is used on smart glasses; the pre-trained large language model deployed inside the smart glasses can interact with the user through voice, and through the interaction between the user and the system, multiple rounds of conversation content can be obtained first; then, tool call prompt words are marked in the multiple rounds of conversation content; finally, the pre-trained large language model is used to call the target tool related to the tool call prompt word, so as to complete the intelligent calling of the target tool and make accurate task responses for the user.

[0035] The following is combined with Figure 1 The overall composition structure of the system for tool invocation in task-based dialogue provided by some embodiments of the present application is exemplified.

[0036] like Figure 1 As shown, some embodiments of the present application provide a system diagram for tool calling in a task-based conversation. The system for tool calling in the task-based conversation may include: MR glasses 100 (as a specific example of smart glasses) and a tool calling terminal 200. The tool calling terminal 200 can be deployed in the internal processor of the MR glasses, or deployed in the cloud to communicate with the MR glasses. Specifically, the user can interact with the system in the tool calling terminal 200 through voice; that is, the user and the system are deployed with a pre-trained large language model for interaction. After obtaining the content of multiple rounds of conversations between the user and the system, the pre-trained large language model can recognize the intent of the multiple rounds of conversation content and mark the tool calling prompt words; finally, the pre-trained large language model can call the target tool corresponding to the tool calling prompt word in the scene where the user is located.

[0037] In some embodiments of the present application, the type of smart glasses can be flexibly selected, but the embodiments of the present application are not limited thereto.

[0038] It should be noted that the following embodiments are all described by taking the example of the tool calling end 200 being deployed in the internal processor of the MR glasses.

[0039] In order to achieve accurate calling of tools in task-based conversations, it is first necessary to obtain a pre-trained large language model through model training.

[0040] The following is combined with Figure 2 The implementation process of obtaining a pre-trained large language model performed by the smart glasses 100 provided in some embodiments of the present application is exemplified.

[0041] Please see the attached Figure 2 , Figure 2 A flowchart of a method for obtaining a pre-trained large language model is provided for some embodiments of the present application.

[0042] Before executing the following method, in a specific embodiment of the present application, a tool call model is first constructed. Specifically, the tool call mark is defined in the input and output of the LLM, such as the prompt in the input <tools>< / tools> The callable tool as the starting mark includes the tool name, function description, parameter description, and whether the parameter is required or optional.<tool_call>< / tool_call> The tool to be called as the starting tag includes the tool name, parameter name and parameter value. The tag that defines the thinking process is as follows <think>< / think> For example, the input is a list of dialog interactions, such as:

[0043] The input is:

[0044] {'user': 'Open photo'}

[0045] The output is:

[0046] {'assistant': {'name': 'app_open', 'arguments': '{"app_name": "Camera"}'}}

[0047] The input is:

[0048] {'user': 'Close photo taking'}

[0049] The output is text containing tool tag json: such as:

[0050] {'assistant': {'name': 'app_close', 'arguments': '{"app_name": "Camera"}'}}.

[0051] Secondly, build tool libraries for different scenarios in LLM. Define different tool libraries according to different application scenarios. Application scenarios can include application management, volume management, brightness management, device management tools, as well as application scenario management such as navigation, travel, and watching movies. Use multiple tool libraries to form a scene agent (i.e., LLM). For example, the agent of the movie scene includes tools related to movie search and playback, as well as application management, volume management, brightness management, and device management tools, and prepares for volume and brightness adjustment and scene switching commands that users may issue. Depending on the scene the user is in, they can enter the corresponding scene in LLM. It should be noted that the agent is based on LLM and belongs to a set of tool calling processes in a scene.

[0052] In some embodiments of the present application, the method for obtaining a pre-trained large language model may include:

[0053] S210: Acquire a training data set, wherein the training data set includes multiple rounds of dialogue samples and tool call results.

[0054] For example, in a specific embodiment of the present application, a training dataset is formed by obtaining multi-round dialogue samples and tool call results, wherein the multi-round dialogue samples can be generated based on a single-round intention tool call corpus.

[0055] Specifically, to enhance the processing capabilities of single-round interactions, we generated a rich set of single-round interaction samples. For example, to enable scene switching within various scene agents, we enhanced the application management tool to generalize the application opening function. For example, the query "I want to buy a phone" can now be interpreted as a shopping intent, rather than being limited to actions like "open the shopping app."

[0056] When designing prompts for input into the agent, we generated multiple roles and context segments and incorporated them into the prompts, triggering the opening of various applications in as diverse a manner as possible. A prompt might be something like: "You are a {1}, using the following text as a reference {2}. If you want to open the shopping app, you should say to the device:" {1} could also be "a math teacher who is proficient in erhu playing," and {2} could be task-related or unrelated, such as "Xiaoming is late for class today."

[0057] Subsequently, we used the conversation generation LLM (a specific example of a large language model for conversation generation) and the tool invocation LLM (a specific example of a tool invocation model) to generate multiple rounds of conversation samples. It is understandable that the tool invocation LLM is a commonly used tool on the market, and its accuracy is difficult to guarantee.

[0058] In some embodiments of the present application, S210 may include: inputting the dialogue target into a dialogue generation large language model to obtain a dialogue output result; wherein the dialogue output result includes dialogue content, dialogue state and action data; inputting the historical dialogue sample corresponding to the action data in the dialogue output result into a tool call model to obtain the tool call result; using the dialogue generation large language model to extract the dialogue state for each round of dialogue in the dialogue output result to obtain extracted data; comparing the tool call result with the dialogue state, the extracted data and the action data respectively to determine the multi-round dialogue samples.

[0059] For example, in a specific embodiment of the present application, a conversation target is randomly generated, using the target as a prompt. The conversation generation LLM generates data containing conversation content, conversation state, and action data, which is referred to as generated data A. At the location where the action data is generated, the historical conversation is used as input to the tool invocation LLM, generating tool invocation results as data B. The conversation generation LLM extracts each round of conversation, generating extracted conversation states (as a specific example of extracted data) as data C. Data B is then compared with A and C to determine whether to retain the conversation content as a multi-round conversation sample. The purpose of this comparison and verification is to eliminate samples with incorrect results output by the tool invocation LLM and ensure the accuracy of the training dataset. The conversation state is used to identify user intent. For example, if the user intent is to open music software, the tool invocation result B is also to open music software. Conversation targets can correspond to different content in different application scenarios. By randomly setting conversation targets for different scenarios, the initial large language model can be trained to accurately call tools in different scenarios. In other words, based on the conversation targets, the initial large language model can be placed in its corresponding scenario. This training process allows each scenario to correspond to a set of call flows in the pre-trained large language model.

[0060] The dialogue generation LLM and the tool call LLM serve the same basic LLM. The tool call adds a tool configuration LLM on top of the basic LLM. Therefore, you only need to deploy one LLM. When using the tool call function, you can use the tool tag in the prompt word to distinguish them.

[0061] In some embodiments of the present application, S210 may include: confirming that the tool call result is consistent with at least two of the dialogue state, the extracted data, and the action data. Figure 1 If the dialogue output result is the same, the dialogue output result is used as a multi-round dialogue sample.

[0062] For example, in a specific embodiment of the present application, the tool call result B is compared with the generated dialogue state in A, the extracted dialogue state C, and the action data in A. If the user intention located by the two data is Figure 1If the conversation is consistent, the conversation content is used as a multi-round conversation sample; otherwise, it is discarded.

[0063] Through comparative verification, a training data set with higher accuracy can be obtained, which facilitates the subsequent accurate training of the model.

[0064] S220: Train the initial large language model using the training data set to obtain the pre-trained large language model.

[0065] For example, in a specific embodiment of the present application, the initial large language model is trained using the multi-round dialogue samples and tool call results obtained above to obtain a pre-trained large language model.

[0066] In some embodiments of the present application, S220 may include: using the text content in the multi-round dialogue samples as the input of the initial large language model, and the tool call result as the output of the initial large language model to perform model training to obtain the pre-trained large language model.

[0067] For example, in a specific embodiment of the present application, the above-constructed tool call tag is used to mark and convert the content in multiple rounds of dialogue samples, which is used as the input of the initial large language model. The above-mentioned tool call result is used as the output of the initial large language model, thereby realizing model training and obtaining a pre-trained large language model.

[0068] This pre-trained large language model is more sophisticated than the above-mentioned general tool call LLM. The pre-trained large language model of this application can set different calling schemes for different scenarios in which the user is in. During the interaction between the user and the pre-trained large language model, it can accurately identify the user's intention and realize accurate calling of the tool; it can also switch scenes according to the content of multiple rounds of conversations during the interaction with the user, and realize tool calling after the scene switching, thereby improving the accuracy of tool calling.

[0069] The following is combined with Figure 3 The specific process of tool invocation in a task-based conversation performed by the smart glasses 100 provided in some embodiments of the present application is exemplified.

[0070] Please see the attached Figure 3 , Figure 3 A flowchart of a method for invoking a tool in a task-based conversation is provided for some embodiments of the present application. The method for invoking a tool in a task-based conversation may include:

[0071] S310, obtaining multiple rounds of conversation content between the user and a pre-trained large language model; wherein the pre-trained large language model contains multiple different application scenarios.

[0072] For example, in a specific embodiment of the present application, a user can interact with the pre-trained large language model obtained in the above embodiment, and the pre-trained large language model obtains the content of multiple rounds of conversations in real time.

[0073] S320: Marking tool invocation prompt words based on the multiple rounds of conversation content. The tool invocation prompt words represent the user's intention.

[0074] For example, in a specific embodiment of the present application, the pre-trained large language model deployed by the smart glasses 100 can identify user intent in multiple rounds of conversation while interacting with the user, marking tool call prompts present in the conversation. For example, if the user says "volume up" and "brightness down," two tool call prompts, "volume up" and "brightness down," can be marked. The number of tool call prompts is determined based on the identification of multiple rounds of conversation content and is not specifically limited in this embodiment of the present application.

[0075] S330, using the pre-trained large language model, output and call the target tool in the scene corresponding to the tool call prompt word; the target tool includes: tool name, tool parameter name and tool parameter value; the target tool includes application software corresponding to the multiple different application scenarios, and the function control tool of the application software.

[0076] For example, in a specific embodiment of the present application, after labeling is completed, the pre-trained large language model can call the corresponding target tool in the tool library corresponding to the scene of the current user based on the scene. For example, the target tools are the volume adjustment tool and the brightness adjustment tool (as a specific example of the tool name), and the volume or brightness (as a specific example of the tool parameter name) can be adjusted accordingly according to the set unit adjustment value (as a specific example of the tool parameter value).

[0077] In some embodiments of the present application, S330 may include: determining the scenario in which the user is located based on the multi-round conversation content; and selecting the target tool from the tool library corresponding to the scenario using the pre-trained large language model.

[0078] For example, in a specific embodiment of the present application, the user's scene can be located through the content of multiple rounds of conversations between the user and a pre-trained large language model; or, the user's scene can be located based on the application software entered by the user; the specific scene can be determined based on actual conditions, and the embodiments of the present application are not limited to this.

[0079] For example, a multi-turn conversation example:

[0080] User: Can you find me a Romance movie to watch?

[0081] Action = Provide Slot, Movie Theme = Romance, Operation = Find Movie

[0082] System: How about After? It is one of the 5 movies that I have found, and has an average rating of 5.8.

[0083] Action = Make a Suggestion, Movie Name = After, Rating = 5.80

[0084] Search results = 5 movies found

[0085] User: What about the other movies?

[0086] Action = Request Other

[0087] System: I also found Pillow Talk that has a rating of 7.5.

[0088] Action = Making a Suggestion, Movie Name = Pillow Talk, Rating = 7.50

[0089] User: Sure.

[0090] Action = Confirm.

[0091] The above conversation indicates that the user is currently using a video application and needs to search for a movie. The action "search" can be used as a tool-invoking prompt; by invoking the search tool (as a specific example of a target tool) in the video application, the movie search can be performed.

[0092] In some embodiments of the present application, the database of the pre-trained large language model stores historical multi-round dialogue data in different scenarios; when the pre-trained large language model switches to a historical scenario again, the historical multi-round dialogue data corresponding to the historical scenario is called.

[0093] For example, in a specific embodiment of the present application, within a scene in the pre-trained large language model, it can receive up to N rounds of historical conversations (users and pre-trained large language models). After processing the system-level tools, the input of the pre-trained large language model should remove this part of the conversation and continue the conversation within the scene. When the user switches to a new scene (such as switching from video software to music software), the historical conversations of the old scene (i.e., historical multi-round conversation data) should be saved; when entering the old scene again (i.e., returning to the video software), it should be retrieved. The tool calls in the historical multi-round conversation data of the current user are saved in the database; the user preferences are determined based on the historical multi-round conversation data and used as a reference in subsequent conversation recommendations. For example, if the user is accustomed to adjusting the brightness according to the external environment, the brightness adjustment tool can be stored in the database for the user's subsequent use.

[0094] The following is combined with Figure 4 The specific process of tool invocation in task-based dialogue provided by some embodiments of the present application is exemplified.

[0095] Please see the attached Figure 4 , Figure 4 A flowchart of a method for calling a tool in a task-based conversation is provided for some embodiments of the present application.

[0096] S410: Generate multiple rounds of dialogue data using a conventional dialogue generation LLM, and determine a tool calling result using a general LLM tool calling function.

[0097] S420 , comparing and verifying the dialogue states, action data, and extracted dialogue states in the multi-round dialogue data with the tool call results to determine a training data set.

[0098] S430: Train the initial large language model using the training data set to obtain a pre-trained large language model.

[0099] S440: Obtain multiple rounds of conversation content between the user and the pre-trained large language model, and mark tool call prompt words based on the multiple rounds of conversation content.

[0100] S450: Using a pre-trained large language model, output and call a target tool in a scene corresponding to the tool call prompt word.

[0101] It can be understood that the specific implementation process of S410~S450 can refer to the method embodiment provided above. To avoid repetition, detailed description is appropriately omitted here.

[0102] Please refer to Figure 5 , Figure 5A block diagram illustrating the components of a tool-invoking apparatus in a task-based conversation provided by some embodiments of the present application is provided. It should be understood that the tool-invoking apparatus in this task-based conversation corresponds to the aforementioned method embodiment and is capable of executing each step involved in the aforementioned method embodiment. The specific functions of the tool-invoking apparatus in this task-based conversation can be found in the description above, and a detailed description is omitted here to avoid repetition.

[0103] Figure 5 The device for calling a tool in a task-based dialogue includes at least one software function module that can be stored in a memory in the form of software or firmware or solidified in the device for calling a tool in a task-based dialogue. The device for calling a tool in a task-based dialogue is applied to smart glasses and includes: a dialogue acquisition module 510, which is used to obtain the content of multiple rounds of dialogues between a user and a pre-trained large language model; wherein the pre-trained large language model contains multiple different application scenarios; a marking module 520, which is used to mark tool call prompt words based on the content of the multiple rounds of dialogues; the tool call prompt words represent user intentions; a tool calling module 530, which is used to use the pre-trained large language model to output and call a target tool in the scene corresponding to the tool call prompt word; the target tool includes: a tool name, a tool parameter name and a tool parameter value; the target tool includes application software corresponding to the multiple different application scenarios, and a function control tool of the application software.

[0104] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method, and will not be described in detail here.

[0105] Some embodiments of the present application further provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the operations corresponding to any of the above methods provided in the above embodiments.

[0106] Some embodiments of the present application further provide a computer program product, which includes a computer program, wherein when the computer program is executed by a processor, it can implement the operations corresponding to any of the above methods provided in the above embodiments.

[0107] like Figure 6 As shown, some embodiments of the present application provide an electronic device 600, which includes: a memory 610, a processor 620, and a computer program stored in the memory 610 and executable on the processor 620, wherein the processor 620 can implement a method as described in any of the above embodiments when reading the program from the memory 610 through the bus 630 and executing the program.

[0108] Processor 620 can process digital signals and can include various computing architectures, such as a complex instruction set computer architecture, a reduced instruction set computer architecture, or an architecture that implements a combination of multiple instruction sets. In some examples, processor 620 can be a microprocessor.

[0109] The memory 610 can be used to store instructions executed by the processor 620 or data related to the execution of instructions. These instructions and / or data may include code for implementing some or all functions of one or more modules described in the embodiments of this application. The processor 620 of the embodiment of the present disclosure can be used to execute the instructions in the memory 610 to implement the method shown above. The memory 610 includes dynamic random access memory, static random access memory, flash memory, optical storage, or other memory known to those skilled in the art.

[0110] The foregoing is merely an embodiment of the present application and is not intended to limit the scope of protection of the present application. Various modifications and variations are possible for those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included within the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined or explained in subsequent figures.

[0111] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0112] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

Claims

1. A method for calling a tool in a task-based dialogue, characterized in that: The method is applied to smart glasses and includes: Obtaining multiple rounds of conversations between a user and a pre-trained large language model; wherein the pre-trained large language model contains multiple different application scenarios; Based on the content of the multiple rounds of conversations, marking tool invocation prompt words; the tool invocation prompt words represent the user's intention; Outputting and calling a target tool in a scenario corresponding to the tool calling prompt word using the pre-trained large language model; the target tool includes: a tool name, a tool parameter name, and a tool parameter value; the target tool includes application software corresponding to the multiple different application scenarios, and a function control tool of the application software; Before obtaining the multi-round conversation content between the user and the pre-trained large language model, the method further includes: obtaining a training dataset, wherein the training dataset includes multi-round conversation samples and tool call results; training an initial large language model using the training dataset to obtain the pre-trained large language model; The obtaining of the training data set includes: inputting the dialogue target into a dialogue generation large language model to obtain a dialogue output result; wherein the dialogue output result includes dialogue content, dialogue state and action data; inputting historical dialogue samples corresponding to the action data in the dialogue output result into a tool call model to obtain the tool call result; using the dialogue generation large language model to extract the dialogue state for each round of dialogue in the dialogue output result to obtain extracted data; and comparing the tool call result with the dialogue state, the extracted data and the action data respectively to determine the multi-round dialogue samples.

2. The method according to claim 1, wherein The method of using the pre-trained large language model to output and call a target tool in a scene corresponding to the tool call prompt word includes: Determining the scenario in which the user is located based on the contents of the multiple rounds of conversations; The pre-trained large language model is used to select the target tool from a tool library corresponding to the scenario.

3. The method according to claim 1, wherein The comparing the tool call result with the dialogue state, the extracted data, and the action data to determine the multi-round dialogue samples includes: If it is confirmed that the tool call result is consistent with at least two intentions among the dialogue state, the extracted data, and the action data, the dialogue output result is used as a multi-round dialogue sample.

4. The method according to claim 1 or 3, wherein: The step of training the initial large language model using the training data set to obtain the pre-trained large language model includes: The text content in the multi-round dialogue samples is used as the input of the initial large language model, and the tool call result is used as the output of the initial large language model for model training to obtain the pre-trained large language model.

5. The method according to claim 1 or 2, wherein: The database of the pre-trained large language model stores historical multi-round dialogue data in different scenarios; when the pre-trained large language model switches to a historical scenario again, the historical multi-round dialogue data corresponding to the historical scenario is called.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program is executed by a processor to perform the method according to any one of claims 1 to 5.

7. An electronic device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the computer program executes the method according to any one of claims 1 to 5 when run by the processor.

8. A computer program product, characterized in that The computer program product comprises a computer program, wherein the computer program is executed by a processor to perform the method according to any one of claims 1 to 5.

Images