Tool screening method and device, electronic equipment, storage medium and program product

Abstract

The invention discloses a tool screening method and device, electronic equipment, a storage medium and a program product. The tool screening method comprises the steps that historical calling data corresponding to a user, a preset tool library and scene tags of all tools to be called in the preset tool library are obtained; determining a tool calling scene and a preset calling priority strategy of each to-be-called tool in a tool calling request initiated by a user; determining a candidate calling tool set in the to-be-called tools based on the historical calling data, the scene labels, the tool calling scenes and a preset calling priority strategy; and calling a preset large language model to determine a target calling tool in the candidate calling tool set. The tools are screened by integrating the historical calling data of the user, the scene label of each to-be-called tool, the preset priority strategy and other multi-dimensional information, it can be ensured that the screened candidate tools not only conform to the past use habits of the user, but also adapt to the calling scene of the current calling tool of the user, and the user experience is improved. And the suitability and the accuracy of tool screening are improved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a tool screening method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] In recent years, the capabilities of large language models have expanded from natural language generation to complex functions such as reasoning, planning, and code generation. However, large language models are limited by training data and need to rely on tool enhancement techniques to call external tools to overcome their functional limitations. Traditional tool calling methods obtain the feature vectors of user requests and tools, and then filter out matching tools through feature matching search. However, existing solutions have obvious technical flaws: on the one hand, they only rely on the feature similarity between user requests and tools for matching, without considering the relevance of tools to the user's current task scenario; on the other hand, they lack analysis of users' historical calling behavior and fail to combine users' past tool usage habits and preferences to comprehensively select tools. Summary of the Invention

[0003] This invention provides a tool screening method, apparatus, electronic device, storage medium, and program product to improve the adaptability and accuracy of tool screening.

[0004] In one aspect, the present invention provides a tool screening method, comprising:

[0005] Obtain the user's historical call data, preset tool library, and scene tags for each tool to be called within the preset tool library; wherein each tool to be called corresponds to at least one scene tag;

[0006] Obtain tool call requests initiated by users, and determine the user's tool call scenario and the preset call priority strategy for each tool to be called based on the tool call requests;

[0007] Based on historical call data, scenario tags corresponding to each tool to be called, tool call scenarios, and preset call priority strategies, a set of candidate tools for calling is determined from each tool to be called;

[0008] Invoke the preset large language model, and determine the target invoking tool from the candidate invoking tool set based on the preset large language model.

[0009] In one aspect of this invention, a tool screening device is provided, comprising:

[0010] The tag acquisition module is used to acquire the user's corresponding historical call data, preset tool library, and scene tags of each tool to be called in the preset tool library; wherein, each tool to be called corresponds to at least one scene tag;

[0011] The scenario determination module is used to obtain the tool call request initiated by the user, and determine the user's tool call scenario and the preset call priority strategy of each tool to be called based on the tool call request;

[0012] The toolset determination module is used to determine the candidate toolset for each tool to be called based on historical call data, the scenario tags corresponding to each tool to be called, the tool call scenario, and the preset call priority strategy.

[0013] The target tool determination module is used to call a preset large language model and determine the target calling tool from the candidate calling tool set based on the preset large language model.

[0014] In another aspect of the present invention, an electronic device is provided, comprising:

[0015] At least one processor; and

[0016] Memory that is communicatively connected to at least one processor;

[0017] The memory stores a computer program that can be executed by at least one processor, which enables the at least one processor to perform the tool screening method of any embodiment of the present invention.

[0018] In another aspect of the present invention, a computer-readable storage medium is provided, comprising: computer instructions, which enable a processor to execute the tool screening method of any embodiment of the present invention when executed.

[0019] In another aspect of the present invention, a computer program product is provided, comprising: a computer program configured to enable a processor to execute the tool screening method of any embodiment of the present invention when executed.

[0020] This invention acquires a user's historical usage data, a preset tool library, and scenario tags for each tool to be invoked within the preset tool library; each tool to be invoked corresponds to at least one scenario tag; it acquires the tool invocation request initiated by the user and determines the user's tool invocation scenario and the preset invocation priority strategy for each tool to be invoked based on the tool invocation request; based on the historical usage data, the scenario tags corresponding to each tool to be invoked, the tool invocation scenario, and the preset invocation priority strategy, it determines a candidate tool set among the tools to be invoked; it invokes a preset large language model and determines the target tool to be invoked from the candidate tool set based on the preset large language model. This invention filters tools by comprehensively considering multi-dimensional information such as the user's historical usage data, the scenario tags of each tool to be invoked, and the preset priority strategy, ensuring that the selected candidate tools not only conform to the user's past usage habits but also adapt to the user's current tool invocation scenario, thus improving the adaptability and accuracy of tool selection.

[0021] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a flowchart of a tool screening method provided according to Embodiment 1 of the present invention;

[0024] Figure 2 This is a flowchart of another tool screening method provided according to Embodiment 2 of the present invention;

[0025] Figure 3 This is a flowchart of another tool screening method provided according to Embodiment 3 of the present invention;

[0026] Figure 4 This is a flowchart of a toolkit for constructing candidate invocations according to Embodiment 3 of the present invention;

[0027] Figure 5 This is a structural diagram of a tool screening device according to Embodiment 4 of the present invention;

[0028] Figure 6 This is an electronic device block diagram of an execution tool screening method provided in Embodiment 5 of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, apparatus, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0031] Figure 1 This invention provides a flowchart of a tool selection method. This invention is applicable to scenarios where, based on the user's actual task processing needs, a matching target tool is accurately searched from a preset tool library containing various tools, and then the user's actual task processing needs are met by invoking the target tool. This method can be executed by a tool selection device, which can be implemented in hardware and / or software. This tool selection device can be configured in electronic devices such as semiconductor testing servers or distributed computing clusters composed of multiple servers. Figure 1 As shown, the method includes:

[0032] S110. Obtain the user's corresponding historical call data, preset tool library, and scene tags of each tool to be called in the preset tool library; wherein, each tool to be called corresponds to at least one scene tag.

[0033] Historical call data refers to a series of data that records the user's past calls to tools in the preset tool library. For example, historical call data may include the call frequency and call time interval of each tool to be called, reflecting the historical usage of each tool in the preset tool library.

[0034] A preset tool library can be understood as a pre-built resource library that stores multiple tools to be invoked. This resource library is used to provide users with multiple callable tool resources. For example, the preset tool library may include all the tools integrated in the underlying operating system that the user relies on when invoking tools to perform a target task. The tools integrated in the underlying operating system may be third-party development tools or custom functional tools.

[0035] The tools to be invoked can be understood as tools in a preset tool library that users can invoke. Each tool to be invoked in the preset tool library corresponds to at least one scenario tag. The scenario tag is used to identify the scenario in which the tool to be invoked is applicable. For example, the scenario tag of a tool to be invoked may include tags such as life services, route planning, or data processing, which can meet the needs of users in different scenarios.

[0036] Specifically, when a user invokes a tool while performing a target task, the system can extract the user's corresponding preset tool library, historical call data of each tool to be invoked in the preset tool library, and scene tags of each tool to be invoked in the preset tool library from the configuration file or user behavior database of the underlying operating system.

[0037] S120. Obtain the tool call request initiated by the user, and determine the user's tool call scenario and the preset call priority strategy of each tool to be called based on the tool call request.

[0038] In this context, a tool call request can be understood as an instruction initiated by the user that contains the user's own tool usage needs. For example, a tool call request may include information such as a preset call priority strategy or user requirement description information. The tool call request serves as a trigger condition for tool filtering, and tools can be filtered based on the preset call priority strategy or user requirement description information contained in the tool call request.

[0039] A preset invocation priority strategy can be understood as a pre-defined rule used to determine the order in which tools are invoked among multiple tools to be invoked. For example, a preset invocation priority strategy may include orders such as most frequently used, most recently used, or least recently used.

[0040] Tool call scenario refers to the specific scenario in which a user needs to use a tool. It can reflect the user's scenario requirements for calling the tool to be called. It is used together with historical call data, scenario tags corresponding to each tool to be called, and preset call priority strategies to filter the candidate tool set among each tool to be called.

[0041] Specifically, the tool call request initiated by the user can be obtained through the user interface of the underlying operating system on which the user invokes the tool when performing the target task. The tool call request initiated by the user can be parsed to obtain the preset call priority strategy used for tool invocation. Furthermore, the tool call request initiated by the user can be analyzed to obtain the user's tool invocation scenario.

[0042] S130. Based on historical call data, scenario tags corresponding to each tool to be called, tool call scenarios, and preset call priority strategies, determine the candidate tool set for each tool to be called.

[0043] The candidate call tool set can be understood as a set of tools selected from a preset tool library based on historical call data, scenario tags, tool call scenarios, and preset call priority strategies, which is used to provide the selection range for the target call tool.

[0044] Specifically, the system obtains the scenario tags of each tool to be invoked in the preset tool library. Based on the tool invocation scenario of the user, the scenario tags of each tool to be invoked in the preset tool library are compared and analyzed one by one to obtain the scenario tags that match the tool invocation scenario. The tools corresponding to the scenario tags that match the tool invocation scenario can be selected as part of the candidate tools to be invoked in the tool set. Based on the preset invocation priority strategy, the system can extract the invocation frequency, number of successful invocations, and invocation time interval of each tool to be invoked in the preset tool library from the historical invocation data. The tools to be invoked can be evaluated through a series of indicator data, and the preset tool library can be filtered according to the evaluation results to obtain another part of the candidate tools to be invoked in the tool set.

[0045] S140. Invoke the preset large language model, and determine the target invoking tool from the candidate invoking tool set based on the preset large language model.

[0046] The preset large language model can be understood as a large language model that has been trained and has the ability to understand natural language and perform logical reasoning. For example, the preset large language model may include models such as Llama 2 or Gemma 1. The preset large language model is used to filter target calling tools in the candidate calling tool set.

[0047] The target invocation tool can be understood as the tool that best meets the user's tool invocation request requirements, determined by a pre-set large language model from the candidate invocation tool set. As the final output of the tool screening process, the target invocation tool is used to directly serve the user's actual tool invocation needs.

[0048] Specifically, a pre-built tool filtering prompt word for a large language model can be constructed. The pre-built large language model can be trained based on pre-built training data and tool filtering prompt words. The pre-trained pre-built large language model can be called. The scene tags of the tools to be called in the candidate tool set, the tool call request initiated by the user, the pre-built call priority strategy, and the user's corresponding historical call data are used as input parameters for the pre-built large language model. The pre-built large language model can be guided by the pre-built tool filtering prompt words to filter the tools to be called in the candidate tool set, and the target call tool that meets the user's current needs is determined from the candidate tool set.

[0049] For example, the components of a tool that pre-builds a pre-defined large language model for filtering prompts may include: task instructions, input information, filtering rules, output information, etc.

[0050] For example, the task instruction could be "Filter the target calling tools that match the user's data statistical analysis needs from the candidate tools set", or "Determine the target calling tool from the candidate tools based on the preset calling priority strategy and scene tag matching degree". The input information may include: the scene tag of the tool to be called, the tool calling request initiated by the user, the preset calling priority strategy or the user's corresponding historical calling data, etc. The filtering rules may include: "Prioritize filtering based on the matching degree between the tool calling scene and the scene tag", or "Sort the tools to be called in descending order according to the historical calling frequency, and select the tool with the highest frequency as the first choice". The output information is the target tool identifier or the target tool name.

[0051] This invention involves acquiring a user's historical usage data, a preset tool library, and scenario tags for each tool to be invoked within the preset tool library; each tool to be invoked corresponds to at least one scenario tag; acquiring a tool invocation request initiated by the user, and determining the user's tool invocation scenario and a preset invocation priority strategy for each tool to be invoked based on the tool invocation request; determining a candidate tool set among the tools to be invoked based on the historical usage data, the scenario tags corresponding to each tool to be invoked, the tool invocation scenario, and the preset invocation priority strategy; invoking a preset large language model, and determining the target tool to be invoked from the candidate tool set based on the preset large language model. This invention, by comprehensively considering multi-dimensional information such as the user's historical usage data, the scenario tags of each tool to be invoked, and the preset priority strategy for tool selection, ensures that the selected candidate tools not only conform to the user's past usage habits but also fit the user's current tool invocation scenario, thus improving the adaptability and accuracy of tool selection.

[0052] Furthermore, in this embodiment of the invention, step S120 is further refined. Specifically, the steps for determining the user's tool invocation scenario based on the tool invocation request are detailed, including:

[0053] Call a preset text classification model; wherein the training labels of the preset text classification model include at least one scene training label; input the tool call request into the preset text classification model to obtain the scene matching score of the tool call request under each scene training label of the preset text classification model; compare each scene matching score with a preset scene matching score threshold, and take the label content corresponding to the scene label that is higher than the preset threshold as the tool call scene.

[0054] The pre-trained text classification model can be understood as a pre-trained machine learning model that can identify the scene category to which the input text belongs and output the matching value between the input text and each scene category. In other words, each scene category can be determined based on the scene training labels defined by the pre-trained text classification model. That is, the scene training labels used by the pre-trained text classification model during training directly correspond to the various scene categories that can be identified in the end. During the training process of the pre-trained text classification model, it can be trained based on at least one scene training label. For example, scene training labels may include labels such as: life services, route planning, or data processing.

[0055] Scene matching score can be understood as the matching value between the input text of a pre-defined text classification model and each scene category, reflecting the degree of fit between the input text and the scene training labels of the pre-defined text classification model. For example, the scene matching score can be between 0 and 1, with a score closer to 1 indicating a higher degree of fit.

[0056] The preset scene matching score threshold can be understood as a pre-set critical value used to determine whether the scene training labels are effective, and is used to filter effective scenes.

[0057] Specifically, during the training process of the preset text classification model, it can be trained based on at least one scene training label. The pre-trained preset text classification model can be called through a preset code interface or directly loaded into a local program. User tool call requests can be input into the pre-trained preset text classification model, which can analyze the requests to determine the degree of fit between the requests and the scene training labels. This fit can be quantified as a scene matching score for the tool call request under each scene training label in the preset text classification model. Furthermore, the preset text classification model can compare each scene matching score with a pre-set scene matching score threshold used to determine the validity of scene training labels, and output the label content corresponding to scene labels that exceed the threshold. This label content can be used as the user's tool call scenario. It is understood that the preset text classification model outputs at least one label content, meaning the user's tool call scenario can be multiple scenarios. By comparing the scene matching scores under each scene training label with preset thresholds, effective tool calling scenarios can be selected, ensuring that the determined tool calling scenarios are highly consistent with user needs, which helps to accurately construct a candidate tool set.

[0058] Example 2

[0059] Figure 2 This invention provides a flowchart of another tool screening method for Embodiment 2 of the present invention. The present invention is a refinement of the above embodiments. Specifically, it refines the specific steps of how to determine the tool calling scenario based on the tool calling request and the specific steps of how to determine the candidate calling tool set among each tool to be called.

[0060] like Figure 2 As shown, another tool-based screening method may include the following specific steps:

[0061] S210. Obtain the user's corresponding historical call data, preset tool library, and scene tags of each tool to be called in the preset tool library; wherein, each tool to be called corresponds to at least one scene tag.

[0062] S220. Obtain the tool call request initiated by the user, and determine the user's tool call scenario and the preset call priority strategy of each tool to be called based on the tool call request.

[0063] S230. Extract at least one first candidate tool to be called from each tool to be called, which corresponds to a scenario tag that matches the tool calling scenario.

[0064] The first candidate tool to be invoked can be understood as a set of tools initially selected from among the tools to be invoked based on the scene tags that match the tool invocation scenario. In other words, the scene tags of the first candidate tool to be invoked match the tool invocation scenario, and the first candidate tool to be invoked is part of the candidate invocation tool set.

[0065] Specifically, it iterates through all the scene tags of the tools to be called in the preset tool library, filters out the tool set whose scene tags match the tool call scene, and uses the tools in the tool set as the first candidate tools to be called.

[0066] S240. According to the preset call priority strategy, extract the call frequency of each tool to be called and the call time interval between the last time each tool was called from the historical call data.

[0067] Specifically, historical call data can be statistically analyzed according to a preset call priority strategy to extract the call frequency of each tool to be called and the time interval between the last call to each tool. It should be noted that for the same tool to be called, the call frequency and / or time interval will be different for different preset call priority strategies.

[0068] S250. For each tool to be invoked, obtain the preset scene score for each scene label corresponding to the tool to be invoked.

[0069] The preset scene score can be understood as a quantified value pre-set for each scene label of the tool to be called, reflecting the degree of adaptation of the tool to be called under that scene label. For example, the preset scene score can take the value between 0 and 1. The closer the preset scene score is to 1, the higher the adaptation of the tool to be called under that scene label.

[0070] Specifically, for each tool to be invoked in the preset tool library, the preset scene score corresponding to each scene tag can be extracted from the tool attribute configuration corresponding to each tool to be invoked, or from the preset scene score mapping table of the preset tool library. The preset scene score is used to quantify the adaptability of the tool in different scenes.

[0071] S260. Based on each call frequency, each call time interval, and each preset scenario score, extract at least one second candidate tool to be called from each tool to be called.

[0072] The second candidate tools to be invoked refers to another set of tools selected from the toolset based on the invocation frequency, invocation time interval and preset scenario score of each tool to be invoked. It is another component of the candidate invocation toolset.

[0073] Specifically, each tool to be invoked can be evaluated based on its invocation frequency, invocation time interval, and preset scenario score to obtain evaluation results for each tool to be invoked. Based on the evaluation results, a set of tools that meet the evaluation requirements can be extracted from each tool to be invoked. Tools in this set of tools can be used as second candidate tools to be invoked, and this set of tools must include at least one second candidate tool to be invoked.

[0074] S270. The first candidate calling tool and the second candidate calling tool are used as a candidate calling tool set.

[0075] Specifically, the first candidate calling tool and the second candidate calling tool are merged. If there is overlap between the tools, a tool deduplication operation can be performed to form the final set of candidate calling tools.

[0076] S280. Invoke the preset large language model, and determine the target invoking tool from the candidate invoking tool set based on the preset large language model.

[0077] In this embodiment of the invention, historical call data corresponding to the user, a preset tool library, and scene tags of each tool to be called in the preset tool library are obtained. The tool call request initiated by the user is obtained, and the user's tool call scenario and the preset call priority strategy of each tool to be called are determined based on the tool call request. At least one first candidate tool to be called is extracted from each tool to be called, corresponding to a scene tag that matches the tool call scenario. According to the preset call priority strategy, the call frequency of each tool to be called and the call time interval between the last call and the last call of each tool to be called are extracted from the historical call data. For each tool to be called, the preset scene score of each scene tag corresponding to the tool to be called is obtained. At least one second candidate tool to be called is extracted from each tool to be called based on each call frequency, each call time interval, and each preset scene score. Each first candidate tool and each second candidate tool to be called are used as a candidate tool set. A preset large language model is called, and the target tool to be called is determined from the candidate tool set based on the preset large language model. The embodiments of the present invention can ensure that the candidate calls are directly related to the user's tool call scenarios by accurately matching scene tags; by considering the user's historical behavior, that is, by comprehensively considering the call frequency and call time interval of each tool to be called, the activity of each tool to be called can be dynamically evaluated, which improves the flexibility of tool selection.

[0078] Furthermore, in this embodiment of the invention, step S260 is further refined. Specifically, the method for extracting at least one second candidate tool from each tool to be invoked based on each invocation frequency, each invocation time interval, and each preset scenario score is refined, including:

[0079] S2601. Determine the first call score of each calling tool based on the call frequency.

[0080] The first call score can be understood as a numerical value used to quantify the frequency of tool use, reflecting a user's preference for a particular tool. The higher the first call score, the more frequently the tool is used by the user.

[0081] Specifically, the calling frequency of each tool to be invoked can be obtained, and each tool to be invoked can be evaluated based on its calling frequency to obtain an evaluation value, which can be used as the first calling score of each tool.

[0082] For example, the step of determining the first call score for each calling tool based on each call frequency may include: extracting the highest call frequency from the call frequencies of each calling tool; and using the ratio of the call frequency of each calling tool to the highest call frequency as the first call score for each calling tool.

[0083] S2602. Determine the second call score for each tool to be called based on each call time interval.

[0084] The second call score can be understood as a quantitative value reflecting the activity level of a tool, serving as another dimension for tool evaluation. A higher second call score indicates that the tool is used by users less frequently, and thus, higher tool activity.

[0085] Specifically, the call time interval of each tool to be invoked can be obtained, and each tool to be invoked can be evaluated based on the call time interval to obtain an evaluation value, which can be used as the second call score of each tool.

[0086] For example, the steps of determining the second call score for each tool to be called based on each call time interval may include: obtaining a preset tiered time interval and a lower time limit for each tiered interval within the preset tiered time interval; obtaining the difference between the call time interval and each lower time limit, and determining the tiered interval to which the lower time limit corresponding to the smallest non-negative difference belongs as the target tiered time interval; querying the preset time score associated with the target tiered time interval according to a preset time score rule library, and using the preset time score as the second call score.

[0087] The preset tiered time intervals refer to a series of pre-defined time ranges used to classify the time intervals of the invoked tools. The preset tiered time threshold intervals are distributed in a tiered manner, and different time ranges can obtain different preset time scores, i.e., the second invoke score.

[0088] The target tiered time threshold range refers to a time range within which the time interval falls within the preset tiered time threshold range. The target tiered time threshold range can accurately reflect the time range in which the tool to be invoked is invoked, and is used for the selection of the score in the subsequent second invocation.

[0089] Specifically, the system obtains a preset tiered time interval and a preset time score rule library, extracts the lower limit value of each tiered interval within the preset tiered time interval, compares the call time interval with the lower limit value of each tiered interval, obtains the difference between the call time interval and each lower limit value, and takes the tiered interval to which the lower limit value of the smallest non-negative difference belongs as the target tiered time interval to which the call time interval belongs. The system can query the preset time score associated with the target tiered time interval in the preset time score rule library, and can use the associated preset time score as the second call score.

[0090] S2603. For each tool to be invoked, the preset scene scores of each scene tag corresponding to the tool to be invoked are accumulated to obtain the scene score corresponding to each tool to be invoked.

[0091] Specifically, each tool to be invoked corresponds to at least one scene tag. For a single tool to be invoked, the preset scene scores of each scene tag corresponding to the tool to be invoked are obtained, and the preset scene scores are accumulated to obtain the scene score corresponding to each tool to be invoked.

[0092] S2604. Obtain the sum of the scores from the first call score, the second call score, and the scenario score.

[0093] Specifically, the scores from the first call, the second call, and the scenario are summed to obtain the total score.

[0094] S2605. Compare the accumulated score value with the preset score threshold, and select the tool to be called corresponding to the accumulated score value that is higher than the preset score threshold as the second candidate tool to be called.

[0095] Specifically, a preset score threshold is obtained, the accumulated score value is compared with the preset score threshold, and the tool to be called corresponding to the accumulated score value that is higher than the preset score threshold is selected as the second candidate tool to be called.

[0096] Example 3

[0097] Figure 3 This is a flowchart of another tool screening method provided in Embodiment 3 of the present invention; Figure 4 This is a flowchart of constructing a candidate call tool set provided in Embodiment 3 of the present invention; the embodiments of the present invention are optimizations of the above embodiments, specifically, supplementing a specific application scenario for tool screening.

[0098] When the Model Context Protocol (MCP) integrates too many tools, exposing all tool descriptions to a large model simultaneously can lead to token explosion, resulting in issues such as excessively long prompts, context redundancy, and decreased inference efficiency. Existing solutions often employ static tool exposure methods, lacking dynamic scheduling capabilities based on user task scenarios and tool call history, leading to low model efficiency and wasted resources.

[0099] The present invention aims to solve the problem of token explosion caused by too many MCP tools. It proposes an MCP tool intelligent filtering mechanism that combines call history and scenario classification, thereby significantly reducing the context burden while ensuring the flexibility of tool calls.

[0100] Another tool screening method proposed in this embodiment of the invention, such as Figure 3 As shown, the process includes the following steps: obtaining task request information, i.e., receiving tool call requests initiated by users; classifying tool call requests using a preset text classification model to obtain the scene matching score of the tool call request under the scene label, and identifying the scene matching score corresponding to the task through the model; determining a candidate tool set by combining the scene matching score with the user's corresponding historical call data, and having a preset large language model generate a target MCP tool based on the candidate tool set, i.e., the call parameters of the target tool; calling the corresponding target MCP tool based on the described call parameters, and triggering the target MCP tool to execute the task using the generated call parameters; updating the list of recently used tools, recording the tool called this time in the frequently used list for easy subsequent quick calls.

[0101] like Figure 4As shown, the process for constructing a candidate calling toolset provided by this embodiment of the invention includes: inputting a tool calling request into a preset text classification model to obtain the scene matching score of the tool calling request under each scene training label of the text classification model; extracting at least one first candidate calling tool corresponding to a scene label that matches the tool calling scene from each calling tool; extracting the calling frequency of each calling tool and the calling time interval between the last calling of each calling tool and the last calling of each calling tool from historical calling data according to a preset calling priority strategy; determining the first calling score of each calling tool based on each calling frequency; determining the second calling score of each calling tool based on each calling time interval; for each calling tool, accumulating the preset scene scores of each scene label corresponding to the calling tool to obtain the scene score corresponding to each calling tool; obtaining the score accumulation value of the first calling score, the second calling score, and the scene score; comparing the score accumulation value with a preset score threshold, and using the calling tool corresponding to the score accumulation value higher than the preset score threshold as the second candidate calling tool; and using each first candidate calling tool and each second candidate calling tool as a candidate calling toolset.

[0102] Among them, the cumulative score value ,in, For tool indexing, As a preset constant, For tools The call history score, i.e. the first call score, is calculated based on the preset call priority strategy as follows: ,in, For the tool to be called call frequency, For tools The scene score is calculated by summing the preset scene scores of each scene tag corresponding to the tool to be invoked for each tool to be invoked. The closer the scene is to the scene, the closer the scene score is to 1.

[0103] The score for the second call is determined as follows: , All of these are adjustable parameters. This represents the time since the last call, which can be in minutes or hours.

[0104] in, They are respectively The corresponding weights, where , All are adjustable parameters. The default value can be... .

[0105] The calculation steps for the candidate tool N, which matches the historical data, are as follows: ,in, The default value can be set to the preset score threshold. , This can be adjusted based on the token budget.

[0106] In this embodiment of the invention, the description information of the candidate tool set can also be dynamically injected into the Prompt, instead of exposing all tool descriptions. After a successful invocation, the usage frequency record of the tool is updated so that it can be given priority in the next candidate tool construction. As the user task iterates, the scenario matching rules and usage frequency statistics are dynamically updated to achieve continuous optimization.

[0107] By introducing a call scoring mechanism, the usability and hit rate of tool calls can be improved. By introducing a scenario matching mechanism, the context relevance of tool exposure can be enhanced. This invention enables dynamic optimization configuration of MCP in multi-tool scenarios, supporting larger-scale tool integration.

[0108] Example 4

[0109] Figure 5 A schematic diagram of another tool screening device provided in Embodiment 4 of the present invention. (See attached diagram.) Figure 5 As shown, the device includes: a tag acquisition module 310, a scene determination module 320, a toolset determination module 330, and a target tool determination module 340.

[0110] The tag acquisition module 310 is used to acquire the user's corresponding historical call data, preset tool library, and scene tags of each tool to be called in the preset tool library; wherein, each tool to be called corresponds to at least one scene tag;

[0111] The scenario determination module 320 is used to obtain the tool call request initiated by the user, and determine the user's tool call scenario and the preset call priority strategy of each tool to be called based on the tool call request;

[0112] The toolset determination module 330 is used to determine the candidate toolset for invocation from among the tools to be invoked based on historical invocation data, the scenario tags corresponding to each tool to be invoked, the tool invocation scenario, and the preset invocation priority strategy.

[0113] The target tool determination module 340 is used to call a preset large language model and determine the target calling tool from the candidate calling tool set based on the preset large language model.

[0114] Optionally, the scene determination module 320 is specifically used to call a preset text classification model; wherein, the training labels of the preset text classification model include at least one scene training label; the tool call request is input into the preset text classification model to obtain the scene matching score of the tool call request under each scene training label of the preset text classification model; the scene matching score is compared with a preset scene matching score threshold, and the label content corresponding to the scene label that is higher than the preset threshold is taken as the tool call scene.

[0115] Optionally, the toolset determination module 330 includes: a first tool determination unit, used to extract at least one first candidate tool to be called from each tool to be called, corresponding to a scenario tag that matches the tool's calling scenario; a time interval determination unit, used to extract the calling frequency of each tool to be called and the time interval between the last time each tool to be called was called from historical calling data according to a preset calling priority strategy; a first scenario score determination unit, used to obtain a preset scenario score for each scenario tag corresponding to each tool to be called; a second tool determination unit, used to extract at least one second candidate tool to be called from each tool to be called based on each calling frequency, each calling time interval, and each preset scenario score; and a toolset determination unit, used to use each first candidate tool and each second candidate tool as a candidate calling toolset.

[0116] Optionally, the second tool determination unit includes: a first score determination subunit, used to determine a first call score for each tool to be called based on each call frequency; a second score determination subunit, used to determine a second call score for each tool to be called based on each call time interval; a second scene score determination subunit, used to accumulate the preset scene scores of each scene tag corresponding to each tool to be called to obtain the scene score corresponding to each tool to be called; a score accumulation subunit, used to obtain the score accumulation value of the first call score, the second call score, and the scene score; and a second tool determination subunit, used to compare the score accumulation value with a preset score threshold, and use the tool to be called corresponding to the score accumulation value that is higher than the preset score threshold as the second candidate tool to be called.

[0117] Optionally, the first score determination sub-unit is specifically used to extract the highest call frequency among the call frequencies of each tool to be called; the ratio of the call frequency of each tool to be called to the highest call frequency is used as the first call score of each tool to be called.

[0118] Optionally, the second score determination subunit is specifically used to obtain a preset tiered time interval and the lower limit value of the time for each tiered interval within the preset tiered time interval; obtain the difference between the call time interval and each lower limit value, determine the tiered interval to which the lower limit value corresponding to the smallest non-negative difference belongs as the target tiered time interval; query the preset time score associated with the target tiered time interval according to the preset time score rule library, and use the preset time score as the second call score.

[0119] The tool screening device provided in the embodiments of the present invention can execute the tool screening method provided in any embodiment of the present invention, and has the corresponding beneficial effects of executing the method.

[0120] Example 5

[0121] Embodiment 5 of the present invention provides an electronic device for executing a tool screening method, a computer-readable storage medium, and a computer program product.

[0122] Figure 6 A schematic diagram of an electronic device that can be used to implement the tool screening method of any embodiment of the present invention is provided. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown in the embodiments of the present invention, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the embodiments of the present invention described and / or claimed herein.

[0123] like Figure 6 The electronic device includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 may also store various programs and data required for device operation. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0124] Multiple components in the electronic device are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, optical disk, etc.; and a communication unit 19, such as a network card, modem, wireless transceiver, etc. The communication unit 19 allows the electronic device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0125] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing units, graphics processing units, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as tool selection methods.

[0126] In some embodiments, the tool screening method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the tool screening method may be performed. Alternatively, in other embodiments, processor 11 may be configured as the tool screening method by any other suitable means (e.g., by means of firmware).

[0127] Various embodiments of the systems and technologies described above in these embodiments of the present invention can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard products (ASICs), systems-on-a-chip (SoCs), payload programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input system, and at least one output system, and transmitting data and instructions to the storage system, the at least one input system, and the at least one output system.

[0128] Computer programs for implementing the methods of embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing system, such that when executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0129] In the context of embodiments of the present invention, a computer-readable storage medium may be a tangible storage medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, system, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal storage medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, erasable programmable read-only memory (EPROM or flash memory), optical fibers, compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0130] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display system (e.g., a cathode ray tube or liquid crystal display monitor) for displaying information to the user; and a keyboard and pointing system (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of systems can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including: sound input, voice input, or tactile input).

[0131] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., communication networks) of any form or storage medium. Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0132] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product within the cloud computing service system. This addresses the shortcomings of traditional physical hosts and virtual private servers, such as high management difficulty and weak business scalability.

[0133] It should be understood that the various forms of procedures shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and no limitation is imposed herein.

[0134] The specific embodiments described above do not constitute a limitation on the scope of protection of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A tool screening method, characterized in that, The method includes: Obtain the user's historical call data, a preset tool library, and scene tags for each tool to be called within the preset tool library; wherein each tool to be called corresponds to at least one scene tag; Obtain the tool call request initiated by the user, and determine the user's tool call scenario and the preset call priority strategy for each of the tools to be called based on the tool call request; Based on the historical call data, the scenario tags corresponding to each of the tools to be called, the tool call scenarios, and the preset call priority strategy, a set of candidate call tools is determined among the tools to be called. A preset large language model is invoked, and a target invocation tool is determined from the candidate invocation tool set based on the preset large language model.

2. The method according to claim 1, characterized in that, Determining the user's tool invocation scenario based on the tool invocation request includes: Invoke a preset text classification model; wherein the training labels of the preset text classification model include at least one scene training label; The tool call request is input into the preset text classification model to obtain the scene matching score of the tool call request under each scene training label of the preset text classification model; The matching scores of each scenario are compared with a preset scenario matching score threshold, and the tag content corresponding to the scenario tag that is higher than the preset threshold is used as the tool invocation scenario.

3. The method according to claim 1, characterized in that, The step of determining a candidate tool set from among the tools to be invoked based on the historical invocation data, the scenario tags corresponding to each of the tools to be invoked, the tool invocation scenario, and the preset invocation priority strategy includes: Extract at least one first candidate tool for invocation from each of the tools to be invoked, which corresponds to a scenario tag that matches the tool invocation scenario. According to the preset call priority strategy, the call frequency of each tool to be called and the call time interval between the last time each tool to be called was extracted from the historical call data; For each of the tools to be invoked, obtain the preset scene score for each scene tag corresponding to the tool to be invoked; Based on the calling frequency, the calling time interval, and the preset scenario score, at least one second candidate tool to be called is extracted from each of the tools to be called. Each of the first candidate invocation tools and each of the second candidate invocation tools are used as the candidate invocation tool set.

4. The method according to claim 3, characterized in that, The step of extracting at least one second candidate tool for invocation from each of the tools to be invoked based on the invocation frequency, the invocation time interval, and the preset scenario score includes: The first call score of each of the aforementioned call frequencies is determined; The second invocation score of each of the aforementioned invocation time intervals is determined; For each tool to be invoked, the preset scene scores of each scene tag corresponding to the tool to be invoked are accumulated to obtain the scene score corresponding to each tool to be invoked; Obtain the sum of the scores of the first call score, the second call score, and the scenario score; The accumulated score is compared with a preset score threshold, and the tool to be invoked corresponding to the accumulated score that is higher than the preset score threshold is selected as the second candidate tool to be invoked.

5. The method according to claim 4, characterized in that, The determination of the first call score for each of the aforementioned tools based on their respective call frequencies includes: Extract the highest call frequency from the call frequencies of each of the aforementioned tools to be invoked; The ratio of the calling frequency of each of the aforementioned tools to the highest calling frequency is used as the first calling score of each of the aforementioned tools.

6. The method according to claim 4, characterized in that, The determination of the second invocation score for each of the tools to be invoked based on each of the invocation time intervals includes: Obtain a preset stepped time interval, and the lower limit value of the time for each stepped interval within the preset stepped time interval; Obtain the difference between the call time interval and each of the time lower limits, and determine the step interval to which the time lower limit corresponding to the smallest non-negative difference belongs as the target step time interval; According to the preset time score rule library, query the preset time score associated with the target time interval, and use the preset time score as the second call score.

7. A tool screening device, characterized in that, The device includes: The tag acquisition module is used to acquire the user's corresponding historical call data, preset tool library, and scene tags of each tool to be called in the preset tool library; wherein, each tool to be called corresponds to at least one scene tag; The scenario determination module is used to obtain the tool call request initiated by the user, and determine the user's tool call scenario and the preset call priority strategy of each tool to be called based on the tool call request; The toolset determination module is used to determine a candidate toolset for invocation from among the tools to be invoked based on the historical invocation data, the scenario tags corresponding to each of the tools to be invoked, the tool invocation scenario, and the preset invocation priority strategy. The target tool determination module is used to call a preset large language model and determine the target calling tool based on the preset large language model in the candidate calling tool set.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the tool screening method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the tool screening method according to any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the tool screening method according to any one of claims 1-6.

Images