Tool recommendation method and device and electronic equipment

Abstract

The invention discloses a tool recommendation method and device and electronic equipment. The method comprises the following steps: receiving tool use demand information input by a target object; converting the tool use demand information into a demand semantic vector; a first candidate tool set corresponding to the demand semantic vector is determined from a first hash space, a second candidate tool set corresponding to the demand semantic vector is determined from a second hash space, the first hash space is a hash space corresponding to tool information of all tool sets, and the second hash space is a hash space corresponding to tool information of all tool sets; the second hash space is a hash space corresponding to the tool calling requirement; determining a target candidate tool set according to the first candidate tool set and the second candidate tool set; and recommending the target candidate tool set to the target object. The technical problem that a huge tool set affects accurate prediction of a large language model on tool calling requirements in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to a tool recommendation method, device, and electronic device. Background Art

[0002] With the rise of Large Language Model (LLM), many companies have developed various artificial intelligence projects based on LLM of Transformer architecture and actively explored the industrial application of LLM. LLM's powerful text generation capability has triggered a boom in artificial intelligence applications. However, with the deepening of practice, LLM has revealed its limitations in real-time perception in complex environments.

[0003] To make up for this deficiency, intelligent agent applications built around LLM came into being. LLM development organizations have effectively met the needs of actual use by introducing functions such as tool calling into the LLM training process and combining them with custom tools developed by users. With the help of tool calling, intelligent agents not only expand their own capabilities, but also solve complex problems, improve decision-making quality, enhance adaptability, and promote multi-agent collaboration, thereby achieving a higher level of autonomy and intelligence.

[0004] Currently, intelligent agent applications directly input the tool set of a single application as a prompt, or provide meta information such as tool name, description, parameters, etc. in the context of the conversation history in JSON format. However, as companies actively develop intelligent agent projects, the number of tool sets has increased dramatically, resulting in a sharp increase in the use of tokens (the basic processing unit of text, hereinafter referred to as tokens).

[0005] In ultra-large context scenarios, the huge toolset significantly affects LLM's accurate prediction of tool call requirements.

[0006] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0007] The embodiments of the present application provide a tool recommendation method, device and electronic device to at least solve the technical problem in the related art that a large tool set affects the accurate prediction of a large language model for tool calling requirements.

[0008] According to one aspect of an embodiment of the present application, a tool recommendation method is provided, including: receiving tool usage requirement information input by a target object; converting the tool usage requirement information into a requirement semantic vector; determining a first candidate toolset corresponding to the requirement semantic vector from a first hash space, and determining a second candidate toolset corresponding to the requirement semantic vector from a second hash space, wherein the first hash space is a hash space corresponding to tool information of all toolsets, and the second hash space is a hash space corresponding to tool call requirements; determining a target candidate toolset based on the first candidate toolset and the second candidate toolset; and recommending the target candidate toolset to the target object.

[0009] Optionally, before receiving the tool usage requirement information input by the target object, the method also includes: obtaining an original data set, wherein the original data set includes query data and positive sample data, and the positive sample data includes a JSON representation of the toolset and tool call status; using an embedding model to vector encode the original data set to obtain a vector representation; performing a nearest neighbor search on the query vector in the vector representation to obtain a search result; determining a vector in the search result whose distance to the query vector is less than a low distance threshold as a positive sample group of the query data; determining a vector in the search result whose distance to the query vector is greater than a high distance threshold as a negative sample group of the query data, wherein the high distance threshold is greater than the low distance threshold; and adjusting the parameters of the embedding model through the positive sample group and the negative sample group to obtain a target embedding model.

[0010] Optionally, the first hash space is determined by: obtaining JSON representations of all toolsets; vectorizing the JSON representations of all toolsets using a target embedding model to obtain a first semantic vector; hashing the first semantic vector using a preset hash function to generate a first hash value; and determining the first hash space based on the first hash value.

[0011] Optionally, the second hash space is determined in the following manner: obtaining historical tool call information and obtaining historical usage related to the historical tool call information; using a target embedding model to vectorize the historical tool call information and the historical usage, respectively, to obtain a second semantic vector and a third semantic vector; using a preset hash function to hash the second semantic vector and the third semantic vector, respectively, to generate a second hash value and a third hash value; and determining the second hash space based on the second hash value and the third hash value.

[0012] Optionally, determining a first candidate toolset corresponding to the requirement semantic vector from the first hash space includes: inputting the requirement semantic vector into a preset hash function to generate a requirement hash value of the requirement semantic vector; determining a first target hash space from the first hash space based on the requirement hash value; determining a first target semantic vector from the first target hash space whose similarity to the requirement semantic vector is greater than a first threshold; and determining the first candidate toolset based on the tool JSON representation corresponding to the first target semantic vector.

[0013] Optionally, determining a second candidate toolset corresponding to the requirement semantic vector from the second hash space includes: obtaining a requirement hash value of the requirement semantic vector; determining a second target hash space from the second hash space based on the requirement hash value; determining a target call record from the second target hash space whose similarity with the requirement semantic vector is greater than a second threshold; and determining the second candidate toolset based on the target call record.

[0014] Optionally, a target candidate toolset is determined based on the first candidate toolset and the second candidate toolset, including: obtaining the intersection of the first candidate toolset and the second candidate toolset to obtain a third candidate toolset; performing similarity calculations on the candidate tool vector corresponding to the JSON representation of each tool in the third candidate toolset and the requirement semantic vector to obtain a similarity result; and determining the tools corresponding to the candidate tool vectors having a similarity result greater than a third threshold as the target candidate toolset.

[0015] According to another aspect of an embodiment of the present application, a tool recommendation device is also provided, including: a receiving module for receiving tool usage requirement information input by a target object; a conversion module for converting the tool usage requirement information into a requirement semantic vector; a first determination module for determining a first candidate toolset corresponding to the requirement semantic vector from a first hash space, and determining a second candidate toolset corresponding to the requirement semantic vector from a second hash space, wherein the first hash space is a hash space corresponding to tool information of all toolsets, and the second hash space is a hash space corresponding to tool call requirements; a second determination module for determining a target candidate toolset based on the first candidate toolset and the second candidate toolset; and a recommendation module for recommending the target candidate toolset to the target object.

[0016] According to another aspect of the embodiments of the present application, an electronic device is also provided, including: a memory for storing program instructions; a processor, connected to the memory, for executing program instructions to implement the following functions: receiving tool usage requirement information input by a target object; converting the tool usage requirement information into a requirement semantic vector; determining a first candidate toolset corresponding to the requirement semantic vector from a first hash space, and determining a second candidate toolset corresponding to the requirement semantic vector from a second hash space, wherein the first hash space is a hash space corresponding to the tool information of all toolsets, and the second hash space is a hash space corresponding to the tool call requirement; determining a target candidate toolset based on the first candidate toolset and the second candidate toolset; and recommending the target candidate toolset to the target object.

[0017] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned tool recommendation method by running the computer program.

[0018] According to another aspect of the embodiments of the present application, a computer program product is provided, including computer instructions, which implement the above-mentioned tool recommendation method when executed by a processor.

[0019] In an embodiment of the present application, tool usage requirement information input by a target object is received; the tool usage requirement information is converted into a requirement semantic vector; a first candidate toolset corresponding to the requirement semantic vector is determined from a first hash space, and a second candidate toolset corresponding to the requirement semantic vector is determined from a second hash space, wherein the first hash space is a hash space corresponding to tool information of all toolsets, and the second hash space is a hash space corresponding to tool call requirements; a target candidate toolset is determined based on the first candidate toolset and the second candidate toolset; the target candidate toolset is recommended to the target object, thereby achieving the purpose of efficiently and personally recommending tools to users, thereby achieving the technical effect of reducing the computational burden of a large language model in the tool call process and improving the accuracy and response speed of tool recommendations, thereby solving the technical problem in the related art that a large toolset affects the accurate prediction of a large language model for tool call requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0021] Figure 1 is a hardware structure block diagram of a computer terminal for implementing a tool recommendation method according to an embodiment of the present application;

[0022] Figure 2 is a flow chart of a tool recommendation method according to an embodiment of the present application;

[0023] Figure 3 It is a structural diagram of a tool recommendation device according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] First, some nouns or terms that appear in the process of explaining the embodiments of the present application are subject to the following explanations:

[0027] Collaborative search: Collaborative search is a search technology based on collaborative filtering. It provides personalized and highly relevant search results by analyzing and utilizing the historical search behaviors and interaction data of multiple users or agents, aiming to improve the search experience and the accuracy of results.

[0028] Hash space partitioning: Hash space partitioning is a method of evenly distributing data to multiple storage units through hash functions, aiming to achieve fast data location and load balancing.

[0029] Agent Tools: In agent tool calls, tools are defined as functions or resources that agents can use to interact with external data sources or environments. These tools are usually functions or APIs (collectively referred to as functions), and agents call these functions to perform specific tasks or obtain information.

[0030] In related technologies, in ultra-large context scenarios, LLM faces the following challenges in tool calls: 1. A large amount of toolset information occupies the conversation history context, resulting in a surge in token usage and compressing the amount of valid tokens available to users; 2. The huge toolset significantly affects LLM's accurate prediction of tool call requirements; 3. The above factors work together to greatly affect system performance and availability.

[0031] In order to solve the problems existing in the related art, the present application embodiment provides a tool recommendation method, which can be run on Figure 1 In the computer terminal shown, the computer terminal is explained below.

[0032] The tool recommendation method embodiment provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a tool recommendation method. Figure 1 As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and / or wireless network. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. Those skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0033] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0034] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the tool recommendation method in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, the above-mentioned tool recommendation method is realized. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0035] The transmission module 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0036] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0037] It should be noted that, in some optional embodiments, the above Figure 1 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0038] In the above-mentioned operating environment, an embodiment of the present application provides an embodiment of a tool recommendation method. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] Figure 2 is a flowchart of a tool recommendation method according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0040] Step S202: receiving tool usage requirement information input by the target object.

[0041] Step S204: convert the tool usage requirement information into a requirement semantic vector.

[0042] In the above steps S202 to S204, the business system receives the tool usage demand information initiated by the user (target object). The tool usage demand information can be expressed in the form of natural language, such as "I need to know the recent weather conditions" or "How to calculate the exchange rate." Once the tool usage demand information is received, the system will use a pre-trained or fine-tuned Embedding model (i.e., an embedding model) to convert the tool usage demand information (such as the demand text) into a dense semantic vector representation, i.e., a demand semantic vector. The Embedding model can encode the semantic information of the text into a vector, so that similar demands are close in distance in the vector space. Through this vectorization process, the system converts the user's needs into a form that can be mathematically analyzed and searched, in preparation for subsequent steps.

[0043] In an optional embodiment, the tool usage demand information can also be based on the Prompt Engine and ZeroShot COT Prompt technology to mine tool call demand in the user's conversation history context, and use the reasoning ability of a large language model to mine user needs, thereby obtaining the above-mentioned tool usage demand information.

[0044] Step S206, determining a first candidate toolset corresponding to the requirement semantic vector from the first hash space, and determining a second candidate toolset corresponding to the requirement semantic vector from the second hash space, wherein the first hash space is a hash space corresponding to the tool information of all toolsets, and the second hash space is a hash space corresponding to the tool call requirement.

[0045] In the above step S206, the system uses the demand semantic vector to search in two different hash spaces. The first hash space contains the hash representation of the tool information of all tool sets. The first hash space is obtained by vectorizing the tool description and dividing the space using hash functions such as LSH. The system calculates the similarity between the demand semantic vector and each tool vector in the first hash space, and then selects the tools most relevant to the user's needs from the first hash space according to the similarity to form the first candidate tool set.

[0046] The second hash space is designed specifically for tool call requirements, and it contains hash representations of all users' historical tool call requirements and feedback. Similarly, the system will use the demand semantic vector to search in the second hash space, dig out the historical call records of other users with similar requirements, and extract the tools that best meet the requirements to form the second candidate toolset.

[0047] Step S208: determining a target candidate toolset based on the first candidate toolset and the second candidate toolset.

[0048] In the above step S208, after collecting the first and second candidate tool sets, the system will combine the two candidate sets according to certain algorithms or rules to determine the final target candidate tool set. For example, operations such as weighted average, intersection or union will be used to consider the direct relevance of the tools (first candidate set) and the matching degree of historical usage context (second candidate set). The goal is to find tools that meet the current demand semantics and have good historical performance, so as to improve the accuracy and personalization of recommendations.

[0049] Step S210: recommending the target candidate toolset to the target object.

[0050] In the above step S210, the system presents the target candidate tool set to the user (target object) in the form of a recommendation list. For example, it can be in the form of an agent dialogue interface, in-application recommendation or API response, depending on the implementation method of the business system. The user can select the most suitable tool according to the recommendation list to call it to meet his needs. At the same time, the user's selection and feedback information will be further archived in the second hash space for future collaborative search and recommendation, forming a closed loop of dynamic optimization.

[0051] Through the above steps S202 to S210, the embodiment of the present application can effectively solve the challenges faced by large language models when processing large-scale tool sets, such as a surge in token usage, decreased tool call prediction accuracy, and reduced system performance, thereby providing a more efficient, accurate, and personalized tool screening service. The following is an explanation.

[0052] In step S202 of the tool recommendation method, before receiving the tool usage requirement information input by the target object, the method also includes: obtaining an original data set, wherein the original data set includes query data and positive sample data, and the positive sample data includes a JSON representation of the toolset and tool call status; using an embedding model to vector encode the original data set to obtain a vector representation; performing a nearest neighbor search on the query vector in the vector representation to obtain a search result; determining a vector in the search result whose distance to the query vector is less than a low distance threshold as a positive sample group of the query data; determining a vector in the search result whose distance to the query vector is greater than a high distance threshold as a negative sample group of the query data, wherein the high distance threshold is greater than the low distance threshold; and adjusting the parameters of the embedding model through the positive sample group and the negative sample group to obtain a target embedding model.

[0053] In the embodiment of the present application, the system needs to obtain the original data set, which can be obtained by fusing the open source data set and the internal business data. The original data set contains query data (usually the needs or questions raised by the user) and positive sample data. The positive sample data mainly includes the JSON representation of the internal tool set of the enterprise and the actual call status of these tools, where the JSON representation contains the tool name, description, parameters and other detailed information, and the tool call status records the tool's performance in actual use, call frequency, call results and other metadata.

[0054] Use a pre-selected embedding model, such as the bge-large-zh-v1.5 model, to vector encode the query data and positive sample data in the original dataset. The embedding model can convert text information into a dense vector representation, so that semantically similar texts are close in distance in the vector space. This process converts the original natural language description into a mathematical vector, which is convenient for subsequent calculation and analysis.

[0055] The system performs an Approximate Nearest Neighbor (ANN) search on the query vector in the vector representation, for example, by calculating the similarity or distance between the query vector and all positive sample vectors. The nearest neighbor search aims to find the positive sample vector that is most similar to the query vector, and its core is to find the "nearest neighbor" in text semantics. The search results usually contain the distance or similarity information of all positive sample vectors and the query vector. Based on the search results, the system determines the vectors whose distance to the query vector is less than a low distance threshold as the positive sample group of the query data. The low distance threshold is used to filter out samples that are very similar to the query vector, that is, the system believes that these samples are highly relevant to the query requirements and can be used as positive examples to train the model. At the same time, the vectors in the search results whose distance to the query vector is greater than the high distance threshold are determined as the negative sample group of the query data. The high distance threshold is a larger threshold used to filter out samples that are not relevant to the query vector as negative examples to help the model learn to distinguish between relevant and irrelevant tools. By distinguishing between positive sample groups and negative sample groups, the system can use them to adjust the parameters of the Embedding model to optimize the model's understanding and prediction capabilities of tool call requirements. The adjustment process involves retraining or fine-tuning the model to ensure that the model can better extract the semantic features of query requirements and the semantic features of tool capabilities and align them, thereby improving the accuracy and generalization ability of the model. The adjusted or optimized embedding model is called the target embedding model.

[0056] In an optional embodiment, for example, when fine-tuning the parameters of the bge-large-zh-v1.5 model, an Nvidia V100 graphics card can be used, and the learning rate is set to 1e-5, the batch size is 32, the training epoch is set to 30, and vector normalization is enabled. Other default parameters are used for processing, so that the model can simultaneously extract the query demand semantics and the capability semantics represented by the tool JSON, thereby achieving the purpose of aligning the semantics of different spaces. In order to ensure that the model does not forget, the model can be merged through the LM_Cocktail tool, and the weight of the native model is set to 0.6 and the weight of the fine-tuning model is set to 0.4. This can greatly promote the usability of the fine-tuning model, and minimize the loss of native capabilities while increasing the recognition mode.

[0057] In the above tool recommendation method, the first hash space is determined in the following way: obtaining the JSON representation of all tool sets; using the target embedding model to vectorize the JSON representation of all tool sets to obtain a first semantic vector; using a preset hash function to hash the first semantic vector to generate a first hash value; and determining the first hash space based on the first hash value.

[0058] In the embodiment of the present application, the method for determining the first hash space includes the following steps:

[0059] Step 1: Get the JSON representation of all toolsets.

[0060] First, the system needs to collect JSON representations of all available tool sets within the enterprise. JSON representation is a structured data format that usually contains detailed information such as the tool name, description, parameters, and return format. It is the basis for understanding and matching tool call requirements. These JSON representations come from intelligent tools developed within the enterprise, as well as possible third-party tool APIs.

[0061] In an optional embodiment, in order to promote the collection of internal enterprise toolsets and the construction of hash spaces, an archiving interface can be provided externally. Users can register tools into the enterprise toolset by calling the tool archiving interface in the platform and passing APIs, JSON representations of request body requirements, or optional Python tool call build scripts.

[0062] Step 2: Use the target embedding model to vectorize the JSON representation of all toolsets to obtain the first semantic vector.

[0063] After collecting the JSON representations of all tools, the system will use the target embedding model obtained through positive and negative sample mining and fine-tuning to vectorize these JSON representations. After specific training or fine-tuning, the target embedding model can more effectively extract the semantic features of tool descriptions and convert text information into dense vector representations in high-dimensional vector space, namely the first semantic vector. This process converts the description of the tool set from natural language to mathematical vectors, which facilitates subsequent similarity calculations and searches.

[0064] Step 3: Use a preset hash function to hash the first semantic vector to generate a first hash value.

[0065] The system will hash the semantic vector of each tool using a preset hash function, mapping the high-dimensional vector to a low-dimensional hash value. Locality Sensitive Hashing (LSH) is a commonly used hash function that ensures that semantically similar vectors are mapped to similar hash values, thereby speeding up the subsequent search process for similar tools. Through hashing, the first semantic vector of each tool is converted to the first hash value.

[0066] Step 4: Determine a first hash space according to the first hash value.

[0067] According to the generated first hash value, the system maps the hash values ​​of all tools to the corresponding hash space, namely the first hash space. The hash space distributes vectors into multiple "buckets" or spaces through hash functions, so that similar vectors are concentrated together. Therefore, when searching, only tool vectors in "buckets" with the same or similar hash values ​​as the target can be checked, which greatly speeds up the search and reduces the consumption of computing resources.

[0068] The establishment of the first hash space provides a fast positioning and screening mechanism for subsequent searches in the tool set based on the demand semantic vector. When a user proposes a tool usage requirement, the system can use the demand semantic vector generated by the user's demand information to obtain the corresponding hash value through hashing, and then quickly search for similar tool vectors in the first hash space to form the first candidate tool set. This process not only improves the search efficiency, but also ensures the accuracy of the recommendation results.

[0069] In the above tool recommendation method, the second hash space is determined in the following manner: obtaining historical tool call information and obtaining historical usage related to the historical tool call information; using the target embedding model to vectorize the historical tool call information and the historical usage to obtain a second semantic vector and a third semantic vector; using a preset hash function to hash the second semantic vector and the third semantic vector to generate a second hash value and a third hash value; and determining the second hash space based on the second hash value and the third hash value.

[0070] In the embodiment of the present application, the system needs to collect the user's past call information for various tools, which is usually recorded in the business system, including user queries, requested tool names, parameters, and call contexts. At the same time, the system also collects user feedback on the use of these tools after calling them, which includes historical data such as the results of tool calls, user satisfaction evaluations, and call frequencies. These historical data provide an important basis for subsequent collaborative searches, enabling the system to make personalized recommendations based on the user's actual needs and usage habits.

[0071] After collecting historical tool call information and historical usage, the system uses the previously obtained target embedding model to vectorize this information. The target embedding model can convert text and descriptions into dense semantic vectors. For historical tool call information, the generated vector is called the second semantic vector; for historical usage, the generated vector is called the third semantic vector. The second semantic vector captures the semantic features of user queries and needs, while the third semantic vector reflects the semantic features of user feedback and evaluation after using the tool. The generation of these two types of vectors provides a basis for subsequent collaborative analysis and search.

[0072] The system uses a preset hash function to hash the generated second semantic vector and third semantic vector to obtain a second hash value and a third hash value, respectively. The hashing process enables similar semantic vectors to be mapped to similar hash values, which helps to quickly locate and filter out similar needs and usage in large-scale data sets. The system determines the second hash space based on the generated second hash value and third hash value. The second hash space contains hash representations of user needs and usage. By mapping similar hash values ​​to the same "bucket" or storage location, the system can quickly retrieve historical call records and usage of other users with similar needs from historical data when the user proposes a new tool call requirement, thereby forming a collaborative-based tool recommendation. Through this process, the system can combine the current user's demand description with the needs and feedback of historical users to provide more personalized and accurate tool recommendations, enhancing the system's intelligence and user experience.

[0073] In actual applications, the construction of the second hash space is a dynamic process. As users continue to call tools and accumulate feedback, the system will continuously update the second hash space to ensure that it contains the latest demand and usage information, thereby providing more comprehensive and real-time data support for collaborative search.

[0074] In step S206 of the above-mentioned tool recommendation method, a first candidate toolset corresponding to the requirement semantic vector is determined from the first hash space, including: inputting the requirement semantic vector into a preset hash function to generate a requirement hash value of the requirement semantic vector; determining a first target hash space from the first hash space based on the requirement hash value; determining a first target semantic vector from the first target hash space whose similarity with the requirement semantic vector is greater than a first threshold; and determining the first candidate toolset based on the tool JSON representation corresponding to the first target semantic vector.

[0075] In an embodiment of the present application, after the system receives the tool usage requirement information input by the user and converts it into a requirement semantic vector, the requirement semantic vector is hashed through a preset hash function to generate a requirement hash value. The preset hash function (such as LSH) can map the requirement semantic vector to a low-dimensional hash space, and this hash value is used as a representative of the requirement in the hash space for subsequent rapid retrieval. After the requirement hash value is generated, the system will use this hash value to search for the corresponding target hash space in the first hash space, that is, the first target hash space. The first hash space was previously established by processing the JSON representation of all tool sets, where the semantic vector of each tool is hashed and distributed into different "buckets". By finding the "bucket" that matches the requirement hash value, the system can quickly locate a tool set that is semantically similar to the user's requirements.

[0076] After determining the first target hash space, the system will further filter out the first target semantic vector whose similarity with the demand semantic vector exceeds the preset first threshold. The first threshold is a parameter used to filter vectors with low similarity, ensuring that only tool vectors that are closely related to the demand are selected, which helps to improve the accuracy and efficiency of the recommendation. The system will map back to the original tool JSON representation based on the filtered first target semantic vector. This step involves dehashing, converting the hash value into the corresponding semantic vector, and then finding the tool information corresponding to these vectors. Through this process, the system is able to determine the first candidate toolset, which contains tool information that is highly matched with the user's current needs. These tools are sorted according to the similarity between their descriptions and user needs, so that they can be provided to users for further selection and use.

[0077] Through the above steps, the system can efficiently screen out the first candidate toolset that is most relevant to user needs from a large-scale toolset. This process utilizes hash space partitioning and similarity calculation, effectively reducing the consumption of computing resources, improving the accuracy and response speed of recommendations, and solving the problem of a surge in token usage and a decrease in tool call prediction accuracy when large language models process large-scale toolsets.

[0078] In step S206 of the tool recommendation method, a second candidate toolset corresponding to the requirement semantic vector is determined from the second hash space, including: obtaining the requirement hash value of the requirement semantic vector; determining a second target hash space from the second hash space based on the requirement hash value; determining a target call record from the second target hash space whose similarity with the requirement semantic vector is greater than a second threshold; and determining the second candidate toolset based on the target call record.

[0079] In the embodiment of the present application, after the first candidate toolset is screened out from the first hash space, the system will continue to use the previously generated demand semantic vector and calculate its hash value again through a preset hash function. This demand hash value will be used to search in the second hash space to find historical call records closely related to user needs. The second hash space is constructed by processing historical tool call information and historical usage, which contains hash representations of user feedback and evaluation information on tool calls. The system will use the demand hash value to locate the second target hash space in the second hash space, that is, the "bucket" where the call records similar to the user's current needs are located. This process utilizes the fast positioning characteristics of the hash algorithm and significantly improves the search efficiency. In the second target hash space, the system will further screen out target call records whose similarity with the demand semantic vector exceeds the second threshold. The second threshold is a pre-set value used to filter out historical call records with a low correlation with user needs, ensuring that the recommended toolset is based on the previous use experience that is highly relevant to user needs. Through this screening step, it can be ensured that the collaboratively recommended toolset is historically verified and can effectively meet user needs. Finally, the system determines the second candidate toolset based on the screened target call records. This step requires reverse lookup to extract the historical tool information associated with the call record to form a candidate set. This tool information is not only based on the direct match between the tool description and user needs, but also incorporates historical usage and feedback, so that the second candidate tool set includes tools that have been proven to effectively meet similar needs in the past, thereby increasing the accuracy and practicality of the recommendation.

[0080] Through the above steps, the system can not only filter out the tool sets that match the user's needs from the description of the tool itself (the first candidate tool set), but also mine the tool sets that are suitable for the current needs from the feedback of historical user calls and tool use (the second candidate tool set). The combination of the two can significantly improve the comprehensiveness and personalization of recommendations. This process fully utilizes the advantages of hash space partitioning, which not only ensures the efficiency of the search, but also ensures the accuracy and personalization of the results, providing users with a more intelligent and customized tool call recommendation service.

[0081] In step S208 of the tool recommendation method, a target candidate toolset is determined based on the first candidate toolset and the second candidate toolset, including: obtaining the intersection of the first candidate toolset and the second candidate toolset to obtain a third candidate toolset; performing similarity calculation between the candidate tool vector corresponding to the JSON representation of each tool in the third candidate toolset and the requirement semantic vector to obtain a similarity result; and determining the tool corresponding to the candidate tool vector having a similarity result greater than a third threshold as the target candidate toolset.

[0082] In an embodiment of the present application, after obtaining the first candidate tool set through the first hash space and the second candidate tool set through the second hash space, the system will then calculate the intersection of the two sets of candidate sets and perform deduplication processing. The intersection operation means that the system is looking for tools that are both highly semantically relevant to the current user needs (based on the search results of the first hash space) and have been proven to be effective in historical calls and usage records (based on the search results of the second hash space). By taking the intersection, tools that only match historical data but may not be very relevant to current needs, or tools that are only relevant to current needs but have poor historical use effects can be filtered out, thereby obtaining a third candidate tool set, which contains tool information with high relevance and high historical performance. After determining the third candidate tool set, the system will vectorize the JSON representation of each tool in the set again to obtain a candidate tool vector. Then, the system will calculate the similarity between these candidate tool vectors and the previously generated demand semantic vectors to obtain a series of similarity results. The similarity calculation is measured by the distance between vectors (such as cosine similarity, Euclidean distance, etc.). The closer the value is to 1, the higher the similarity, and vice versa. The system sets a third threshold to filter the tools in the third candidate tool set. Only those candidate tool vectors whose similarity to the required semantic vector is greater than the third threshold will be selected. The setting of this threshold further ensures that the target candidate tool set recommended to the user is not only closely related in semantics, but also has shown good results in historical usage data. Through this refined sorting process, the system can provide a high-quality candidate tool set that takes into account the specificity of current needs and integrates historical user feedback and experience, providing strong support for the user's final tool selection.

[0083] Through the above steps, the system can use two different hash spaces (based on tool description and historical call data) to search and filter based on the user's tool usage needs, and then use vector similarity calculation and threshold filtering to finally determine an accurate target candidate tool set. This process reflects the core idea of ​​the system to achieve intelligent recommendation by integrating multi-source information, which helps to improve the accuracy and practicality of tool recommendations, reduce resource waste, and improve user experience.

[0084] Figure 3 is a structural diagram of a tool recommendation device according to an embodiment of the present application, such as Figure 3 As shown, the device comprises:

[0085] A receiving module 30, used to receive tool usage requirement information input by a target object;

[0086] A conversion module 32, used to convert tool usage requirement information into a requirement semantic vector;

[0087] A first determination module 34 is used to determine a first candidate toolset corresponding to the requirement semantic vector from a first hash space, and to determine a second candidate toolset corresponding to the requirement semantic vector from a second hash space, wherein the first hash space is a hash space corresponding to tool information of all toolsets, and the second hash space is a hash space corresponding to tool call requirements;

[0088] A second determination module 36 is used to determine a target candidate toolset based on the first candidate toolset and the second candidate toolset;

[0089] The recommendation module 38 is used to recommend the target candidate toolset to the target object.

[0090] The tool recommendation device further includes a processing module 40, which is used to obtain an original data set, wherein the original data set includes query data and positive sample data, and the positive sample data includes a JSON representation of a toolset and tool call status; use an embedding model to vector encode the original data set to obtain a vector representation; perform a nearest neighbor search on the query vector in the vector representation to obtain a search result; determine the vectors in the search results whose distance to the query vector is less than a low distance threshold as a positive sample group of the query data; determine the vectors in the search results whose distance to the query vector is greater than a high distance threshold as a negative sample group of the query data, wherein the high distance threshold is greater than the low distance threshold; adjust the parameters of the embedding model through the positive sample group and the negative sample group to obtain a target embedding model.

[0091] In the first determination module in the above-mentioned tool recommendation device, the first determination module is also used to determine the first hash space. Specifically, the first hash space is determined in the following manner: obtaining the JSON representation of all tool sets; using the target embedding model to vectorize the JSON representation of all tool sets to obtain a first semantic vector; using a preset hash function to hash the first semantic vector to generate a first hash value; and determining the first hash space based on the first hash value.

[0092] In the first determination module in the above-mentioned tool recommendation device, the first determination module is also used to determine the second hash space. Specifically, the second hash space is determined in the following manner: obtaining historical tool call information, and obtaining historical usage related to the historical tool call information; using the target embedding model to vectorize the historical tool call information and the historical usage, respectively, to obtain a second semantic vector and a third semantic vector; using a preset hash function to hash the second semantic vector and the third semantic vector, respectively, to generate a second hash value and a third hash value; and determining the second hash space based on the second hash value and the third hash value.

[0093] In the first determination module in the above-mentioned tool recommendation device, the first determination module is also used to input the requirement semantic vector into a preset hash function to generate a requirement hash value of the requirement semantic vector; determine the first target hash space from the first hash space based on the requirement hash value; determine the first target semantic vector from the first target hash space whose similarity with the requirement semantic vector is greater than a first threshold; and determine the first candidate toolset based on the tool JSON representation corresponding to the first target semantic vector.

[0094] In the first determination module in the above-mentioned tool recommendation device, the first determination module is also used to obtain the requirement hash value of the requirement semantic vector; determine the second target hash space from the second hash space based on the requirement hash value; determine the target call record whose similarity with the requirement semantic vector is greater than the second threshold from the second target hash space; and determine the second candidate toolset based on the target call record.

[0095] In the second determination module in the above-mentioned tool recommendation device, the second determination module is also used to obtain the intersection of the first candidate tool set and the second candidate tool set to obtain a third candidate tool set; perform similarity calculation on the candidate tool vector corresponding to the JSON representation of each tool in the third candidate tool set and the requirement semantic vector to obtain a similarity result; and determine the tool corresponding to the candidate tool vector whose similarity result is greater than a third threshold as the target candidate tool set.

[0096] It should be noted that Figure 3 The tools shown are recommended for performing Figure 2 The tool recommendation method shown in the figure, therefore the relevant explanations and instructions in the above tool recommendation method are also applicable to the tool recommendation device, and will not be repeated here.

[0097] An embodiment of the present application also provides an electronic device, including: a memory for storing program instructions; a processor, connected to the memory, for executing program instructions to implement the following functions: receiving tool usage requirement information input by a target object; converting the tool usage requirement information into a requirement semantic vector; determining a first candidate toolset corresponding to the requirement semantic vector from a first hash space, and determining a second candidate toolset corresponding to the requirement semantic vector from a second hash space, wherein the first hash space is a hash space corresponding to tool information of all toolsets, and the second hash space is a hash space corresponding to tool call requirements; determining a target candidate toolset based on the first candidate toolset and the second candidate toolset; and recommending the target candidate toolset to the target object.

[0098] It should be noted that the above electronic equipment is used to execute Figure 2 The tool recommendation method shown, therefore the relevant explanations and instructions in the above tool recommendation method are also applicable to the electronic device and will not be repeated here.

[0099] An embodiment of the present application also provides a non-volatile storage medium, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the following tool recommendation method by running the computer program: receiving tool usage requirement information input by a target object; converting the tool usage requirement information into a requirement semantic vector; determining a first candidate toolset corresponding to the requirement semantic vector from a first hash space, and determining a second candidate toolset corresponding to the requirement semantic vector from a second hash space, wherein the first hash space is a hash space corresponding to tool information of all toolsets, and the second hash space is a hash space corresponding to tool call requirements; determining a target candidate toolset based on the first candidate toolset and the second candidate toolset; and recommending the target candidate toolset to the target object.

[0100] It should be noted that the above non-volatile storage medium is used to execute Figure 2 The tool recommendation method shown, therefore the relevant explanations in the above tool recommendation method are also applicable to the non-volatile storage medium and will not be repeated here.

[0101] An embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the tool recommendation method in each embodiment of the present application.

[0102] The embodiments of the present application also provide a computer program, which, when executed by a processor, implements the steps of the tool recommendation method in each embodiment of the present application.

[0103] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0104] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0105] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0106] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0107] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0108] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0109] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A tool recommendation method, characterized in that: include: Receive tool usage requirement information input by the target object; Converting the tool usage requirement information into a requirement semantic vector; Determine a first candidate toolset corresponding to the requirement semantic vector from a first hash space, and determine a second candidate toolset corresponding to the requirement semantic vector from a second hash space, wherein the first hash space is a hash space corresponding to tool information of all toolsets, and the second hash space is a hash space corresponding to tool call requirements; Determine a target candidate toolset based on the first candidate toolset and the second candidate toolset; The target candidate toolset is recommended to the target object.

2. The method according to claim 1, characterized in that: Before receiving the tool usage requirement information input by the target object, the method further includes: Obtaining an original data set, wherein the original data set includes query data and positive sample data, and the positive sample data includes a JSON representation of a tool set and tool call status; Using an embedding model to vector encode the original data set to obtain a vector representation; Performing a nearest neighbor search on the query vector in the vector representation to obtain search results; Determine the vectors in the search results whose distances from the query vector are less than a low distance threshold as a positive sample group of the query data; Determine the vectors in the search results whose distance from the query vector is greater than a high distance threshold as a negative sample group of the query data, wherein the high distance threshold is greater than the low distance threshold; The parameters of the embedding model are adjusted by using the positive sample group and the negative sample group to obtain a target embedding model.

3. The method according to claim 2, characterized in that The first hash space is determined by: Get a JSON representation of all toolsets; Using the target embedding model to vectorize the JSON representations of all tool sets to obtain a first semantic vector; Using a preset hash function to hash the first semantic vector to generate a first hash value; The first Hash space is determined according to the first Hash value.

4. The method according to claim 2, characterized in that: The second hash space is determined by: Obtaining historical tool call information, and obtaining historical usage information related to the historical tool call information; Using the target embedding model to vectorize the historical tool call information and the historical usage respectively to obtain a second semantic vector and a third semantic vector; Using a preset hash function to hash the second semantic vector and the third semantic vector respectively to generate a second hash value and a third hash value; The second Hash space is determined according to the second Hash value and the third Hash value.

5. The method according to claim 1, characterized in that Determining a first candidate toolset corresponding to the requirement semantic vector from the first hash space includes: Inputting the demand semantic vector into a preset hash function to generate a demand hash value of the demand semantic vector; Determine a first target hash space from the first hash space according to the required hash value; Determine, from the first target hash space, a first target semantic vector whose similarity to the requirement semantic vector is greater than a first threshold; The first candidate toolset is determined according to the tool JSON representation corresponding to the first target semantic vector.

6. The method according to claim 5, characterized in that Determining a second candidate toolset corresponding to the requirement semantic vector from the second hash space includes: Obtaining a demand hash value of the demand semantic vector; Determine a second target hash space from the second hash space according to the required hash value; Determine, from the second target hash space, a target call record whose similarity to the requirement semantic vector is greater than a second threshold; The second candidate toolset is determined according to the target call record.

7. The method according to claim 1, characterized in that Determining a target candidate toolset based on the first candidate toolset and the second candidate toolset includes: Obtaining the intersection of the first candidate tool set and the second candidate tool set to obtain a third candidate tool set; Performing similarity calculation between the candidate tool vector corresponding to the JSON representation of each tool in the third candidate tool set and the requirement semantic vector to obtain a similarity result; The tools corresponding to the candidate tool vectors whose similarity results are greater than a third threshold are determined as the target candidate tool set.

8. A tool recommendation device, characterized in that: include: A receiving module, used for receiving tool usage requirement information input by a target object; A conversion module, used for converting the tool usage requirement information into a requirement semantic vector; A first determination module is used to determine a first candidate toolset corresponding to the requirement semantic vector from a first hash space, and to determine a second candidate toolset corresponding to the requirement semantic vector from a second hash space, wherein the first hash space is a hash space corresponding to tool information of all toolsets, and the second hash space is a hash space corresponding to tool call requirements; A second determining module, configured to determine a target candidate toolset based on the first candidate toolset and the second candidate toolset; A recommendation module is used to recommend the target candidate toolset to the target object.

9. An electronic device, characterized in that: include: A memory for storing program instructions; A processor, connected to the memory, is used to execute program instructions to implement the following functions: receiving tool usage requirement information input by a target object; converting the tool usage requirement information into a requirement semantic vector; determining a first candidate toolset corresponding to the requirement semantic vector from a first hash space, and determining a second candidate toolset corresponding to the requirement semantic vector from a second hash space, wherein the first hash space is a hash space corresponding to tool information of all toolsets, and the second hash space is a hash space corresponding to tool call requirements; determining a target candidate toolset based on the first candidate toolset and the second candidate toolset; and recommending the target candidate toolset to the target object.

10. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the tool recommendation method described in any one of claims 1 to 7 by running the computer program.

11. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the tool recommendation method described in any one of claims 1 to 7 is implemented.

Images