Large language model tool matching method and system based on knowledge graph

Abstract

The invention discloses a big language model tool matching method and system based on a knowledge graph, belongs to the technical field of artificial intelligence, and solves the problem that an API (Application Program Interface) tool is difficult to efficiently manage and accurately select in a real-time information processing process of an existing big language model. According to the method, a multi-source data fusion technology is introduced to collect and sort information of external API tools of different data sources, and a dynamic relation weight adjustment mechanism is utilized to dynamically adjust the weight of the relation between different API nodes according to the actual use condition of the API tools and user feedback; representing the characteristics of each API node through a graph embedding technology based on an intelligent matching algorithm of deep learning, and calculating the matching degree of each candidate API and a request by using a sequence processing model; according to the method, the knowledge graph and the intelligent matching algorithm are combined, an external API tool used by a large language model can be dynamically managed and optimized, and the information processing accuracy and the system response speed are improved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology and relates to a large language model tool matching method and system based on knowledge graph. Background Art

[0002] Large language models have shown excellent performance in processing language tasks. However, large language models have a significant limitation: the knowledge base of large language models is not updated in real time, which limits their application scenarios that require real-time information, such as weather forecasts or stock price updates. To solve this problem, researchers at home and abroad have tried to expand the functionality of the model by integrating external APIs, such as real-time news updates, weather forecasts, etc. Although this approach has improved the practicality of the model to a certain extent, it still faces the challenge of how to efficiently manage and accurately select a large number of available API tools.

[0003] In the existing technology, the selection of API tools by large models is often not intelligent enough, and it is impossible to dynamically select the most appropriate API tool according to specific user requests. For example, in response to the demand for real-time information, the model may need to select the API that provides the fastest data update, but the existing technology cannot effectively evaluate and compare the timeliness and relevance of each API. For example, in the paper "Integrating API Tools into Large Language Models for Improving Real-time Information Processing" (Zhang H, Lee K, Zhang M, et al. arXiv preprint arXiv: 2019.05422, 2019.), this method improves the accuracy of selection by establishing a simple tool index, but this method does not include dynamic evaluation of tool performance, which limits its effectiveness in a changing environment. For example, in the paper "A Machine Learning Approach for Dynamic Tool Selection in Large Language Models" (Liu J, Wang X. Journal of Artificial Intelligence Research, 2020.), it attempts to predict the applicability of tools through machine learning technology, but this method relies on a large amount of historical data and a complex training process, and it is difficult to quickly adapt to the addition of new tools.

[0004] It can be seen that although the existing technology provides certain solutions, in actual application scenarios where there are many API tools and changing requirements, the selection of API tools by large models still has problems of inefficiency and inaccurate selection, which limits the breadth and practicality of large language models in practical applications. Summary of the invention

[0005] The technical solution of the present invention is used to solve the problem that the existing large language model is difficult to efficiently manage and accurately select API tools during real-time information processing.

[0006] The present invention solves the above technical problems through the following technical solutions:

[0007] A large language model tool matching method based on knowledge graph includes the following steps:

[0008] S1. The system receives the user request and parses the text in the request;

[0009] S2. Build a knowledge graph, collect and integrate API-related information using multi-source data fusion technology, adjust the weights of the relationships between different API nodes based on a dynamic relationship weight adjustment mechanism, and query related API tools in the knowledge graph based on the parsed text;

[0010] S3, using an intelligent matching algorithm to select the most relevant API tool from the relevant API tools queried in step S2;

[0011] S4. The system calls the most relevant API tool, sets the API request, receives the data returned by the API and performs data processing; the data processing includes data cleaning, format conversion and semantic optimization;

[0012] S5. The system evaluates and updates the knowledge graph for the relevant API tools found each time.

[0013] S6. After the user receives the information, the system requests the user to evaluate the service quality. The system collects the user's feedback on the use of the API tool and updates the knowledge graph.

[0014] Furthermore, the multi-source data fusion technology in step S2 is specifically:

[0015] S21. Use crawler technology to crawl various types of data related to the API from specified data sources; the data sources include social media, developer forums, and official API documents; the data include function description data, interface specification data, response time data, usage restriction data, user feedback data, documents, and sample codes;

[0016] S22, clean the data and remove irrelevant characters and noise information in the data, and unify the data format;

[0017] S23, extract key information from text using entity recognition and relationship extraction;

[0018] S24. Use text similarity algorithms to identify duplication and complementarity of information in different data sources;

[0019] S25. Merge the information identified as related to form a unified information node.

[0020] Furthermore, the use of a text similarity algorithm to identify the repetition and complementarity of information in different data sources in step S24 is specifically as follows: the text similarity algorithm calculates the TF-IDF vectors of information in different data sources and calculates the cosine similarity between different TF-IDF vectors to obtain similarity scores between different TF-IDF vectors; if the similarity scores between the TF-IDF vectors corresponding to the information in two different data sources are high, then the information is identified as related information; if the similarity scores between the TF-IDF vectors corresponding to the information in two different data sources are low, then the information is identified as unrelated information.

[0021] Furthermore, the step S25 of merging the information identified as related to form a unified information node specifically includes:

[0022] S251. Use entity recognition technology in natural language processing to identify key entities in the text; the key entities include API names and technical terms;

[0023] S252, using an entity matching algorithm to identify and associate descriptions of the same API mentioned in different data sources; the entity matching algorithm includes a string matching algorithm and a fuzzy matching algorithm;

[0024] S253, using knowledge fusion technology to integrate information from different data sources; the knowledge fusion technology includes record linking, authenticity scoring, scoring synthesis, and information integration;

[0025] The record linking specifically involves identifying and matching multiple records describing the same entity from different data sources, so as to identify which data records actually point to the same entity; the authenticity scoring specifically involves applying machine learning technology to score the authenticity of information about the same entity in the captured data, and to determine which information is considered more reliable, and the applied machine learning technology is a decision tree or a random forest; the score synthesis specifically involves synthesizing multiple records of the same entity through weighted averaging, a voting system, or a maximum likelihood estimation method, giving priority to data sources with high scores; the information integration specifically involves merging the above-mentioned evaluated and screened information into a unified view.

[0026] Furthermore, the dynamic relationship weight adjustment mechanism in step S2 is specifically as follows:

[0027] S26. Record the calling frequency, response time, failure rate and user satisfaction of each API, and calculate the functional matching degree between APIs;

[0028] S27. Define weight update rules based on the user satisfaction of each API and the functional matching between APIs;

[0029] S28, using a reinforcement learning algorithm to learn an optimal weight adjustment strategy; the reinforcement learning algorithm is a multi-armed bandit or a Q learning algorithm;

[0030] S29. Periodically update the record data of each API, recalculate and adjust the weight.

[0031] Furthermore, in step S27, based on the user satisfaction of each API and the functional matching degree between the APIs, the weight update rule is defined as follows:

[0032] The weight adjustment factor is calculated based on user satisfaction and failure rate. If there is functional similarity between APIs, the similarity weights between APIs are adjusted according to the weight adjustment factor values ​​of the APIs. The expression of the weight adjustment factor is as follows:

[0033] WAF = α × satisfaction - β × failure rate

[0034] Among them, α and β are adjustment coefficients, which are used to balance the influence of satisfaction and failure rate in weight adjustment; if there is functional similarity between APIs, the similarity weight between them is increased or decreased according to the size of the WAF value.

[0035] Furthermore, the use of the reinforcement learning algorithm to learn the optimal weight adjustment strategy in step S28 is specifically as follows: setting a reward function based on the overall performance of the system, and increasing the reward function value when the combination of APIs shows higher user satisfaction and lower failure rate; after each API call, updating the adjustment coefficient in real time according to the result of the reward function;

[0036] The reinforcement learning algorithm is a multi-armed bandit or Q learning, and the update adjustment coefficient expression is as follows:

[0037] Q(s,a)←Q(s,a)+η(r+γmax a‘ Q(s',a')-Q(s,a))

[0038] Where s is the current state, a is the current action, s' is the new state reached after taking action a, a' is the possible action to be taken in the next state s', η is the learning rate, γ is the discount factor, r is the reward, Q(s,a) is the current Q value of taking action a in state s, max a‘ Q(s',a') is the maximum Q value of all possible actions in the new state s'.

[0039] Furthermore, the smart matching algorithm in step S3 is specifically:

[0040] S31. Use graph embedding technology to represent the characteristics of each API node; the graph embedding technology includes Node2Vec or GraphSAGE, and use a neighbor aggregation function to update the representation of the node. The expression of the neighbor aggregation function is as follows:

[0041]

[0042] Among them, W is the weight matrix, is the feature vector of node v in layer l, is, u is, N(v) is the set of neighbor nodes of node v, MEAN() is a function for calculating the average value of feature vectors of all neighbor nodes, CONCAT() is a function for concatenating the feature vector of node v itself and the average feature vector of its neighbors, W(l) is the weight matrix of the lth layer, which is used for linear transformation of the concatenated vector, σ is an activation function, which is optional. In this embodiment, the activation function is ReLU;

[0043] S32, combining the characteristics of the user request and the API node, using a sequence processing model to evaluate the matching degree between each API and the user request; the sequence processing model is a Transformer model;

[0044] S33. Using the trained sequence processing model, based on the functional matching degree and performance indicators of the API, the most relevant API tool is selected using a sorting algorithm; the performance indicators are response time data and usage restriction data, and the sorting algorithm is softmax sorting.

[0045] Furthermore, the sequence processing model in step S32 is a Transformer model; the expression of the Transformer model is as follows:

[0046]

[0047] Among them, Q is the query matrix, K is the key matrix, and K T is the transposed matrix of the key matrix, V is the value matrix, d k The dimension of the key.

[0048] The present invention also provides a large language model tool matching system based on knowledge graph, including a request receiving module, a knowledge graph building module, an intelligent matching module, a data processing module, a system evaluation module and a user evaluation module;

[0049] The request receiving module is used for the system to receive user requests and parse the text in the request;

[0050] The knowledge graph construction module is used to construct a knowledge graph, collect and integrate API-related information using multi-source data fusion technology, adjust the weights of the relationships between different API nodes based on a dynamic relationship weight adjustment mechanism, and query related API tools in the knowledge graph based on the parsed text;

[0051] The intelligent matching module is used to select the most relevant API tool from the related API tools queried using an intelligent matching algorithm;

[0052] The data processing module is used to call the most relevant API tool, set API requests, receive data returned by the API and perform data processing; the data processing includes data cleaning, format conversion and semantic optimization;

[0053] The system evaluation module is used for the system to evaluate the relevant API tools queried each time and update the knowledge graph;

[0054] The user evaluation module is used for the system to request users to evaluate the service quality after receiving information. The system collects user feedback after using the API tool and updates the knowledge graph.

[0055] The advantages of the present invention are:

[0056] The present invention introduces multi-source data fusion technology to collect and organize information from external API tools from different data sources, and uses a dynamic relationship weight adjustment mechanism to dynamically adjust the weights of the relationships between different API nodes according to the actual usage of the API tools and user feedback, thereby enhancing the functionality and practicality of the knowledge graph. The intelligent matching algorithm based on deep learning proposed in the present invention represents the characteristics of each API node through graph embedding technology, captures the complex relationships and attributes between nodes, and uses a sequence processing model to calculate the degree of matching between each candidate API and the request, thereby improving the matching accuracy of the API tool and enabling the system to quickly and accurately respond to complex and changing user needs. By combining the knowledge graph and the intelligent matching algorithm, the system can more accurately match user needs with suitable API tools to ensure the real-time and accuracy of information. The system continuously optimizes itself through user feedback, so that the system performs well in processing large-scale and complex data requests, meeting the society's demand for efficient and personalized information processing.

[0057] The present invention combines knowledge graphs and intelligent matching algorithms to dynamically manage and optimize external API tools used by large language models. By building a comprehensive knowledge graph, it not only stores detailed information about API tools, but also continuously updates the knowledge graph content based on real-time performance feedback and user reviews, making the selection of API tools not only based on static data, but a dynamic optimization process. This way of matching and managing API tools significantly improves the accuracy of information processing and the response speed of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 is a flow chart of a large language model tool matching method based on a knowledge graph according to an embodiment of the present invention;

[0059] Figure 2 It is a schematic diagram of an example of recording API tool data according to an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and specific embodiments:

[0062] Embodiment 1

[0063] like Figure 1 Specifically, a large language model tool matching method based on a knowledge graph is disclosed, comprising the following steps:

[0064] S1. The system receives the user request and parses the text in the request;

[0065] Suppose a user wants to get real-time updates about the current global epidemic. He can make a request to the system through a chat interface: "Please provide the latest statistics on the current global COVID-19 epidemic." At this stage, the system uses natural language processing technology to extract keywords such as "global", "COVID-19 epidemic", and "latest statistics". These keywords will be used as indexes for subsequent searches of the knowledge graph.

[0066] Specifically, in order to more accurately parse user requests, a context-aware NLP model (natural language processing model) is used in this embodiment, based on the BERT or RoBERTa pre-trained model of the Transformer architecture. Taking the BERT model as an example, by fine-tuning the BERT model, it is better adapted to the domain-specific corpus of API matching, with the user's natural language request as input and the output as key information in the text, such as keywords, intent, etc. By using a context-aware NLP model, it is possible to understand the complex relationship between words and capture long-distance dependencies, improve the accuracy of key information extraction, reduce ambiguity, and improve subsequent matching accuracy.

[0067] S2. Build a knowledge graph, collect and integrate API-related information using multi-source data fusion technology, adjust the weights of the relationships between different API nodes based on a dynamic relationship weight adjustment mechanism, and query related API tools in the knowledge graph based on the parsed text;

[0068] The multi-source data fusion technology described in step S2 is specifically:

[0069] S21. Use crawler technology to crawl various types of data related to the API from specified data sources; the data sources include social media, developer forums, and official API documents; the data include function description data, interface specification data, response time data, usage restriction data, user feedback data, documents, and sample codes;

[0070] Specifically, the function description data includes the main functions, uses, input and output formats of the API; the interface specification data includes the request method of the API (such as GET, POST, etc.), parameter description, data type, request limit and other data; the response time data includes the average time, fastest response time and slowest response time of the API response request; the usage restriction data includes the call frequency limit, access rights, cost and other data of the API; the user feedback data includes evaluations, discussions and use cases from social media, developer forums and technical blogs; the documents and sample codes include the official documents, sample codes and tutorial links of the API.

[0071] Based on crawler technology, the system automatically collects the above data regularly to ensure the freshness and extensiveness of the data and provide original data for multi-source data fusion.

[0072] S22, clean the data and remove irrelevant characters and noise information in the data, and unify the data format;

[0073] S23, extract key information from text using entity recognition and relationship extraction;

[0074] This embodiment uses entity recognition (NER) technology to identify and classify named entities from text, identify proper nouns in the text and mark their types; and uses relationship extraction technology to identify and extract relationships between entities from text, and determine which entities in the text have a certain specific relationship. Specifically, this embodiment can use the BERT model to implement entity recognition and relationship extraction.

[0075] S24, using a text similarity algorithm to identify the repetition and complementarity of information in different data sources; the text similarity algorithm is to calculate the TF-IDF vectors of information in different data sources and calculate the cosine similarity between different TF-IDF vectors to obtain the similarity scores between different TF-IDF vectors; if the similarity score between the TF-IDF vectors corresponding to the information in two different data sources is high, then the information is identified as related; if the similarity score between the TF-IDF vectors corresponding to the information in two different data sources is low, then the information is identified as unrelated;

[0076] S25, merging the information identified as related to form a unified information node;

[0077] The step S25 of merging the identified related information to form a unified information node is as follows:

[0078] S251. Use entity recognition technology in natural language processing to identify key entities in the text; the key entities include API names and technical terms;

[0079] S252, using an entity matching algorithm to identify and associate descriptions of the same API mentioned in different data sources; the entity matching algorithm includes a string matching algorithm and a fuzzy matching algorithm;

[0080] S253, using knowledge fusion technology to integrate information from different data sources; the knowledge fusion technology includes record linking, authenticity scoring, scoring synthesis, and information integration;

[0081] The record linking is specifically to identify and match multiple records describing the same entity from different data sources, so as to identify which data records actually point to the same entity.

[0082] The authenticity scoring is specifically to apply machine learning technology to score the "authenticity" of the information of the same entity in the captured data to determine which information is considered more reliable. This embodiment can be implemented using a decision tree or a random forest.

[0083] The score synthesis is specifically to synthesize multiple records of the same entity by weighted average, voting system or maximum likelihood estimation, and give priority to data sources with high scores. For example, the weight parameter is used to emphasize that the reliability of official documents is higher than that of social media, and the expression is as follows:

[0084] Integrated Value=Σω i ×v i

[0085] Among them, ω i is the data source weight, v i The value extracted from the data source.

[0086] The information set is used to merge the above-mentioned evaluated and filtered information into a unified and accurate view, ensuring that the final data set is comprehensive and accurate for subsequent use and analysis.

[0087] The core of multi-source data fusion technology is how to effectively merge information from different data sources into a unified and accurate view. The present invention uses multi-source data fusion technology to collect and organize information from external API tools from different data sources, such as function descriptions, interface specifications, response times, and usage restrictions, and integrates the above information through natural language processing technology. This information is stored in the graph database, and each tool is a node, containing detailed attributes and performance indicators, which increases the richness and diversity of the knowledge graph and improves the accuracy and practicality of the information.

[0088] The dynamic relationship weight adjustment mechanism in step S2 is specifically:

[0089] S26. Record the calling frequency, response time, failure rate and user satisfaction of each API, and calculate the functional matching degree between APIs;

[0090] Specifically, in order to monitor the usage of the API, the system needs to collect and analyze data such as the frequency of API calls, response time, failure rate, and user satisfaction. Figure 2 This is a data example, showing the usage statistics of several APIs within a certain period of time. These data can be automatically collected by the log analysis system, and then sorted and counted through the data processing process.

[0091] The calculation of the functional matching degree between the APIs specifically includes converting the user request and the API description text into TF-IDF vectors, calculating the cosine similarity between different TF-IDF vectors, and obtaining the similarity scores between different TF-IDF vectors.

[0092] S27. Define weight update rules based on the user satisfaction of each API and the functional matching between APIs;

[0093] According to the user satisfaction of each API and the functional matching degree between the APIs, the weight update rule is defined in step S27 as follows: the weight adjustment factor is calculated based on the user satisfaction and the failure rate, and if there is functional similarity between the APIs, the similarity weight between the APIs is adjusted according to the weight adjustment factor value of the API;

[0094] The weight update can be calculated based on a combination of user satisfaction and failure rate. The expression of the weight adjustment factor (WAF) is as follows:

[0095] WAF = α × satisfaction - β × failure rate

[0096] Among them, α and β are adjustment coefficients, which are used to balance the influence of satisfaction and failure rate in weight adjustment.

[0097] If there is functional similarity between APIs, increase or decrease the similarity weight between them according to the size of the WAF value. For example, if API_A and API_B have similar functions, but API_A has a lower WAF than API_B, then reduce the similarity weight from API_A to API_B, otherwise increase it.

[0098] The defined weight updating rules disclosed in the present invention dynamically adjust the weights of the relationships between APIs based on API usage data and user feedback. If the user satisfaction of an API is high, the similarity weights of APIs with similar functions may increase.

[0099] S28, using a reinforcement learning algorithm to learn an optimal weight adjustment strategy; the reinforcement learning algorithm is a multi-armed bandit or a Q learning algorithm;

[0100] The method of using the reinforcement learning algorithm to learn the optimal weight adjustment strategy in step S28 is as follows: setting a reward function based on the overall performance of the system, and increasing the reward function value when the combination of APIs shows higher user satisfaction and lower failure rate; after each API call, updating the adjustment coefficient in real time according to the result of the reward function;

[0101] Specifically, in order to adjust the weights more accurately, the present invention uses an online learning algorithm to update the adjustment coefficients α and β in step S27 in real time. The adjustment coefficients α and β will affect the importance of satisfaction and failure rate in weight adjustment.

[0102] The reinforcement learning algorithm is a multi-armed bandit or Q-Learning, which is used to dynamically adjust the adjustment coefficients α and β.

[0103] Taking the Q-Learning algorithm as an example, the adjustment coefficient update expression is as follows:

[0104] Q(s,a)←Q(s,a)+η(r+γmax a‘ Q(s',a')-Q(s,a))

[0105] Where s is the current state, a is the current action, s' is the new state reached after taking action a, a' is the possible action to be taken in the next state s', η is the learning rate, γ is the discount factor, r is the reward, Q(s,a) is the current Q value of taking action a in state s, max a‘Q(s',a') is the maximum Q value of all possible actions in the new state s'. The adjustment coefficients α and β can directly affect the decision-making and strategy optimization in the learning process, so that the entire system can self-optimize and adjust based on feedback.

[0106] S29, periodically update the record data of each API, recalculate and adjust the weight;

[0107] In the knowledge graph, the relationships between different API nodes, such as functional similarity and data dependency, are represented by weights. The dynamic relationship weight adjustment mechanism proposed in the present invention can dynamically adjust the weights of these relationships based on the actual usage of the API and user feedback.

[0108] Based on the keywords extracted in step S1, the system queries the knowledge graph for relevant API tools. Each node in the knowledge graph represents an API, and the node attributes describe the API's functions, response time, data coverage, etc. The system selects the API that best meets current needs by evaluating the relevance and performance indicators of each API.

[0109] S3, using an intelligent matching algorithm to select the most relevant API tool from the relevant API tools queried in step S2;

[0110] The smart matching algorithm in step S3 is specifically:

[0111] S31. Use graph embedding technology to represent the characteristics of each API node; the graph embedding technology includes Node2Vec or GraphSAGE. These embeddings can capture the complex relationships and properties between nodes, further combine node attributes and structural information, improve the quality and expressiveness of embedding, and provide a mathematical basis for subsequent matching.

[0112] Taking GraphSAGE as an example, a neighbor aggregation function is used to update the representation of a node. The expression of the neighbor aggregation function is as follows:

[0113]

[0114] Among them, W is the weight matrix, is the feature vector of node v in layer l, is, u is, N(v) is the set of neighbor nodes of node v, MEAN() is the function for calculating the average value of the feature vectors of all neighbor nodes, CONCAT() is the function for concatenating the feature vector of node v itself and the average feature vector of its neighbors, W(l) is the weight matrix of the lth layer, which is used to linearly transform the concatenated vector, σ is the activation function, which is optional. In this embodiment, the activation function is ReLU.

[0115] By adopting graph embedding technology, the expressive power of node embedding is enhanced, which can better capture the complex relationships between APIs and improve the relevance and accuracy of subsequent matching.

[0116] S32, combining the characteristics of the user request and the API node, using a sequence processing model to evaluate the matching degree between each API and the user request; the sequence processing model is a Transformer model;

[0117] Specifically, the Transformer architecture can combine the sequence features of user requests and the node representation of graph embedding to process sequence dependencies. The Transformer model can more effectively handle long-distance dependency problems, improve the efficiency and accuracy of sequence data processing, and is superior to traditional RNN and LSTM models.

[0118] This example uses the Transformer architecture and a self-attention layer to calculate the relationship between the parts in the sequence. The expression is as follows:

[0119]

[0120] Among them, Q is the query matrix, K is the key matrix, and K T is the transposed matrix of the key matrix, V is the value matrix, d k The dimension of the key.

[0121] S33, using the trained sequence processing model, based on the functional matching degree and performance indicators of the API, and using a sorting algorithm to select the most relevant API tool; the performance indicators are response time data and usage restriction data, and the sorting algorithm is softmax sorting;

[0122] Furthermore, the present invention can further optimize the selected API tool list using an optimization algorithm to ensure that the final selection can minimize response time and cost while meeting functional requirements; the optimization algorithm includes a genetic algorithm or a simulated annealing algorithm.

[0123] The deep learning-based intelligent matching algorithm proposed in the present invention represents the characteristics of each API node through graph embedding technology, captures the complex relationships and attributes between nodes, and uses the sequence processing model to calculate the matching degree between each candidate API and the request. Taking into account the response time and usage restrictions of API tools, the system selects the API tool with the highest score after sorting as the most relevant API tool and calls it. This not only improves the matching accuracy of API tools, but also significantly optimizes the processing efficiency, enabling the system to respond quickly and accurately to complex and changeable user needs.

[0124] S4. The system calls the most relevant API tool, sets the API request, receives the data returned by the API and performs data processing; the data processing includes data cleaning, format conversion and semantic optimization;

[0125] Specifically, the system calls the API tool most relevant to the request through the network, sets the API request, receives the data returned by the API, and processes the data to meet the input requirements of the large language model. For example, in order to adapt to the display requirements of the large language model user interface, the data in JSON format needs to be converted into easy-to-read text or charts. By processing the data returned by the API, the system can better meet the detailed needs of users, ensure that the selected API tool can effectively serve the specific needs of users, and improve the overall service quality.

[0126] S5. The system evaluates and updates the knowledge graph for the relevant API tools found each time.

[0127] The system regularly monitors the performance of all integrated API tools and automatically updates the knowledge graph based on the monitoring results. Specifically, if the actual response time data of an API tool is longer than expected, the system will adjust the performance score of the API tool. Furthermore, the system also optimizes the matching algorithm through user feedback data to ensure that the knowledge graph data is always up to date and improve the processing efficiency and accuracy of future requests.

[0128] S6. After the user receives the information, the system requests the user to evaluate the service quality. The system collects the user's feedback on the use of the API tool and updates the knowledge graph;

[0129] Specifically, the system collects user feedback on API tools, including satisfaction survey data and specific usage feedback data. These data directly affect the performance scores of relevant API tools in the knowledge graph, and are used to analyze and evaluate the effectiveness of API tools and their performance in practical applications, and further guide the update of knowledge graphs and the optimization of intelligent matching algorithms.

[0130] The present invention also provides a large language model tool matching system based on knowledge graph, including a request receiving module, a knowledge graph building module, an intelligent matching module, a data processing module, a system evaluation module and a user evaluation module;

[0131] The request receiving module is used for the system to receive user requests and parse the text in the request;

[0132] The knowledge graph construction module is used to construct a knowledge graph, collect and integrate API-related information using multi-source data fusion technology, adjust the weights of the relationships between different API nodes based on a dynamic relationship weight adjustment mechanism, and query related API tools in the knowledge graph based on the parsed text;

[0133] The intelligent matching module is used to select the most relevant API tool from the related API tools queried using an intelligent matching algorithm;

[0134] The data processing module is used to call the most relevant API tool, set API requests, receive data returned by the API and perform data processing; the data processing includes data cleaning, format conversion and semantic optimization;

[0135] The system evaluation module is used for the system to evaluate the relevant API tools queried each time and update the knowledge graph;

[0136] The user evaluation module is used for the system to request users to evaluate the service quality after receiving information. The system collects user feedback after using the API tool and updates the knowledge graph.

[0137] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A large language model tool matching method based on knowledge graph, characterized in that: The following steps are involved: S1. The system receives the user request and parses the text in the request; S2. Build a knowledge graph, collect and integrate API-related information using multi-source data fusion technology, adjust the weights of the relationships between different API nodes based on a dynamic relationship weight adjustment mechanism, and query related API tools in the knowledge graph based on the parsed text; S3, using an intelligent matching algorithm to select the most relevant API tool from the relevant API tools queried in step S2; S4. The system calls the most relevant API tool, sets the API request, receives the data returned by the API and performs data processing; the data processing includes data cleaning, format conversion and semantic optimization; S5. The system evaluates and updates the knowledge graph for the relevant API tools found each time. S6. After the user receives the information, the system requests the user to evaluate the service quality. The system collects the user's feedback on the use of the API tool and updates the knowledge graph.

2. The large language model tool matching method based on knowledge graph according to claim 1 is characterized in that: The multi-source data fusion technology described in step S2 is specifically: S21. Use crawler technology to crawl various types of data related to the API from specified data sources; the data sources include social media, developer forums, and official API documents; the data include function description data, interface specification data, response time data, usage restriction data, user feedback data, documents, and sample codes; S22, clean the data and remove irrelevant characters and noise information in the data, and unify the data format; S23, extract key information from text using entity recognition and relationship extraction; S24. Use text similarity algorithms to identify duplication and complementarity of information in different data sources; S25. Merge the information identified as related to form a unified information node.

3. The large language model tool matching method based on knowledge graph according to claim 2 is characterized in that: The use of a text similarity algorithm to identify the repetition and complementarity of information in different data sources in step S24 is specifically as follows: the text similarity algorithm calculates the TF-IDF vectors of information in different data sources and calculates the cosine similarity between different TF-IDF vectors to obtain similarity scores between different TF-IDF vectors; if the similarity scores between the TF-IDF vectors corresponding to the information in two different data sources are high, then the information is identified as related information; if the similarity scores between the TF-IDF vectors corresponding to the information in two different data sources are low, then the information is identified as unrelated information.

4. The large language model tool matching method based on knowledge graph according to claim 3 is characterized in that: The step S25 of merging the identified related information to form a unified information node is as follows: S251. Use entity recognition technology in natural language processing to identify key entities in the text; the key entities include API names and technical terms; S252, using an entity matching algorithm to identify and associate descriptions of the same API mentioned in different data sources; the entity matching algorithm includes a string matching algorithm and a fuzzy matching algorithm; S253, using knowledge fusion technology to integrate information from different data sources; the knowledge fusion technology includes record linking, authenticity scoring, scoring synthesis, and information integration; The record linking specifically involves identifying and matching multiple records describing the same entity from different data sources, so as to identify which data records actually point to the same entity; the authenticity scoring specifically involves applying machine learning technology to score the authenticity of information about the same entity in the captured data, and to determine which information is considered more reliable, and the applied machine learning technology is a decision tree or a random forest; the score synthesis specifically involves synthesizing multiple records of the same entity through weighted averaging, a voting system, or a maximum likelihood estimation method, giving priority to data sources with high scores; the information integration specifically involves merging the above-mentioned evaluated and screened information into a unified view.

5. The large language model tool matching method based on knowledge graph according to claim 2 is characterized in that: The dynamic relationship weight adjustment mechanism in step S2 is specifically: S26. Record the calling frequency, response time, failure rate and user satisfaction of each API, and calculate the functional matching degree between APIs; S27. Define weight update rules based on the user satisfaction of each API and the functional matching between APIs; S28. Use reinforcement learning algorithm to learn the optimal weight adjustment strategy; The reinforcement learning algorithm is a multi-armed bandit or a Q learning algorithm; S29. Periodically update the record data of each API, recalculate and adjust the weight.

6. The large language model tool matching method based on knowledge graph according to claim 5 is characterized in that: In step S27, based on the user satisfaction of each API and the functional matching degree between the APIs, the weight update rule is defined as follows: The weight adjustment factor is calculated based on user satisfaction and failure rate. If there is functional similarity between APIs, the similarity weights between APIs are adjusted according to the weight adjustment factor values ​​of the APIs. The expression of the weight adjustment factor is as follows: WAF = α × satisfaction - β × failure rate Among them, α and β are adjustment coefficients, which are used to balance the influence of satisfaction and failure rate in weight adjustment; if there is functional similarity between APIs, the similarity weight between them is increased or decreased according to the size of the WAF value.

7. The large language model tool matching method based on knowledge graph according to claim 6 is characterized in that: The method of using the reinforcement learning algorithm to learn the optimal weight adjustment strategy in step S28 is specifically as follows: setting a reward function based on the overall performance of the system, and increasing the reward function value when the combination of APIs shows higher user satisfaction and lower failure rate; After each API call, the adjustment coefficient is updated in real time based on the result of the reward function; The reinforcement learning algorithm is a multi-armed bandit or Q learning, and the update adjustment coefficient expression is as follows: Q(s,a)←Q(s,a)+η(r+γmax a‘ Q(s‘,a‘)-Q(s,a)) Where s is the current state, a is the current action, s' is the new state reached after taking action a, a' is the possible action to be taken in the next state s', η is the learning rate, γ is the discount factor, r is the reward, Q(s,a) is the current Q value of taking action a in state s, max a‘ Q(s',a') is the maximum Q value of all possible actions in the new state s'.

8. The large language model tool matching method based on knowledge graph according to claim 2 is characterized in that: The smart matching algorithm in step S3 is specifically: S31. Use graph embedding technology to represent the characteristics of each API node; the graph embedding technology includes Node2Vec or GraphSAGE, and use a neighbor aggregation function to update the representation of the node. The expression of the neighbor aggregation function is as follows: Where W is the weight matrix, is the feature vector of node v in layer l, is, u is, N(v) is the set of neighbor nodes of node v, MEAN() is a function for calculating the average value of feature vectors of all neighbor nodes, CONCAT() is for concatenating the feature vector of node v itself and the average feature vector of its neighbors, W(l) is the weight matrix of the lth layer, which is used for linear transformation of the concatenated vector, σ is an activation function, which is optional. In this embodiment, the activation function is ReLU; S32, combining the characteristics of the user request and the API node, using a sequence processing model to evaluate the matching degree between each API and the user request; the sequence processing model is a Transformer model; S33. Using the trained sequence processing model, based on the functional matching degree and performance indicators of the API, the most relevant API tool is selected using a sorting algorithm; the performance indicators are response time data and usage restriction data, and the sorting algorithm is softmax sorting.

9. The large language model tool matching method based on knowledge graph according to claim 8 is characterized in that: The sequence processing model in step S32 is a Transformer model; the expression of the Transformer model is as follows: Among them, Q is the query matrix, K is the key matrix, and K T is the transposed matrix of the key matrix, V is the value matrix, d k The dimension of the key.

10. A large language model tool matching system based on knowledge graph, characterized in that: It includes a request receiving module, a knowledge graph building module, an intelligent matching module, a data processing module, a system evaluation module and a user evaluation module; The request receiving module is used for the system to receive user requests and parse the text in the request; The knowledge graph construction module is used to construct a knowledge graph, collect and integrate API-related information using multi-source data fusion technology, adjust the weights of the relationships between different API nodes based on a dynamic relationship weight adjustment mechanism, and query related API tools in the knowledge graph based on the parsed text; The intelligent matching module is used to select the most relevant API tool from the related API tools queried using an intelligent matching algorithm; The data processing module is used to call the most relevant API tool, set API requests, receive data returned by the API and perform data processing; the data processing includes data cleaning, format conversion and semantic optimization; The system evaluation module is used for the system to evaluate the relevant API tools queried each time and update the knowledge graph; The user evaluation module is used for the system to request users to evaluate the service quality after receiving information. The system collects user feedback after using the API tool and updates the knowledge graph.

Images