A dual-layer debugging-driven API arrangement method and system based on LLM

Abstract

The present application belongs to API orchestration technology, and discloses an API orchestration method and system based on LLM dual-layer debugging drive, which includes: obtaining original API information, including response examples and tool descriptions; prompting LLM to generate response architecture and summary and generate API documents according to the response examples; LLM generates enhanced tool descriptions according to API documents and tool descriptions; vectorizing tool descriptions and requirement instructions, performing similarity matching, and obtaining the most relevant top N tools and their documents; decomposing tasks into subtasks through LLM, and assigning main API and alternative API to each subtask; according to the main API, LLM generates pseudocode for orchestrating API calls and converts it into executable code; if the execution fails, judging whether the error is a design layer or implementation layer error according to the error information, and if it is a design layer error, using the alternative API to repair the pseudocode. This method improves the efficiency and accuracy of API orchestration by automatically generating documents, task planning and debugging.

Description

Technical Field

[0001] The present application relates to computer software development and API orchestration technology, and specifically to an API orchestration method and system based on LLM-based dual-layer debugging drive. Background Art

[0002] In modern software development and system integration, API (application programming interface) calls are the core technical means to connect multiple systems, services, and functions. Through API calls, developers can integrate different systems, services, and functions. Currently, API calls are not only applicable to simple requirements, but also widely used in complex demand scenarios, in which API orchestration and management has become a key technical issue.

[0003] With the rapid development of large language model (LLM) technology, API orchestration methods based on LLM have gradually become mainstream, guiding LLM to complete complex API call tasks by designing task descriptions and API document fragments. However, existing technologies have problems such as inconsistency of API documents, redundant information, and missing documents, which increase the difficulty of LLM in parsing APIs and performing orchestration. In existing API orchestration methods, LLM is prone to blind iteration when handling complex tasks, resulting in low task execution efficiency. In addition, LLM cannot accurately determine the root cause of API call failures during the reflection phase, which increases the difficulty of error location and repair. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an API orchestration method and system based on LLM-based dual-layer debugging driver, aiming to improve the efficiency and accuracy of API calls.

[0005] In order to solve the above technical problems, this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides an API arrangement method for a dual-layer debugging driver based on LLM, the method comprising:

[0007] Crawling original API information from the API platform, wherein the original API information includes an API response example and a tool description;

[0008] Prompt the LLM to generate an API response schema and an API response summary based on the API response example;

[0009] Prompt the LLM to generate an API document according to the original API information, the API response summary, and the API response architecture;

[0010] prompting LLM to generate an enhanced tool description according to the API document and the tool description;

[0011] Vectorizing the tool description to obtain a tool description vector;

[0012] Vectorize the input demand instruction to obtain a demand instruction vector;

[0013] Performing similarity matching between the requirement instruction vector and the tool description vector, obtaining the top N tools with the highest similarity and API documents corresponding to the top N tools;

[0014] According to the first N tools and the API documents corresponding to the first N tools, prompt the LLM to perform task planning to decompose the task into multiple subtasks, and allocate a main API and an alternative API to each of the subtasks;

[0015] According to the main API, prompt the LLM to generate a first pseudo code for arranging the calling API;

[0016] According to the first pseudo code, prompt LLM to convert into Python code;

[0017] Determine whether the Python code is executed successfully. If the execution is unsuccessful, prompt LLM to determine whether it is a design layer error or an implementation layer error based on the error message of the unsuccessful execution and the API documents corresponding to the first N tools;

[0018] If a design layer error is prompted, the alternative API is used to prompt LLM to generate a second pseudo code for choreographing the API call for repair; if an implementation layer error is prompted, the main API document is used to prompt LLM to repair the Python code for the API choreographing call.

[0019] As an optional implementation of the first aspect of the present application, the step of crawling original API information from the API platform includes: the API platform is a RapidAPI Hub platform, obtaining a RESTful API from the RapidAPI Hub platform, and using the RESTful API as the original API information.

[0020] As an optional implementation of the first aspect of the present application, in the step of crawling original API information from the API platform, the original API information includes an API response example and a tool description, the API response example is used to display the response data structure in JSON format returned after the API call, and the tool description includes an overall description of the original API information tool.

[0021] As an optional implementation of the first aspect of the present application, the LLM is the GPT4 model of OpenAI.

[0022] As an optional implementation of the first aspect of the present application, the step of vectorizing the input demand instruction to obtain the demand instruction vector includes: using OpenAI's text-embedding-ada-002 model to vectorize the user's input demand instruction to convert it into a numerical representation in a high-dimensional vector space.

[0023] As an optional implementation of the first aspect of the present application, cosine similarity is used to represent the similarity match between the requirement instruction vector and the tool description vector, wherein the calculation formula of cosine similarity is: , where represents the demand instruction vector, represents the tool description vector, represents the dot product of vectors, and They represent the module lengths of the requirement instruction vector and the tool description vector respectively.

[0024] As an optional implementation of the first aspect of the present application, the step of determining whether the Python code is executed successfully includes: running the executable Python code in the integrated development environment Pycharm, obtaining the code running result, and the code running result is used to display whether the executable Python code is executed successfully.

[0025] In a second aspect, an embodiment of the present application provides an API orchestration system based on a dual-layer debugging driver of LLM, the system comprising:

[0026] An API knowledge base construction module is used to crawl original API information from an API platform, wherein the original API information includes an API response example and a tool description; prompt an LLM to generate an API response architecture and an API response summary according to the API response example; prompt an LLM to generate an API document according to the original API information, the API response summary and the API response architecture; prompt an LLM to generate an enhanced tool description according to the API document and the tool description; vectorize the tool description to obtain a tool description vector;

[0027] The API orchestration module is used to vectorize the input demand instructions to obtain a demand instruction vector; perform similarity matching between the demand instruction vector and the tool description vector to obtain the top N tools with the highest similarity and the API documents corresponding to the top N tools; prompt the LLM to perform task planning based on the top N tools and the API documents corresponding to the top N tools to decompose the tasks into multiple subtasks, and assign a main API and an alternative API to each of the subtasks; based on the main API, prompt the LLM to generate a first pseudo code for orchestrating the API call; based on the first pseudo code, prompt the LLM to convert into Python code;

[0028] A two-layer debugging module is used to determine whether the Python code is executed successfully. If the execution is unsuccessful, the LLM is prompted to determine whether it is a design layer error or an implementation layer error based on the error message of the unsuccessful execution and the API documents corresponding to the first N tools; if a design layer error is prompted, the alternative API is used to prompt the LLM to generate a second pseudo code for choreographing and calling the API for repair; if an implementation layer error is prompted, the main API document is used to prompt the LLM to repair the first pseudo code for the API orchestration call.

[0029] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in the first aspect.

[0030] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0031] Compared with the prior art, the present invention proposes an API arrangement method for a double-layer debugging driver based on LLM, which has the following advantages:

[0032] 1. Construction of a unified and high-quality API knowledge base: By crawling the original API information from the API platform and using LLM to generate API descriptions in a unified format, the inconsistency, redundancy and incompleteness of existing API documents are solved.

[0033] 2. Avoid the inefficiency of blind iteration: In traditional methods, LLM may repeatedly try failed API calls when scheduling tasks, resulting in increased time costs. The present invention generates a clear pseudo-code blueprint in the design phase through a phased design and implementation strategy to ensure the efficiency and accuracy of task execution.

[0034] 3. Design a two-layer debugging mechanism: By dividing the debugging process into the design layer and the implementation layer, the efficiency and accuracy of error location and repair are significantly improved. In the design phase, task decomposition and API allocation can be re-evaluated; in the implementation phase, syntax, logic or API call errors can be corrected. This mechanism reduces the time and cost of error repair and improves the accuracy of error location.

[0035] 4. Improve the transparency and controllability of the API calling process: By clearly dividing the design phase and the implementation phase, the system can ensure the clarity of API selection and calling logic in the design phase, and effectively control code generation and execution in the implementation phase. This phased strategy improves the operability of the task, reduces the risk of failure caused by design or implementation problems, and ensures the stability of task execution.

[0036] 5. Flexibility in dealing with complex API combinations and multi-task scheduling: Through phased orchestration and a two-layer debugging mechanism, it can efficiently handle complex API combinations, ensure compatibility and collaboration between multiple APIs, and thus cope with more complex orchestration tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a first flow chart of an API arrangement method for a dual-layer debugging driver based on LLM provided by the first embodiment of the present invention;

[0038] Figure 2 is a second flow chart of an API arrangement method for a dual-layer debugging driver based on LLM provided by the first embodiment of the present invention;

[0039] Figure 3 It is a structural diagram of an LLM-based dual-layer debugging-driven API orchestration system provided by the second embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here. In addition, the "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated with each other are in an "or" relationship.

[0042] For ease of understanding, the technical terms involved in the embodiments of the present invention are explained as follows:

[0043] API (Application Programming Interface): Application programming interface, used for interaction between different software systems. API defines the communication protocol between software components, allowing developers to access specific functions or data by calling API. The API used in this application is a RESTful API obtained from the RapidAPI Hub platform. RapidAPI Hub is a leading API management platform that provides a large number of RESTful API services, covering multiple categories such as data analysis, social media management, payment systems, etc. Developers can search, connect and manage various API services through RapidAPI Hub, and easily call these APIs through a RapidAPI key, simplifying the management and integration process of APIs. RESTful API is an API design style based on the HTTP protocol. It performs operations through HTTP methods (such as GET, POST, PUT, DELETE, etc.), and usually returns data in JSON format. RESTful API is lightweight, easy to extend and cross-platform, and is widely used in modern software development.

[0044] The original API information crawled from RapidAPI Hub includes tool name, tool description, API name, API description, API response example, parameter list and parameter description, etc. Among them, API description is a brief description of API functions to help developers understand the purpose of API; API response example shows the response data structure in JSON format returned after API call; parameter list contains information such as parameter name, type and whether it is required for API call; parameter description gives a detailed description of each parameter to help developers use API correctly; tool description is a general description of the tool containing the API to help developers understand the function and purpose of the tool.

[0045] LLM (Large Language Model): A large language model that can process natural language tasks and generate text. LLM is trained with a large amount of text data and has powerful natural language understanding and generation capabilities. It can perform complex reasoning and problem-solving tasks with a designed prompt word. In this invention, the LLM used is the GPT4 model of OpenAI.

[0046] Pseudocode: A descriptive language similar to a programming language that is used to describe the logical flow of an algorithm, but is independent of a specific programming language. Pseudocode is often used in the design phase to help developers clarify the logic before writing actual code.

[0047] Vectorization: The process of converting text or other data into numerical vectors, which is commonly used for similarity calculations in machine learning and natural language processing. In this invention, OpenAI's text-embedding-ada-002 model is used to generate vector representations of text. This model is a pre-trained language model based on the Transformer architecture that can convert text into numerical representations in a high-dimensional vector space. Through this vector representation, the similarity between texts can be calculated, thereby performing efficient retrieval and matching.

[0048] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0049] Example 1

[0050] It should be noted that in the process of API calling, API documentation is an indispensable component. It usually contains a variety of metadata such as API description, parameter description, return value information and usage examples, providing developers with detailed descriptions of API functions and usage methods, and is also an important basis for LLM to understand and use APIs. However, existing API documents have significant problems in supporting LLM for API orchestration. These problems not only increase the difficulty of LLM parsing and using APIs, but also affect the efficiency and accuracy of task execution. First, a large number of API documents from different sources have inconsistencies in format and content. API documents from different sources usually use different formats (such as Markdown, HTML or PDF), and there are significant differences in the terms and structures used when describing API functions, parameters and return values. For example, some API documents concentrate parameter descriptions in the "Request Parameters" section, while others are scattered in multiple paragraphs, making it difficult for LLM to accurately extract key information. Second, the redundant information in API documents further exacerbates this problem. Many API documents contain a large number of repeated descriptions, outdated examples or irrelevant background information, which not only increases the complexity of the documents, but also makes it difficult for LLM to quickly grasp the core functions of the API. Finally, the incompleteness of the documents is also a prominent problem. High-quality API documentation should provide complete function descriptions, parameter descriptions, and return value formats. However, many API documents lack key information, such as detailed descriptions of parameter types or specific formats of return values. This incompleteness not only increases the difficulty for LLM to call the API, but may also cause the call to fail or return an incorrect result.

[0051] In addition to the documentation problem, the existing API orchestration methods, although they have improved the flexibility and execution efficiency of API calls to a certain extent, are prone to blind iteration when dealing with complex tasks. For example, in the ReAct framework, the model generates interleaved reasoning and operations through the "Thought-Action" mechanism, but in some cases, the model may repeatedly try to call failed APIs, resulting in inefficient task execution. This blind iteration not only increases the time cost of task execution, but may also lead to resource waste.

[0052] In addition, when the API selected in the planning stage is executed according to a specific call logic (sequence, condition or loop) and an error occurs, it is impossible to accurately determine the root cause of the error through reflection alone - whether it is improper API selection and arrangement (such as wrong API selection or unreasonable call logic design) or improper API use (such as failure due to errors in return type, number of parameters, parameter type, etc.). Especially when facing APIs with similar functions but subtle differences, LLM often finds it difficult to distinguish these subtle differences due to its lack of in-depth understanding of API design intent and implementation details, which increases the risk of errors and makes it more difficult to locate the problem.

[0053] See also Figure 1 and Figure 2 , Figure 1 This is a flowchart of an API arrangement method based on LLM double-layer debugging driver proposed in the first embodiment of the present application. Figure 2 A second flow chart of an API arrangement method for a dual-layer debugging driver based on LLM is provided for the first embodiment of the present invention, and the steps of the proposed method are as follows.

[0054] Step S1: crawling original API information from the API platform, where the original API information includes API response examples and tool descriptions.

[0055] In some implementations, the API platform includes a Rapid API platform, and the crawled raw API information also includes a tool name, an API name, an API description parameter list, and a parameter description.

[0056] In this step, the system uses the API provided by Selenium to write crawler scripts to crawl the original API information from the Rapid API platform. Through Selenium simulating browser operations, the system automatically logs in and navigates to each tool page, extracts information such as tool name, tool description, API name, API description, API response example, parameter list and parameter description, and stores the crawled structured data in the local database to form the original API library.

[0057] Step S2: Prompt the LLM to generate an API response schema and an API response summary according to the API response example.

[0058] Specifically, use LLM (such as GPT-4) to process the crawled API response examples to generate API response architecture and API response summary. The API response architecture is a structured description of the JSON response data returned by the API, which helps developers understand the data format returned by the API. The API response summary is a brief description of the API function, which helps developers quickly understand the purpose of the API. LLM is guided to generate this information through carefully designed prompts to ensure that the generated response architecture and summary are accurate and easy to understand.

[0059] Step S3: prompt LLM to generate API documentation based on the original API information, API response summary, and API response schema.

[0060] Specifically, LLM is used to generate structured API documents, which include API description, parameter list, and parameter description. By providing LLM with original API information, API response summary, and API response architecture, sufficient contextual information can be provided to the model to help it understand the API's functions, data formats, and usage scenarios. This information enables LLM to generate accurate and easy-to-understand API documents. The generated API documents provide a reliable foundation for subsequent task planning, error judgment, and repair, and for the system's automated orchestration.

[0061] Step S4: According to the API documentation and the tool description, prompt the LLM to generate an enhanced tool description.

[0062] Specifically, by providing API documentation and original tool descriptions to LLM, we can provide LLM with contextual information such as tool functions, API lists, and usage scenarios, helping it generate a more comprehensive tool description. The enhanced tool description not only contains basic information about the tool, but also details the APIs supported by the tool and their uses, ensuring that developers can quickly understand the tool's functions and usage methods. The generated enhanced tool description will be used for subsequent tool retrieval and task planning, improving the system's retrieval efficiency and the accuracy of task execution.

[0063] Step S5: vectorize the tool description to obtain a tool description vector.

[0064] Specifically, the enhanced tool description of each tool in the API library is vectorized using OpenAI's text-embedding-ada-002 model, which is converted into a numerical representation in a high-dimensional vector space. Through vectorization, the tool description is encoded into a numerical form, which is convenient for subsequent similarity calculation and tool retrieval.

[0065] Step S6: vectorize the input demand instruction to obtain a demand instruction vector.

[0066] Specifically, we use OpenAI's text-embedding-ada-002 model to vectorize the user input demand instructions and convert them into numerical representations in a high-dimensional vector space. The vectorized demand instruction vector can effectively capture the semantic information of user requirements and ensure that the most relevant tools and APIs can be quickly found when searching for tools.

[0067] Step S7: perform similarity matching between the requirement instruction vector and the tool description vector, and obtain the top N tools with the highest similarity and the API documents corresponding to the top N tools.

[0068] Specifically, the cosine similarity between the requirement instruction vector and each tool description vector is calculated to measure the semantic similarity between the two. The calculation formula of cosine similarity is as follows:

[0069]

[0070] in, represents the demand instruction vector, represents the tool description vector, represents the dot product of vectors, and Respectively represent the modulus length of the vector. Through this formula, the cosine similarity score of each tool description vector and the demand instruction vector is calculated, and the score range is , the higher the score, the more similar the semantics. After the calculation is completed, the system sorts the cosine similarity scores of all tools, selects the top N tools with the highest scores, and through the retrieved tools, all APIs under the tool can be obtained at the same time.

[0071] Step S8: According to the first N tools and the API documents corresponding to the first N tools, prompt the LLM to perform task planning to decompose the task into multiple subtasks, and allocate a main API and an alternative API to each subtask.

[0072] Specifically, by providing the first N tools and their corresponding API documents to LLM, sufficient contextual information can be provided to help LLM understand the functions of each tool and the applicable scenarios of the API. LLM generates a detailed task decomposition plan based on user needs and API documents to ensure that each subtask has a clear goal and API assignment. The primary API is the preferred API to complete the subtask, and the alternative API is the backup option when the primary API cannot complete the task.

[0073] Step S9: According to the main API, prompt the LLM to generate the first pseudo code for arranging the calling API.

[0074] Specifically, the first pseudocode is designed based on the main API in the mission planning and describes the logical flow of API calls, including steps such as parameter passing, data extraction, and result processing. In the first pseudocode, the system requires LLM to highlight the control flow and data flow of API calls to ensure that the logic is clear and easy to understand. The control flow describes the order and conditional judgment of API calls, and the data flow shows the transfer and processing of data between API calls. By providing LLM with the documentation of the main API and mission planning information, the system can provide sufficient context for the model to help it generate a logically correct and structured first pseudocode.

[0075] Step S10: According to the first pseudo code, prompt LLM to convert into Python code.

[0076] Specifically, by providing the first pseudocode and the main API documentation to the LLM, a clear context can be provided to the LLM to help it generate code that complies with Python syntax specifications. During the code generation process, the LLM will ensure that the logic of the control flow and data flow is consistent with the pseudocode.

[0077] Step S11: Determine whether the Python code is executed successfully. If the execution is unsuccessful, then prompt LLM to determine whether it is a design layer error or an implementation layer error based on the error message of the unsuccessful execution and the API documents corresponding to the first N tools;

[0078] Step S12: If a design layer error is prompted, the alternative API is used to prompt LLM to generate a second pseudo code for choreographing the API call for repair; if an implementation layer error is prompted, the main API document is used to prompt LLM to repair the Python code for the API choreographing call.

[0079] Exemplarily, the executable Python code may be run in an integrated development environment Pycharm to obtain a code running result, which is used to determine whether the Python code is executed successfully.

[0080] Specifically, if the code execution fails, the error information is captured, and combined with the API documents corresponding to the first N tools, the LLM is prompted to analyze the source of the error. The error may come from the design layer or the implementation layer: design layer errors are usually caused by improper API selection or unreasonable task planning, while implementation layer errors are usually caused by code logic or syntax problems. If LLM determines that the error comes from the design layer, the alternative API document is used to regenerate the second pseudocode, redesign the API call logic, and generate the repaired Python code based on the new pseudocode. If the error is judged to come from the implementation layer, the main API document is used to prompt LLM to repair the existing Python code and correct the logic or syntax problems. Through this two-layer debugging mechanism, the system can quickly locate and fix errors to ensure the accuracy and robustness of task execution.

[0081] In summary, the method of this embodiment has the following effects on solving the existing problems pointed out by the present invention:

[0082] (1) The present invention solves the problems of inconsistency, redundancy and incompleteness of existing API documents by building a unified and high-quality API knowledge base. Existing API documents usually have format differences and redundant information, which makes it difficult for LLM to accurately parse and utilize API functions. Through a bottom-up approach, a unified format API description is automatically generated based on LLM, and irrelevant information is removed, ensuring the simplicity and consistency of the document, thereby providing LLM with reliable and efficient API information, greatly improving the accuracy and execution efficiency of API orchestration tasks.

[0083] (2) The present invention effectively avoids the inefficiency of blind iteration of LLM in existing API orchestration. In traditional methods, LLM may repeatedly try failed API calls when arranging tasks, resulting in an increase in the time cost of task execution. However, the present invention uses a phased design and implementation strategy to generate a clear pseudocode blueprint through task decomposition and API allocation in the design phase to ensure the consistency and correctness of the task. In the implementation phase, the pseudocode is converted into executable code, further improving the efficiency and accuracy of task execution.

[0084] (3) The innovative two-layer debugging mechanism significantly improves the efficiency and accuracy of error location and repair. In the prior art, error location and repair are often difficult. The present invention divides the debugging process into design layer debugging and implementation layer debugging, which can re-evaluate task decomposition and API allocation in the design stage and correct syntax, logic or API call errors in the implementation stage. The two-layer debugging mechanism greatly reduces the time and cost of error repair, while improving the accuracy of error location and avoiding resource waste caused by errors.

[0085] (4) The phased design and implementation method of the present invention improves the transparency and controllability of the API calling process. By clearly dividing the design phase and the implementation phase, the system can ensure the clarity of API selection and calling logic in the design phase and effectively control code generation and execution in the implementation phase. This phased strategy not only improves the operability of the task, but also reduces the risk of failure caused by design or implementation problems, ensuring the stability of task execution.

[0086] (5) The present invention demonstrates greater flexibility and adaptability when dealing with complex API combinations and multi-task scheduling. Traditional methods often fail to effectively coordinate the call logic of each API when dealing with complex interactions between multiple APIs. The present invention, through its phased orchestration and two-layer debugging mechanism, can efficiently handle complex API combinations, ensure compatibility and collaborative work between multiple APIs, and thus cope with more complex orchestration tasks.

[0087] Example 2

[0088] See also Figure 3 , which is a schematic diagram of the structure of an API orchestration system based on a dual-layer debugging driver of LLM proposed in the second embodiment of the present application, and the system includes:

[0089] The API knowledge base construction module 100 is used to crawl original API information from the API platform, wherein the original API information includes an API response example and a tool description; prompt the LLM to generate an API response architecture and an API response summary according to the API response example; prompt the LLM to generate an API document according to the original API information, the API response summary and the API response architecture; prompt the LLM to generate an enhanced tool description according to the API document and the tool description; vectorize the tool description to obtain a tool description vector;

[0090] The API orchestration module 200 is used to vectorize the input demand instructions to obtain a demand instruction vector; perform similarity matching between the demand instruction vector and the tool description vector to obtain the top N tools with the highest similarity and the API documents corresponding to the top N tools; prompt the LLM to perform task planning based on the top N tools and the API documents corresponding to the top N tools to decompose the tasks into multiple subtasks, and assign a main API and an alternative API to each of the subtasks; based on the main API, prompt the LLM to generate a first pseudo code for orchestrating the API call; based on the first pseudo code, prompt the LLM to convert the code into Python code;

[0091] The two-layer debugging module 300 is used to determine whether the Python code is executed successfully. If the execution is unsuccessful, the LLM is prompted to determine whether it is a design layer error or an implementation layer error based on the error message of the unsuccessful execution and the API documents corresponding to the first N tools; if a design layer error is prompted, the alternative API is used to prompt the LLM to generate a second pseudo code for choreographing the API call for repair; if an implementation layer error is prompted, the main API document is used to prompt the LLM to repair the Python code for the API orchestration call.

[0092] In the embodiment of the present application, an API orchestration system based on a dual-layer debugging driver of LLM can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and the non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), etc., which is not specifically limited in the embodiment of the present application.

[0093] In the embodiment of the present application, an API arrangement system based on a dual-layer debugging driver of LLM can be a device having an operating system. The operating system can be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0094] The API arrangement system based on LLM dual-layer debugging driver provided in the embodiment of the present application can realize Figure 1 In the method embodiment, each process of implementing the API arrangement method of a dual-layer debugging driver based on LLM is not repeated here to avoid repetition.

[0095] Optionally, an embodiment of the present application also provides an electronic device, including a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, each process of the above-mentioned embodiment of the LLM-based dual-layer debugging-driven API orchestration method is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0096] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned AP method arrangement embodiment of a dual-layer debugging drive based on LLM is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0097] The processor is a processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0098] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0099] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0100] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An API arrangement method for a dual-layer debugging driver based on LLM, characterized in that: The method comprises: Crawling original API information from the API platform, wherein the original API information includes an API response example and a tool description; Prompt the LLM to generate an API response schema and an API response summary based on the API response example; Prompt the LLM to generate an API document according to the original API information, the API response summary, and the API response architecture; prompting LLM to generate an enhanced tool description according to the API document and the tool description; Vectorizing the tool description to obtain a tool description vector; Vectorize the input demand instruction to obtain a demand instruction vector; Performing similarity matching between the requirement instruction vector and the tool description vector, obtaining the top N tools with the highest similarity and API documents corresponding to the top N tools; According to the first N tools and the API documents corresponding to the first N tools, prompt the LLM to perform task planning to decompose the task into multiple subtasks, and allocate a main API and an alternative API to each of the subtasks; According to the main API, prompt the LLM to generate a first pseudo code for arranging the calling API; According to the first pseudo code, prompt LLM to convert into Python code; Determine whether the Python code is executed successfully. If the execution is unsuccessful, prompt LLM to determine whether it is a design layer error or an implementation layer error based on the error message of the unsuccessful execution and the API documents corresponding to the first N tools; If a design layer error is prompted, the alternative API is used to prompt LLM to generate a second pseudo code for choreographing the API call for repair; if an implementation layer error is prompted, the main API document is used to prompt LLM to repair the Python code for the API choreographing call.

2. According to the LLM-based dual-layer debugging driver API arrangement method according to claim 1, it is characterized in that: The steps to crawl raw API information from the API platform include: The API platform is the RapidAPI Hub platform, and the RESTful API is obtained from the RapidAPI Hub platform, and the RESTful API is used as the original API information.

3. According to the LLM-based dual-layer debugging driver API arrangement method of claim 1, it is characterized in that: In the step of crawling the original API information from the API platform, the original API information includes an API response example and a tool description, wherein the API response example is used to display the response data structure in JSON format returned after the API call, and the tool description includes an overall description of the original API information tool.

4. According to the LLM-based dual-layer debugging driver API arrangement method of claim 1, it is characterized in that: The LLM is the GPT4 model of OpenAI.

5. The API arrangement method of a dual-layer debugging driver based on LLM according to claim 1, characterized in that: The steps of vectorizing the input demand instruction to obtain the demand instruction vector include: The user input requirement instructions are vectorized using OpenAI’s text-embedding-ada-002 model to convert them into numerical representations in a high-dimensional vector space.

6. The API arrangement method of a dual-layer debugging driver based on LLM according to claim 1, characterized in that: The step of performing similarity matching between the demand instruction vector and the tool description vector comprises: The cosine similarity is used to represent the similarity match between the requirement instruction vector and the tool description vector, where the calculation formula of the cosine similarity is: ， In the formula, represents the demand instruction vector, represents the tool description vector, represents the dot product of vectors, and They represent the module lengths of the requirement instruction vector and the tool description vector respectively.

7. The API arrangement method of a dual-layer debugging driver based on LLM according to claim 1, characterized in that: The steps of determining whether the Python code is executed successfully include: Run the executable Python code in the integrated development environment Pycharm to obtain the code running result, which is used to show whether the executable Python code is executed successfully.

8. An API orchestration system based on LLM dual-layer debugging driver, characterized in that: The system comprises: An API knowledge base construction module is used to crawl original API information from an API platform, wherein the original API information includes an API response example and a tool description; prompt an LLM to generate an API response architecture and an API response summary according to the API response example; prompt an LLM to generate an API document according to the original API information, the API response summary and the API response architecture; prompt an LLM to generate an enhanced tool description according to the API document and the tool description; vectorize the tool description to obtain a tool description vector; The API orchestration module is used to vectorize the input demand instructions to obtain a demand instruction vector; perform similarity matching between the demand instruction vector and the tool description vector to obtain the top N tools with the highest similarity and the API documents corresponding to the top N tools; prompt the LLM to perform task planning based on the top N tools and the API documents corresponding to the top N tools to decompose the tasks into multiple subtasks, and assign a main API and an alternative API to each of the subtasks; based on the main API, prompt the LLM to generate a first pseudo code for orchestrating the API call; based on the first pseudo code, prompt the LLM to convert into Python code; A two-layer debugging module is used to determine whether the Python code is executed successfully. If the execution is unsuccessful, the LLM is prompted to determine whether it is a design layer error or an implementation layer error based on the error message of the unsuccessful execution and the API documents corresponding to the first N tools; if a design layer error is prompted, the alternative API is used to prompt the LLM to generate a second pseudo code for choreographing the call API for repair; if an implementation layer error is prompted, the main API document is used to prompt the LLM to repair the Python code for the API orchestration call.

9. An electronic device, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor. When the program or instruction is executed by the processor, the steps of the LLM-based dual-layer debugging-driven API arrangement method are implemented as described in any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the steps of the API arrangement method of a dual-layer debugging driver based on LLM are implemented as described in any one of claims 1-7.

Images