Knowledge-guided credible API recommendation method based on large language model

Abstract

The invention discloses a knowledge-guided credible API (Application Program Interface) recommendation method based on a large language model. The core idea of the method is to improve the credibility of LLM (Logical Language Model) in an API recommendation task by utilizing API knowledge. Specifically, in order to solve the problem that the LLM is not credible in API recommendation, the invention provides knowledge-guided data enhancement so as to improve the ability of the LLM to establish the relevance between a user specified type and an API sequence; proposing a knowledge-guided beam search to guide the LLM to generate an API that is real and that can be satisfied in call conditions; and proposing a knowledge-guided reordering optimization API sequence recommendation list, so that the API sequence matched with the interface parameter type is ranked in the top in the recommendation result. The method has the advantages that API recommendation can be realized under the constraint of the interface parameter type, and the credibility of a large model in an API recommendation task can be improved through high-quality priori knowledge in the API knowledge graph.

Description

Technical Field

[0001] The present invention relates to an application programming interface (API) recommendation method, and in particular to a knowledge-guided framework for LLM-based trustworthy API recommendation (KG4LLM). Background Art

[0002] As the degree of informatization in my country continues to deepen, how to improve the efficiency of software development and updating has become a hot research direction in the field of software engineering. In order to speed up software development efficiency and avoid reinventing the wheel, developers often use the APIs provided in libraries or frameworks to write code. Specifically, developers usually call a set of APIs in a specific order in coding to implement a specific function. This set of APIs called in a specific order is also called an API sequence. In recent years, the number of APIs has exploded, and even developers with rich development experience find it difficult to master the use of all APIs. Therefore, studying API recommendation methods, that is, recommending API sequences that can meet the specific functional requirements of developers, can make up for the lack of developers' knowledge of APIs.

[0003] In the process of team-based software development, different units or modules often require different developers to develop synchronously to improve efficiency. In order to be able to integrate smoothly after the development of each unit or module is completed, the parameter form of the interface often needs to be agreed in advance between different units or modules. Therefore, developers often need to implement specific functions under the constraints of predefined interface parameter types. In the API recommendation task for this scenario, the recommended API sequence not only needs to meet the functional requirements of the developer, but also needs to match the predefined interface parameter types. The present invention refers to this form of API recommendation task as API recommendation under specific interface parameter types (APIRIP). Figure 1 Demonstrates an example of APIRIP.

[0004] In recent years, Large Language Model (LLM) has demonstrated amazing capabilities in code generation and recommendation tasks. Therefore, implementing APIRIP tasks based on LLM is a feasible approach. However, the "hallucination" of LLM leads to unreliable problems in API recommendation tasks. Figure 2As shown in the figure, the untrustworthy issues of LLM in the API recommendation task can be divided into three categories: recommending fictitious APIs, recommending APIs whose calling conditions cannot be met, and the external parameter types on which the recommended API sequence depends are inconsistent with the predefined interface parameter types.

[0005] At present, no research work has been found on API recommendation under interface parameter type constraints, nor has any literature been found on using API knowledge to enhance the credibility of LLM in API recommendation tasks. Summary of the invention

[0006] The present invention provides a knowledge-guided trusted API recommendation method based on a large language model for API recommendation scenarios under interface parameter type constraints. The core idea of ​​the method is to use API knowledge to improve the credibility of LLM in API recommendation tasks. Specifically, in response to the untrustworthy problem of LLM in API recommendation (i.e., recommending fictitious APIs, recommending APIs whose calling conditions cannot be met, or the recommended API sequence does not match the interface parameter type), the present invention proposes knowledge-guided data enhancement to enhance the ability of LLM to establish the association between interface parameter types and API sequences; proposes knowledge-guided beam search to guide LLM to generate real APIs whose calling conditions can be met; proposes knowledge-guided reordering to optimize the API sequence recommendation list, so that the API sequence matching the interface parameter type is ranked higher in the recommendation results. The advantage of the present invention is that it can realize API recommendation under the constraint of interface parameter type, and can improve the credibility of the large model in the API recommendation task through high-quality prior knowledge in the API knowledge graph.

[0007] The objective of the present invention is achieved through the following technical solutions:

[0008] A knowledge-guided trusted API recommendation method based on a large language model includes the following steps:

[0009] Step 1: Build an API knowledge base for APIRIP tasks;

[0010] Step 2: Construct a training dataset for the APIRIP task;

[0011] Step 3: Fine-tune the large language model using knowledge-guided data augmentation;

[0012] Step 4: Generate a list of API sequence recommendations using knowledge-guided beam search;

[0013] Step 5: Optimize the API sequence recommendation list using knowledge-guided re-ranking.

[0014] Compared with the prior art, the present invention has the following advantages:

[0015] 1. The method provided by the present invention enables the recommended API sequence to meet the functional requirements of developers while conforming to specific interface parameter types.

[0016] 2. The KG4LLM framework proposed in this paper can improve the credibility of LLM in API recommendation tasks.

[0017] 3. The present invention can be applied to LLMs of various architectures and sizes and has good versatility. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is an instance of the APIRIP task.

[0019] Figure 2 It is an example of the untrustworthy problem of LLM in the API recommendation task.

[0020] Figure 3 It is the overall flow chart of the API recommendation framework KG4LLM proposed in the present invention.

[0021] Figure 4 It is an API ontology structure diagram for API recommendation tasks.

[0022] Figure 5 It is an instance diagram of API knowledge extraction.

[0023] Figure 6 It is an instance graph constructed by the API recommendation dataset.

[0024] Figure 7 It is an instance graph of knowledge-guided data augmentation.

[0025] Figure 8 It is a graph of knowledge-guided beam search and re-ranking instances. DETAILED DESCRIPTION

[0026] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.

[0027] The present invention provides a knowledge-guided trusted API recommendation method based on a large language model for API recommendation scenarios under interface parameter type constraints. First, in the data mining stage, an API knowledge graph is constructed by mining JDKAPI online documents, and an APIRIP dataset is constructed by mining Java open source projects. Secondly, in the large model fine-tuning stage, the dataset is expanded through knowledge-guided data enhancement, and the enhanced dataset is used to fine-tune the large model, thereby enhancing the ability of the large model in API recommendation. Finally, in the API recommendation stage, a knowledge-guided beam search is used based on the fine-tuned large model to generate an API sequence recommendation list, and knowledge-guided reordering is used to optimize the API sequence recommendation list. Figure 3 As shown, the method specifically comprises the following steps:

[0028] Step 1: Build an API knowledge base for APIRIP tasks.

[0029] Building an API knowledge base for APIRIP tasks is the basis for implementing subsequent knowledge-guided data enhancement, knowledge-guided beam search, and knowledge-guided reordering. The specific steps are as follows:

[0030] Step 11: Design the API ontology model. Figure 4 As shown in the figure, the concepts in the API ontology model include: basic data type "basic type", class "class", interface "interface", enumeration "enum", record "record", API or method "API", package "Package" and member "member". The relationships include: inheritance "extend", implementation "implement", inclusion "contain", type "type <dim>"(dim represents the dimension of the variable type), return relation"return <dim>"(dim represents the dimension of the returned value), parameter"<idx,dim> "(idx represents the serial number of the parameter, dim represents the dimension of the parameter). Attributes include: the modifier "field" of the API, method or member. Values ​​include: public "public", private "private", protected "protected", static "static".

[0031] Step 12: Use the requests tool to recursively obtain all API documentation pages from the main page of the Java JDK official online documentation (https: / / docs.oracle.com / en / java / javase / 20 / docs / api / index.html) and save these documentation pages as ".html" files.

[0032] Step 13: Use the BeautifulSoup tool to parse each ".html" file, extract API entities, relationships, attributes, and values ​​from the corresponding fields according to the API ontology designed in step 11, and obtain a set of triples consisting of "entity-relationship-entity" or "entity-attribute-value" triples.

[0033] Step 14: Import all "entity-relationship-entity" and "entity-attribute-value" triples into the Neo4j graph database to obtain the API knowledge graph in graph form.

[0034] Step 2: Build a training dataset for the APIRIP task.

[0035] Construct a training dataset for the APIRIP task to provide training data for the subsequent LLM fine-tuning process. The specific steps are as follows:

[0036] Step 21: Retrieve and download all Java projects with at least one star from GitHub since 2008, and execute steps 22 to 24 for each Java project that meets the requirements.

[0037] Step 22: Find all methods in the Java project that meet the following conditions: (1) have functional annotations; (2) only call JDK API methods.

[0038] Step 23: For the i-th method that meets the conditions described in step 22, add the method's functional annotation doc i As a functional description, the method's parameter list is used as the interface parameter type list param i , and parse out the API sequence apiseq according to the API sequence parsing rules defined by DeepAPI i .

[0039] Step 24: Convert the triple [doc i ,param i ,apiseq i ] is added to the APIRIP dataset as a training data.

[0040] Step 3: Fine-tune the large language model using knowledge-guided data augmentation.

[0041] Generate training data containing more "interface parameter type-API sequence" combinations through knowledge-guided data enhancement to improve LLM's ability to establish a mapping relationship between interface parameter types and API sequences, thereby alleviating the problem of mismatch between API sequences recommended by LLM and interface parameter types. Subsequently, use the enhanced data set to fine-tune LLM. The specific steps are as follows:

[0042] Step 31: Execute steps 32 to 318 for each piece of data in the original APIRIP data set to obtain corresponding enhanced data, and mix the obtained enhanced data with the original data set to obtain an enhanced data set. After the data enhancement operation is performed on each piece of data, execute step 319.

[0043] Step 32: Initialization phase of data enhancement: Initialize the API index i of the current traversal to 0; cache buff of the currently available variable types t It is a set consisting of "base type" type entities, variable types of all static public members, all entities contained in the "java.lang" package, and all entities that support the packaging mechanism; the currently searched API sequence apiseq cur It is an empty list; the specified parameter param currently searched cur It is an empty list; the currently found comment, specified variable type, and API sequence triple list answers is an empty list.

[0044] Step 33: Input the original data for data enhancement, that is, a triple containing function description doc, API sequence apiseq and interface parameter type param; record the currently traversed API index as i; record the currently available variable type cache as buff t ; The enhanced API sequence is apiseq cur ;Remember that the enhanced interface parameter type is param cur ; The enhanced comments, specified variable types, and API sequence triple list are denoted as answers; and the API knowledge base is denoted as G.

[0045] Step 34: Determine whether the size of answers exceeds the specified threshold N. If it exceeds, return answers, otherwise execute step 35. In the present invention, the function of the threshold N is to control the execution time of the data enhancement algorithm to avoid the execution time of the data enhancement algorithm being too long. The value of the threshold N can be selected to be around 1000.

[0046] Step 35: Determine whether i is equal to the length of apiseq. If yes, execute step 36; otherwise, execute step 39.

[0047] Step 36: Determine param cur Is it exactly the same as param? If so, it means duplicate data is generated, so it is discarded and answers is returned directly. Otherwise, execute step 37.

[0048] Step 37: Calculate apiseq using the following formula cur score cur :

[0049]

[0050] Where len(·) refers to the length of the API sequence; return G (·) refers to retrieving the return value type of a given API in the API knowledge base G; requires G (·) refers to retrieving the set of variable types required by a given single API or API sequence in the API knowledge base G. The required variable types of an API include all variable types in the formal parameter list of the API. In addition, if the API is not a static method, the required variable types of the API also include the class, interface, enumeration or record to which the API belongs.

[0051] Step 38: Transform the triple [score cur ,param cur , apiseq cur ] to answers and return answers.

[0052] Step 39: Traverse each overloaded API of apiseq[i] in the API knowledge base and execute steps 310 to 317.

[0053] Step 310: require cur Equals requires G (API).

[0054] Step 311: Protect the scene and set param cur 、apiseq cur , buff t and i are backed up to the cache.

[0055] Step 312: Add the content in require cur In but not in buff t Add the variable type in param cur middle.

[0056] Step 313: Add the return value of the API, as well as the classes, interfaces, enumerations or records that it directly or indirectly inherits or implements, to buff t middle.

[0057] Step 314: Add API to apiseq cur middle.

[0058] Step 315: Let i=i+1, and get answers by recursively executing step 33.

[0059] Step 316: Restore the scene and set param cur 、apiseq cur , buff t and i are restored to the backup in the cache.

[0060] Step 317: If all overloaded forms of the API have been traversed, return to answers, otherwise return to step 39 to continue traversing other overloaded forms of the API.

[0061] Step 318: Sort all [score, param, apiseq] triplets in answers in ascending order according to the size of score, and output the top K [param, apiseq] data pairs, and form K new data with doc respectively. In the present invention, K is mainly used to control the number of new enhanced data output by the data enhancement algorithm to avoid generating too much enhanced data, and the value of K can be set to 5.

[0062] Step 319: Fine-tune LLM using the augmented dataset: For training data in the form of [doc, apiseq, param], replace doc and param with " <sep>" is concatenated as a separator as the model input, apiseq is taken as the expected output, and the LLM is fine-tuned directly using the fine-tuning method provided by LLM itself.

[0063] Step 4: Generate a list of API sequence recommendations using knowledge-guided beam search.

[0064] Knowledge-guided beam search is used to guide LLM to generate real APIs whose calling conditions can be met during the beam search process. The specific steps are as follows:

[0065] Step 41: Obtain a list param of interface parameter types by parsing the interface definition in the user development environment.

[0066] Step 42: Enter the text description doc of the user's desired function.

[0067] Step 43: Replace doc and param with " <sep>" is used as a separator to construct the input.

[0068] Step 44: Initialization phase of beam search: Initialize the beam Beam to an empty list for storing generated API sequences, initialize the result buffer buff to an empty list for storing generated API sequences, and set the beam size (beam size) to bs.

[0069] Step 45: Construct the initial "score-API sequence" tuple [0, " <sos>”] and add it to Beam.

[0070] Step 46: Loop through steps 47 to 424 for Beam until the maximum number of loops T is reached or the number of tuples in results is equal to bs, and then execute step 425.

[0071] Step 47: Initialize Beam′ to an empty list, traverse each tuple [s, apiseq] in Beam, and execute steps 48 to 424 for it.

[0072] Step 48: Determine whether apiseq ends with a " <eos>" is the suffix, if so, add [s, apiseq] to results and return to step 47 to continue traversing other tuples, otherwise, execute step 49.

[0073] Step 49: Use LLM to predict the probability distribution P of the next token to be generated for input and apiseq. The distribution describes the probability value of each lexical unit (token) in the vocabulary.

[0074] Step 410: vocabulary constraint process: initialize the candidate token set tokens to an empty set, and initialize the variable type buffer buff to a set of initial available variable types. The set of initial available variable types includes basic data types, variable types of all static public members in JDK, all variable types in the "java.lang" package, and all variable types that support the automatic boxing mechanism.

[0075] Step 411: Add the return values ​​of all generated APIs in apiseq to buff.

[0076] Step 412: Set the Boolean flag space of the space character to False.

[0077] Step 413: Determine whether apiseq is empty, if so, execute step 414, otherwise execute step 415.

[0078] Step 414: Initialize the API prefix prefix to an empty string, and execute step 418.

[0079] Step 415: Determine whether the last API in apiseq has been generated. If so, execute step 416; otherwise, execute step 417.

[0080] Step 416: Initialize the API prefix prefix to an empty string, set space to True, and execute step 418.

[0081] Step 417: Initialize the API prefix prefix to the last API in apiseq that has not been generated yet, and execute step 418.

[0082] Step 418: Traverse each API in the knowledge base, and execute steps 419 to 421 for each API whose required variable type exists in buff and whose prefix is ​​prefix. The required variable type of the API includes all variable types in the parameter list of the API. In addition, if the API is not a static method, the required variable type of the API also includes the class, interface, enumeration or record to which the API belongs.

[0083] Step 419: If space is True, add a space character before API and execute step 420; otherwise, execute step 420 directly.

[0084] Step 420: Remove the prefix prefix of the API.

[0085] Step 421: Use the LLM tokenizer to perform token segmentation operation and add the first token after token segmentation to tokens.

[0086] Step 422: If space is True, the end marker symbol of the process will be generated. <eos>" is added to tokens.

[0087] Step 423: After concatenating each token in tokens to apiseq, a new candidate API sequence apiseq′ is obtained. The score s′ is calculated for apiseq′ using the following formula, and [s′, apiseq′] is added to Beam′:

[0088]

[0089] Among them, n is the length of apiseq, P(token) is the probability value of the token output by the large language model, and s is the score of apiseq.

[0090] Step 424: Sort all bigrams in Beam′ from large to small according to the scores, then only retain the bigrams with the top bs in Beam′, and assign Beam′ to Beam.

[0091] Step 425: Output Beam as a recommendation list of API sequences.

[0092] Step 5: Optimize the API sequence recommendation list using knowledge-guided re-ranking.

[0093] After obtaining the recommended list of API sequences through knowledge-guided bundle search, the API sequences matching the interface parameter types in the recommended list are ranked higher using knowledge-guided reordering. The specific steps are as follows:

[0094] Step 51: Traverse each tuple in the results list obtained in the knowledge-guided beam search phase, each tuple contains a score s and an API sequence apiseq.

[0095] Step 52: Initialize the external dependent variable type set rely to an empty set.

[0096] Step 53: Set the variable type buffer buff to an empty set.

[0097] Step 54: traverse each API in apiseq in turn, and execute steps 55 to 58 for them, until all APIs in apiseq are traversed, and then execute step 59.

[0098] Step 55: traverse each required variable type type of the API, and execute steps 56 to 57 for type, until each required variable type type is traversed, and then execute step 58. The required variable type of the API includes all variable types in the parameter list of the API. In addition, if the API is not a static method, the required variable type of the API also includes the class, interface, enumeration or record to which the API belongs.

[0099] Step 56: Determine whether type exists in buff, if yes, execute step 58, otherwise execute step 57.

[0100] Step 57: Add type to rely.

[0101] Step 58: Add the return value of the API to buff.

[0102] Step 59: Traverse each variable type in rely, retrieve all parent classes that are directly or indirectly inherited or implemented by the variable type from the knowledge base, and add these parent classes to rely.

[0103] Step 510: Determine whether there is at least one variable type in param that does not exist in rely. If so, multiply the score s in the current tuple by the multiple r, otherwise, keep it unchanged.

[0104] Step 511: re-sort all the tuples in results in descending order of scores.

[0105] Step 512: Output the apiseq in the two-tuple in results in order as a recommended list of API sequences.

[0106] Embodiment 1:

[0107] like Figure 5 As shown in the figure, the API knowledge extraction process is introduced by taking the "java.io" package as an example. First, the "Class Description" field of the API document of the "java.io" package is parsed, and it can be seen that the package contains the "BufferedInputStream" class, the "Closeable" interface, and the "InputStream" class. Subsequently, a "contain" relationship is established between the package and the above-mentioned classes or interfaces.

[0108] Secondly, parse the API documents of the above classes or interfaces respectively. Take the "BufferedInputStream" class as an example to introduce the process. First, get the address of the API document corresponding to the "BufferedInputStream" class from the Apiece document of the "java.io" package. Secondly, parse the "Implements" field in the document to get the interface "Closeable" implemented by the class; parse the "Field Details" field to get the member variable "buf" contained in the class; parse the "Constructor Details" field to get the constructor "BufferedInputStream(InputStream)" of the class; parse the "Method Details" field to get the API method "read()" defined in the class. Establish an "implement" relationship between the "BufferedInputStream" class and the "Closeable" interface, and establish a "contain" relationship between the "BufferedInputStream" class and the "buf" member variable, the "BufferedInputStream(InputStream)" constructor, and the "read()" method.

[0109] Then, parse the "buf" member variable. The variable type of this member is "byte[]" and the access right is "protected". Therefore, add "type" between this member and the "byte" data type. <1> "relationship" (1 represents the dimension of the parameter), and set the "field" attribute of the member to "protected".

[0110] Finally, parse the "BufferedInputStream(InputStream)" constructor and the "read" method. The parsing process of the "read" and "BufferedInputStream(InputStream)" methods is similar. The present invention takes the "BufferedInputStream(InputStream)" constructor as an example to introduce the process. The parameter table of this constructor contains only one parameter "InputStream", and the access right of this constructor is "public". Therefore, add a "parameter<0,0>" relationship between this constructor and the "InputStream" class, and set the "field" attribute of this constructor to "public". Since the return value of a class's constructor can be regarded as the class itself, add a "return" relationship between this constructor and the "BufferedInputStream" class. <0> "relation.

[0111] Embodiment 2:

[0112] like Figure 6 As shown in the figure, the function copyLarge is used as an example to introduce the construction process of training data. Each piece of training data in the training data set contains three parts, namely, function description, interface parameter type and API sequence. First, the function annotation of the method is used as the function description. Secondly, the parameter types contained in the parameter table of the method are "InputStream", "OutputStream" and "byte[]", and these parameters are used as interface parameter types. Finally, all APIs called in the method body are sorted in the order of appearance, that is, "InputStream.read(byte[])→OutputStream.write(byte[],int,int)", and these sorted APIs are used as API sequences.

[0113] Embodiment 3:

[0114] by Figure 7 This paper introduces knowledge-guided data enhancement by taking the training data in as an example. First, clear the interface parameter type of the original data, that is, set it to "void", and change "File.listFiles(FileFilter)" in the API sequence to "File.listFiles(FilenameFilter)". Then, analyze the external parameters that the changed API sequence depends on. Among them, the external parameter that "File.isDirectory()" depends on is "File", and the external parameters that "File.listFiles(FilenameFilter)" depends on are "File" and "FilenameFilter". The parameter "File" that "File.delete()" depends on can be provided by the return value of "File.listFiles(FilenameFilter)", so the API does not depend on other external parameters. Finally, add "File" and "FilenameFilter" to the interface parameter type.

[0115] Embodiment 4:

[0116] by Figure 8 This paper introduces knowledge-guided beam search and knowledge-guided reordering as examples. To enhance the readability of the example, the API parameter table is omitted in this example. The specific settings of this example are: the beam size is 2; the knowledge base contains three API methods: "read.read", "file.file" and "file.read"; the output vocabulary of the large model contains "read", "file", "." and " <e>"Four different tokens; the user query is "Read a file."; the interface parameter type is "file". The initialization bundle list is empty and the initial API sequence is" <s>" and the initial score 0 are added to the bundle list. First, a temporary bundle list is created and initialized to an empty list. Then, the user query and each API sequence in the bundle list are used as inputs of the large model. At this time, the large model outputs the probability value of each token in the vocabulary for the current input query and API sequence. Secondly, a candidate token set is constructed for the current API sequence by querying the knowledge base (such as the initial API sequence " <s>" includes "read" and "file"), and removes tokens that are not in the candidate token set (such as for the initial API sequence " <s>", tokens that do not exist in the candidate token set include ". " and " <e>”) is set to 0. Then, each token in the vocabulary is concatenated to the current API sequence to obtain a new set of API sequences. The score of each API sequence is calculated and added to the temporary bundle list. When all API sequences in the bundle list are traversed, the bundle list is replaced with the temporary bundle list. The data in the temporary bundle list is sorted in descending order according to the scores of the API sequences, and only the top 2 API sequences are retained. The above process is repeated until the fourth cycle, at which time all API sequences in the bundle list are marked with " <e>" ends, and the beam search process ends.

[0117] Subsequently, knowledge-guided reordering is performed. Specifically, it is determined in turn whether the required variable type of each API sequence in the recommendation list is consistent with the parameter variable type defined in the interface. Specifically, "read.read <e>"The required variable type is "read", which is inconsistent with the parameter variable type "file" defined in the interface; "file.read <e> The required variable type of "file" is consistent with the parameter variable type "file" defined in the interface. If they are consistent, the score of the API sequence is multiplied by 2, otherwise it remains unchanged. Then, the list is re-sorted to obtain a new recommendation list.< / e> < / e> < / e> < / e> < / s> < / s> < / s> < / e> < / eos> < / eos> < / sos> < / sep> < / sep> < / dim> < / dim>

Claims

1. A knowledge-guided trusted API recommendation method based on a large language model, characterized by The method comprises the following steps: Step 1: Build an API knowledge base for APIRIP tasks. The specific steps are as follows: Step 11: Design the API ontology model. The concepts in the API ontology model include: basic data type "basic type", class "class", interface "interface", enumeration "enum", record "record", API or method "API", package "Package" and member "member"; the relationships include: inheritance "extend", implementation "implement", inclusion "contain", type "type <dim>"Return relationship" <dim> ", parameter<idx,dim> ", idx represents the serial number of the parameter, dim represents the dimension; attributes include: API, method or member modifier "field"; values ​​include: public "public", private "private", protected "protected", static "static";< / dim> < / dim> Step 12: Use the requests tool to recursively obtain all API documentation pages from the main page of the Java JDK official online documentation, and save these documentation pages as ".html" files; Step 13: Use the BeautifulSoup tool to parse each ".html" file, extract API entities, relationships, attributes, and values ​​from the corresponding fields according to the API ontology designed in step 11, and obtain a set of triples consisting of "entity-relationship-entity" or "entity-attribute-value" triples; Step 14: Import all "entity-relationship-entity" and "entity-attribute-value" triples into the Neo4j graph database to obtain the API knowledge graph in graph form; Step 2: Build a training dataset for the APIRIP task. The specific steps are as follows: Step 21: Retrieve and download all Java projects with at least one star in GitHub, and execute steps 22 to 24 for each Java project that meets the conditions; Step 22: Find all methods in the Java project that meet the following conditions: (1) have functional annotations; (2) only call JDK API methods; Step 23: For the i-th method that meets the conditions described in step 22, add the method's functional annotation doc i As a functional description, the method's parameter list is used as the interface parameter type list param i , and parse out the API sequence apiseq according to the API sequence parsing rules defined by DeepAPI i ; Step 24: Convert the triple [doc i ,param i , apiseq i ] is added to the APIRIP dataset as a training data; Step 3: Fine-tune the large language model using knowledge-guided data augmentation. The specific steps are as follows: Step 31: Execute steps 32 to 318 for each piece of data in the original APIRIP data set to obtain corresponding enhanced data, and mix the obtained enhanced data with the original data set to obtain an enhanced data set. After the data enhancement operation is performed on each piece of data, execute step 319; Step 32: Initialization phase of data enhancement: Initialize the API index i of the current traversal to 0; cache buff of the currently available variable types t It is a set consisting of "base type" type entities, variable types of all static public members, all entities contained in the "java.lang" package, and all entities that support the packaging mechanism; the currently searched API sequence apiseq cur It is an empty list; the specified parameter param currently searched cur It is an empty list; the currently found comment, specified variable type, and API sequence triple list answers is an empty list; Step 33: Input the original data for data enhancement, that is, a triple containing function description doc, API sequence apiseq and interface parameter type param; record the currently traversed API index as i; record the currently available variable type cache as buff t ; The enhanced API sequence is apiseq cur ;Remember that the enhanced interface parameter type is param cur ; Let the enhanced comments, specified variable types, and API sequence triple list be called answers; let the API knowledge base be called G; Step 34: Determine whether the size of answers exceeds the specified threshold N. If it exceeds, return answers, otherwise execute step 35; Step 35: Determine whether i is equal to the length of apiseq. If yes, execute step 36; otherwise, execute step 39. Step 36: Determine param cur Is it exactly the same as param? If so, it means duplicate data is generated, so it is discarded and answers is returned directly. Otherwise, go to step 37. Step 37: Calculate apiseq using the following formula cur score cur : Where len(·) refers to the length of the API sequence; return G (·) refers to retrieving the return value type of a given API in the API knowledge base G; requires G (·) refers to retrieving a given single API or API sequence requirement variable type set in the API knowledge base G; Step 38: Transform the triple [score cur ,param cur , apiseq cur ] is added to answers and answers is returned; Step 39: traverse each overloaded API of apiseq[i] in the API knowledge base and execute steps 310 to 317; Step 310: require cur Equals requires G (API); Step 311: Protect the scene and set param cur 、apiseq cur , buff t and i are backed up to the cache; Step 312: Add the content in require cur In but not in buff t Add the variable type in param cur middle; Step 313: Add the return value of the API, as well as the classes, interfaces, enumerations or records that it directly or indirectly inherits or implements, to buff t middle; Step 314: Add API to apiseq cur middle; Step 315: Let i=i+1, and get answers by recursively executing step 33; Step 316: Restore the scene and set param cur 、apiseq cur , buff t and i are restored to the backup in the cache; Step 317: If all overloaded forms of the API have been traversed, then return to answers, otherwise return to step 39 to continue traversing other overloaded forms of the API; Step 318: Sort all [score, param, apiseq] triplets in answers in ascending order according to the size of the score, and output the top K [param, apiseq] data pairs, and form K new data with doc respectively; Step 319: Fine-tune LLM using the augmented dataset: For training data in the form of [doc, apiseq, param], replace doc and param with " <sep> " as a separator and then concatenated as the model input, apiseq is taken as the expected output, and the LLM is fine-tuned directly using the fine-tuning method provided by LLM itself;< / sep> Step 4: Generate a list of API sequence recommendations using knowledge-guided beam search. The specific steps are as follows: Step 41: Obtain a list param of interface parameter types by parsing the interface definition in the user development environment; Step 42: Input the text description doc of the function expected by the user; Step 43: Replace doc and param with " <sep> " is used as a separator to construct the input;< / sep> Step 44: Initialization phase of beam search: initialize the beam Beam to an empty list for storing generated API sequences, initialize the result buffer buff to an empty list for storing generated API sequences, and set the beam size to bs; Step 45: Construct the initial "score-API sequence" tuple [0, " <sos> ”] and add it to Beam;< / sos> Step 46: looping through steps 47 to 424 for Beam until the maximum number of loops T is reached or the number of tuples in results is equal to bs, and then executing step 425; Step 47: Initialize Beam′ to an empty list, traverse each tuple [s, apiseq] in Beam, and execute steps 48 to 424 for it; Step 48: Determine whether apiseq ends with a " <eos> " is the suffix, if so, add [s, apiseq] to results and return to step 47 to continue traversing other tuples, otherwise, execute step 49;< / eos> Step 49: Use LLM to predict the probability distribution P of the next token to be generated for input and apiseq. The distribution describes the probability value of each lexical unit token in the vocabulary. Step 410: vocabulary constraint process: initialize the candidate token set tokens to an empty set, and initialize the variable type buffer buff to a set of initial available variable types; Step 411: Add the return values ​​of all generated APIs in apiseq to buff; Step 412: Set the Boolean flag space of the space character to False; Step 413: Determine whether apiseq is empty, if so, execute step 414, otherwise execute step 415; Step 414: Initialize the API prefix prefix to an empty string, and execute step 418; Step 415: Determine whether the last API in apiseq has been generated, if yes, execute step 416, otherwise execute step 417; Step 416: Initialize the API prefix prefix to an empty string, set space to True, and execute step 418; Step 417: Initialize the API prefix prefix to the last API in apiseq that has not been generated yet, and execute step 418; Step 418: traverse each API in the knowledge base, and execute steps 419 to 421 for each API whose required variable type exists in buff and whose prefix is ​​prefix; Step 419: If space is True, add a space character before API and execute step 420, otherwise execute step 420 directly; Step 420: remove the prefix prefix of the API; Step 421: Use the LLM tokenizer to perform token segmentation operation on it, and add the first token after token segmentation to tokens; Step 422: If space is True, the end marker symbol of the process will be generated. <eos> " is added to tokens;< / eos> Step 423: After concatenating each token in tokens to apiseq, a new candidate API sequence apiseq′ is obtained. The score s′ is calculated for apiseq′ using the following formula, and [s′, apiseq′] is added to Beam′: Where n is the length of apiseq, P(token) is the probability value of the token output by the large language model, and s is the score of apiseq; Step 424: sort all the bigrams in Beam′ from large to small according to the scores, then only retain the bigrams with the top bs in Beam′, and assign Beam′ to Beam; Step 425: Output Beam as a recommendation list of API sequences; Step 5: Use knowledge-guided reordering to optimize the API sequence recommendation list. The specific steps are as follows: Step 51: traverse each tuple in the results list obtained in the knowledge-guided beam search phase, each tuple contains a score s and an API sequence apiseq; Step 52: Initialize the external dependent variable type set rely to an empty set; Step 53: Set the variable type buffer buff to an empty set; Step 54: traverse each API in apiseq in turn, and execute steps 55 to 58 for each API, until all APIs in apiseq are traversed, and then execute step 59; Step 55: traverse each required variable type type of the API, and execute steps 56 to 57 for type, until each required variable type type is traversed, and then execute step 58; Step 56: Determine whether type exists in buff, if yes, execute step 58, otherwise execute step 57; Step 57: Add type to rely; Step 58: Add the return value of the API to buff; Step 59: traverse each variable type in rely, retrieve all parent classes that are directly or indirectly inherited or implemented by the variable type from the knowledge base, and add these parent classes to rely; Step 510: Determine whether there is at least one variable type in param that does not exist in rely. If so, multiply the score s in the current tuple by the multiple r, otherwise, keep it unchanged. Step 511: re-sort all the 2-tuples in results in descending order of scores; Step 512: Output the apiseq in the two-tuple in results in order as a recommended list of API sequences.

2. The knowledge-guided trusted API recommendation method based on a large language model according to claim 1 is characterized in that In step 37, the required variable types of the API include all variable types in the parameter list of the API. If the API is not a static method, the required variable types of the API also include the class, interface, enumeration or record to which the API belongs.

3. The knowledge-guided trusted API recommendation method based on a large language model according to claim 1 is characterized in that In step 410, the set of initially available variable types includes basic data types, variable types of all static public members in JDK, all variable types in the "java.lang" package, and all variable types that support the automatic boxing mechanism.

4. The knowledge-guided trusted API recommendation method based on a large language model according to claim 1 is characterized in that In step 418, the required variable types of the API include all variable types in the parameter list of the API. If the API is not a static method, the required variable types of the API also include the class, interface, enumeration or record to which the API belongs.

5. The knowledge-guided trusted API recommendation method based on a large language model according to claim 1 is characterized in that In step 55, the required variable types of the API include all variable types in the parameter list of the API. If the API is not a static method, the required variable types of the API also include the class, interface, enumeration or record to which the API belongs.

Images