Method and system for automatically generating reusable APIs based on code snippets

Abstract

This invention discloses a method and system for automatically generating reusable APIs based on code snippets. Specifically, it includes: first, acquiring webpage data to be processed and extracting information contained in the webpage data, including the question title, several Q&A posts corresponding to the question title, and code snippets corresponding to each Q&A post; constructing prompt words for a large language model, including role-specific prompt words, thought chain reasoning prompt words, and few-shot context learning prompt words; inputting the constructed prompt words and each code snippet into the large language model, outputting an API method with a text description; and using regular expressions to clean the API method with the text description, removing the text description from the API method to obtain the API method corresponding to each code snippet. Compared with existing technologies, the method proposed in this invention can generate more accurate APIs.

Description

Technical Field

[0001] This invention belongs to the fields of code generation, large language models, intelligent software engineering, and software repository mining, and particularly relates to a method and system for automatically generating reusable APIs based on code snippets. Background Technology

[0002] Stack Overflow is one of the most popular software Q&A websites today, aiming to help developers solve various technical problems. Although code snippets posted on Stack Overflow are frequently used by developers through copying, pasting, and refactoring, they are usually not compileable or directly executable because they are typically for illustrative purposes. To reuse code snippets from Stack Overflow, users usually convert them into Application Program Interfaces (APIs). For Java code snippets, this process typically requires five steps: (1) restoring the import statement; (2) creating the method name; (3) creating the parameter list; (4) creating the return statement; and (5) creating the throws statement. In addition, other syntax problems that may exist in the code snippet, such as missing variable declarations, must be fixed. The above process is time-consuming, laborious, and error-prone, which greatly reduces the efficiency of developers reusing Stack Overflow code snippets for daily development.

[0003] To facilitate developers' rapid reuse of code snippets from Stack Overflow, Terragni et al. proposed a tool called APIzator in existing technical literature. This tool can automatically API-ize Java code snippets. Specifically, based on 135 Stack Overflow Java code snippets and their corresponding API data pairs in their GitHub projects, they summarized four rules to create parameter lists and return statements for Java code snippets. They used part-of-speech tagging to extract predicates and objects from Stack Overflow question titles to create method names. Finally, they relied on external tools CSNIPPEX and BAKER to restore import statements and fix syntax issues such as missing variable declarations.

[0004] While APIzator can automatically perform APIzation on Java code snippets, it still suffers from drawbacks such as low accuracy in parameter lists and return statements, inaccurate method names, and poor generalization. The reasons for these shortcomings are as follows: First, manually generated rules limit APIzator to code snippets that conform to those rules. For code snippets outside the rules, APIzator struggles to accurately create parameter lists or return statements, resulting in low accuracy. This low accuracy also reflects the difficulty in finding universally applicable manually generated rules. Second, the intent of Stack Overflow code snippets is usually reflected in the snippet itself or in the complete context of the question title, question post, and answer post. Part-of-speech tagging only uses the question title to reflect the code snippet's intent, losing contextual information such as question and answer posts. Therefore, the created method names are often inaccurate or even meaningless. Finally, manually generated rules and external tools like CSNIPPEX and BAKER are only applicable to Java code snippets and cannot be extended to other programming languages ​​such as Python. Furthermore, the functionality of external tools determines the types of syntax errors APIzator can fix, further limiting its generalization ability. Summary of the Invention

[0005] Automatic APIification of code snippets is a difficult task, specifically because:

[0006] Extracting suitable variables or literals from code segments as input parameters or return statements for an API is difficult. For variables lacking declarations, it's necessary not only to determine if they can be parameters or return statements, but also to infer their parameter or return statement types. For literals, they cannot be simply extracted as API parameters because the generated API must not only be reusable but also address the issues raised in this post. Finally, the parts of the code segment extracted as parameters or return statements need modification. Manual rules cannot simultaneously solve all these problems; therefore, how to accurately and efficiently create parameter lists and return statements for APIs is the first technical problem this invention aims to solve.

[0007] Creating accurate method names for APIs that describe their functionality is challenging. Since Stack Overflow answer snippets are typically created by questioners and answerers through a question-and-answer process, the complete context of the answer snippet includes the question title, the question post, and the answer post. When the functionality of a snippet cannot be determined from its complete context, it must be summarized from the snippet itself. This means that method names must either summarize the complete context or the functionality of the snippet itself; the former requires strong text summarization skills, while the latter requires strong code comprehension skills. Part-of-speech tagging technology can only tag parts of speech and is difficult to summarize long texts. If applied only to question titles, it loses a significant amount of contextual information. Therefore, how to generate accurate method names using code snippets and their complete context is the second technical problem this invention aims to solve.

[0008] Generating APIs for code segments in various programming languages ​​is challenging. Each programming language has a unique syntax style, and a model suitable for one language may not be suitable for another. For example, Java requires API parameter lists to specify parameter types, while Python does not. Furthermore, each programming language involves multiple types of syntax errors, and fixing all types of syntax errors is difficult. Current manual rules and external packages only apply to Java, and the functionality of these packages limits the types of syntax errors that can be fixed. Therefore, how to implement a system that supports multiple programming languages ​​and can fix various types of syntax errors is the third technical problem that this invention aims to solve.

[0009] The purpose of this invention is to solve the above-mentioned problems existing in the prior art and to provide a method and system for automatically generating reusable APIs based on code snippets.

[0010] The specific technical solution adopted in this invention is as follows:

[0011] In a first aspect, the present invention provides a method for automatically generating reusable APIs based on code snippets, comprising the following steps:

[0012] S1. Obtain the webpage data to be processed and extract the information contained in the webpage data, the information including the question title, several question and answer posts corresponding to the question title, and code snippets corresponding to each question and answer post;

[0013] S2. Construct prompt words for a large language model, wherein the prompt words include role-specific prompt words, thought chain reasoning prompt words, and few-shot context learning prompt words;

[0014] S3. Input the constructed prompts and each code snippet into the large language model, and output an API method with a text description;

[0015] S4. Use regular expressions to clean the API methods with text descriptions, remove the text descriptions from the API methods with text descriptions, and obtain the API method corresponding to each code snippet.

[0016] Preferably, the large language model adopts the GPT-3.5-turbo model.

[0017] Preferably, the role-assigning prompt includes assigning roles to the large language model and informing the large language model of the task to be solved.

[0018] Preferably, the thought chain reasoning prompts include thought chain reasoning that uses the thought process of professional developers as a large language model.

[0019] Preferably, the few-shot context learning prompts include sample input, sample output, test input, and format constraints. The sample input and sample output are used to inform the large language model of the expected output format; the test input indicates that the large language model is currently converting the code segment and its context; and the format constraints are used to inform the large language model of the input and output formats for the API-based task.

[0020] Secondly, this invention provides a system for automatically generating reusable APIs based on code snippets, comprising:

[0021] The information acquisition module is used to acquire web page data to be processed and extract the information contained in the web page data. The information includes the question title, several question and answer posts corresponding to the question title, and code snippets corresponding to each question and answer post.

[0022] The prompt word acquisition module is used to construct prompt words for a large language model. The prompt words include role-specific prompt words, thought chain reasoning prompt words, and few-shot context learning prompt words.

[0023] The text description API method acquisition module is used to input the constructed prompt words and each code snippet into the large language model and output an API method with a text description.

[0024] The API method retrieval module is used to clean API methods with text descriptions using regular expressions, removing the text descriptions from the API methods to obtain the API method corresponding to each code snippet.

[0025] Thirdly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a method for automatically generating reusable APIs based on code snippets as described in any of the solutions of the first aspect above.

[0026] Fourthly, the present invention provides a computer electronic device, which includes a memory and a processor;

[0027] The memory is used to store computer programs;

[0028] The processor is configured to, when executing the computer program, implement a method for automatically generating reusable APIs based on code snippets as described in any of the first aspects above.

[0029] As a preferred option

[0030] Compared with the prior art, the present invention has the following advantages:

[0031] This invention constructs a Python evaluation set and verifies that the method of this invention can complete the APIzation task of code segments in different programming languages ​​by simply modifying the prompt content. The method of this invention achieves a parameter list accuracy of 69%, a return statement accuracy of 80%, and a method implementation accuracy of 57% on the APIzation task of Python code segments, which means that the method of this invention can be easily extended to other programming languages ​​without performance loss. The proposed method achieves a parameter list accuracy of 66%, a return statement accuracy of 65%, and a method implementation accuracy of 43.5%, which are 9.5%, 7.5%, and 11% higher than the corresponding metrics of the APIzator method, respectively, indicating that the proposed method can generate more accurate APIs.

[0032] Two user surveys were conducted in this embodiment of the invention. 1) First, the quality of method names generated by the method of this invention, human developers, and the APIzator tool was surveyed. The results showed that the proportions of method names generated by Code2API, human developers, and APIzator (i.e., the highest quality method names, with scoring details below) were 74.3%, 60.0%, and 10%, respectively. This indicates that the method names generated by the method of this invention are far superior to those generated by the APIzator tool, and even surpass the level of human developers. 2) Second, the overall quality of the method of this invention, human developers, and the APIzator tool was surveyed. Users were required to select the API they considered optimal from the triples of <method of this invention, human developers, and APIzator tool>. The results showed that among 200 triples, 50.5% of the triples corresponded to the optimal API of this invention, 48.0% of the triples corresponded to the optimal API of human developers, and 1.5% of the triples corresponded to the optimal API of the APIzator tool. This indicates that the overall quality of the method of this invention is far superior to that of the APIzator tool and slightly superior to that of human developers, and also shows that this invention has reached a level comparable to that of human developers in the APIzation task of Java code segments.

[0033] Thanks to the powerful code understanding capabilities of large language models, the method of this invention can fix various syntax errors in code segments without relying on external tools, which improves generalization ability and simplifies system implementation. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the steps of the present invention;

[0035] Figure 2 The workflow of the Google Chrome plugin in this embodiment of the invention;

[0036] Figure 3 This is a system block diagram of the present invention;

[0037] Figure 4 This is the structure of the prompt word in this invention. Detailed Implementation

[0038] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in the various embodiments of the present invention can be combined accordingly without mutual conflict.

[0039] This invention aims to solve the task of API zation of code segments. This task requires transforming code segments into reusable APIs. The input includes the code segment and its context (i.e., the corresponding question post, question title, and answer post), and the output is an API that reflects the intent of the code segment. The following uses the API zation of Java code segments as an example to describe the method of this invention in detail.

[0040] In a preferred embodiment of the present invention, such as Figure 1 As shown, a method for automatically generating reusable APIs based on code snippets is proposed. The specific process is as follows:

[0041] S1. Obtain the webpage data to be processed and extract the information contained in the webpage data. The above information includes the question title, several question and answer posts corresponding to the question title, and the code snippet corresponding to each question and answer post.

[0042] It should be noted that, in order to fairly compare with the existing technology APIzator, this embodiment of the invention uses APIzator's evaluation set. Since this evaluation set only includes answer code segments and answer ID information, and lacks context information for the code segments, this embodiment uses a web crawler to access the Stack Overflow page corresponding to each answer code segment using the answer ID, and parses the HTML elements to obtain the context of each answer code segment. When implementing the Google Chrome extension, the JavaScript script in this embodiment also uses HTML element parsing to obtain the context of the user-selected code segment. Subsequently, the code segment and its context will be used to construct suggestions for calling the large language model.

[0043] S2. Construct prompt words for a large language model. The prompt words include role-specific prompt words, thought chain reasoning prompt words, and few-shot context learning prompt words.

[0044] It should be noted that, in this invention, the aforementioned role-designated prompts include assigning roles to the large language model and informing the large language model of the task to be solved.

[0045] It should be noted that, in this invention, the aforementioned thought chain reasoning prompts include thought chain reasoning that uses the thought process of professional developers as a large language model.

[0046] It should be noted that, in this invention, the aforementioned few-shot context learning prompts include sample input, sample output, test input, and format constraints. The sample input and sample output are used to inform the large language model of the expected output format; the test input indicates that the large language model is currently converting the code segment and its context; and the format constraints are used to inform the large language model of the input and output formats for the specified API task.

[0047] To facilitate a better understanding of the present invention by those skilled in the art, the specific process of constructing prompt words for a large language model is described below in this embodiment.

[0048] Specifically, the basic approach of Code2API is suggestion engineering, which involves prompting a large language model to complete a specified task. Specifically, by using natural language prompts, embodiments of this invention can leverage a large language model that has learned a vast amount of knowledge from an extremely large training corpus to complete specific downstream tasks without additional training. Researchers have successfully applied suggestion engineering to various software engineering tasks, such as code generation, program repair, and code understanding. In embodiments of this invention, three suggestion engineering techniques—role assignment, thought chain reasoning, and few-shot context learning—are used to automatically build APIs for code snippets on Stack Overflow using the knowledge of a large language model.

[0049] In prompting engineering, role assignment is a technique for assigning roles to large language models to solve specific tasks. Assigning roles to large language models helps them understand the task context and produce more accurate and relevant answers. In the task of API zation of Java code snippets, this invention assigns the role of a professional Java developer to the large language model. After assigning roles to the large language model, embodiments of this invention explicitly inform the large language model of the task to be solved: "Given a context containing a question title, a question post, and an answer post, your task is to transform the Java code snippet in the answer post into a Java method based on the context information."

[0050] Thought chain reasoning is an important method in prompting engineering. It allows large language models to break down complex tasks into several simpler steps, producing a series of intermediate results, and ultimately generating a coherent and logical outcome. To achieve this method, this embodiment of the invention first invites professional developers to independently record their thought processes when transforming Stack Overflow Java code snippets, and then discusses them with the developers to summarize a clear and continuous thought process that can generate reusable APIs step by step. Specifically, this embodiment of the invention establishes an eight-step thought process as thought chain reasoning for large language models, which allows large language models to use the same thought process as professional developers to transform code snippets. Specifically, the first step of thought chain reasoning is "restore import statements from the code snippet, which can be empty if not needed." This step is used to resolve compilation errors caused by missing type declarations. The second step is "define a public class Chatgpt to wrap the method," creating a class named Chatgpt for subsequent processing. The third step is "create a public static modifier for the method." Steps 4 through 7 are respectively: "Create the method name based on the context or the code segment itself," "Infer the parameter list from the code segment, which can be empty if necessary," "Infer the return statement from the code segment, which can be empty if necessary," and "Infer the throws statement from the code segment, which can be empty if necessary." These steps guide the large language model to solve subtasks sequentially, such as creating the method name, inferring the parameter list, the return statement, and the throws statement. Step 8 is "Output the complete code based on the above results," used to generate the entire API. Overall, this sequential thought process guides the large language model to progressively generate a reusable API in a manner similar to that of a professional developer.

[0051] Since large language models have not been specifically trained for the APIification task of Java code snippets, this invention employs a few-shot context learning method to help the large language model further understand this task. This method uses some expected input and output examples to enhance the contextual information of the large language model, helping the model to understand the task quickly and accurately. In particular, for each step in the thought chain reasoning, the embodiments of this invention demonstrate the corresponding output based on specific examples. To select the most representative examples for this task, this invention examined 100 Stack Overflow posts and finally selected 5 examples covering all steps of the thought chain reasoning. Figure 4 The example input and example output are shown in the image.

[0052] like Figure 4As shown, the final prompt constructed by this invention consists of six parts: role assignment, thought chain reasoning, sample input, sample output, test input, and format constraints. Their functions are as follows: role assignment provides background information on the APIzation task, establishing context for subsequent steps; thought chain reasoning guides the large language model in a step-by-step manner, enabling the large language model to solve the APIzation task using the same thought process as the developer; sample input and sample output further clarify the task requirements and demonstrate the expected output format, helping the model to more accurately understand the task requirements; test input represents the code segment that the model is currently transforming and its context; format constraints specify the input format of the APIzation task and the output format of the large language model for subsequent processing.

[0053] S3. Input the constructed prompts and each code snippet into the large language model, and output an API method with a text description.

[0054] It should be noted that the above-mentioned large language model adopts the state-of-the-art GPT-3.5-turbo model as the large language model in this embodiment of the invention. This is one of the best instruction-tuned language models and has been proven to have excellent capabilities in tasks such as text summarization and machine translation.

[0055] To ensure the uniqueness of the experimental results, the embodiment of the present invention sets the temperature parameter of the large language model to 0 in all experiments to keep the model output consistent. The embodiment of the present invention also ensures that the prompts constructed for all data do not exceed the maximum input length of GPT-3.5-turbo, which means that the GPT-3.5-turbo model is sufficient to handle API zation tasks of code segments.

[0056] S4. Use regular expressions to clean the API methods with text descriptions, remove the text descriptions from the API methods with text descriptions, and obtain the API method corresponding to each code snippet.

[0057] Furthermore, in this embodiment of the invention, to improve the efficiency of developers reusing Stack Overflow code snippets, a Google Chrome extension (Code2API) is proposed. This extension uses three prompting engineering techniques—role assignment, few-shot context learning, and thought chain reasoning—to guide the Large Language Model (LLM) to automatically complete the APIization task of code snippets in a developer-like manner. The extension consists of three parts: a front-end, middleware, and a back-end. The front-end model runs on the user's browser and is responsible for retrieving the code snippets that the user needs to convert from the Stack Overflow webpage and displaying the generated reusable API methods for the user to use. The middleware model collects the code snippets and context information from the front-end and uses this information to automatically construct prompt words. The back-end model uses the constructed prompt words to call the Large Language Model to generate reusable APIs. Finally, the front-end updates the page to present the generated APIs to the developers.

[0058] The specific functions of each part of the Google Chrome extension are explained below.

[0059] 1) Front-end model

[0060] The front-end has two main functions: retrieving necessary information from the webpage and displaying the generated results. (1) Retrieving code snippets: When the loaded page is a Java or Python-related Stack Overflow page, the front-end will prompt the user to click on the code snippet that needs to generate the API. When the user clicks on the code snippet in an answer post, Code2API will collect the context of the code snippet and send it to the middleware model for parsing. (2) Displaying results: After the back-end model completes API generation, the front-end interface will display the generated reusable API for the user to use.

[0061] 2) Middleware Model

[0062] The middleware model obtains the necessary information (i.e., code snippets, question titles, question posts, and answer posts) from the front-end model and constructs hints according to the hint engineering process described in the previous section. Then, the middleware model sends the constructed hints to the back-end model for inference.

[0063] 3) Backend Model

[0064] The backend model is a large language model, namely GPT-3.5-turbo, which is responsible for analyzing code segments and their contexts and generating reusable APIs for the code segments.

[0065] The internal workflow of the Google Chrome extension (Code2API) is as follows: Figure 2The upper part shows: Given an answer code snippet, the plugin first extracts the complete context of the code snippet, namely the question title, question post, and answer post. Second, based on the code snippet and its context information, it constructs a hint using three hint engineering techniques: role assignment, thought chain reasoning, and few-shot context learning. Then, it uses the hint to call a large language model to obtain the model output. Finally, it uses regular expressions to post-process the output of the large language model to obtain the application programming interface (API) corresponding to the code snippet. Figure 2 The second half demonstrates the use of the Google Chrome extension (Code2API) from an application perspective. First, a user accesses a StackOverflow page through a Google Chrome browser with the Code2API extension installed and selects the answer code snippet they want to convert into an API. Then, the extension automatically calls a large language model using the code snippet's information to complete the API conversion task. Finally, the extension displays the post-processed API in the Google Chrome extension window for the user to use.

[0066] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0067] Conversely, this invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of the invention as defined in the claims. Furthermore, to provide a better understanding of the invention, certain specific details are described in detail below. However, those skilled in the art will fully understand the invention even without these detailed descriptions.

[0068] Example

[0069] To evaluate the Google Chrome extension (Code2API) in this embodiment, multiple experiments were conducted and the following issues were investigated:

[0070] Question 1. Compared to APIzator, can Code2API generate parameter lists, return statements, and method implementations more accurately?

[0071] Question 2. How is the quality of the method names generated by Code2API? Are developers willing to use APIs generated by Code2API?

[0072] Question 3. What impact do thought chain reasoning and few-shot context learning have on model performance?

[0073] Question 4. Is the Code2 API applicable to other programming languages?

[0074] For ease of description below, the APIs generated by the Google Chrome extension (Code2API), human developers, and the APIzator tool will be referred to as APIs respectively in this embodiment of the invention. C API H API A .

[0075] Question 1. Evaluation of parameter list, return statement, and method implementation.

[0076] 1. Experimental setup

[0077] Terragni et al. first proposed the APIzation task for Java code snippets, using 200 APIs generated by human developers as their evaluation set. They used APIs... H For annotation, and by comparing APIs A and API H To evaluate API A Accuracy in parameter lists, return statements, and method implementations. For a fair comparison, this embodiment also uses their evaluation sets, but these sets do not include contextual information about the code segments. As mentioned above, this embodiment collects contextual information about the code segments through a web crawler to construct prompts for calling the large language model.

[0078] 2. Evaluation metrics

[0079] This invention uses the evaluation metric of Terragni et al. to evaluate the Google Chrome plugin Code2API, but with modifications. The modified metric is defined as follows:

[0080] Equivalent Parameter Lists: Given two APIs, the equivalence of their parameter lists is defined as follows: (1) the two parameter lists have the same number of parameters; (2) every pair of parameters in the two parameter lists is equivalent. The equivalence of two parameters is defined as follows: variables that are identical in the method implementation have the same type. Unlike Terragni et al., this embodiment of the invention does not require equivalent parameters to have the same parameter name, because different parameter names may play the same role in the code segment.

[0081] Equivalent return statements: Given two APIs, their return statements are equivalent if both APIs have a return type of void, or if both APIs have the same return type and their method implementations contain the same return statement.

[0082] Equivalent method implementations: Given two APIs, their method implementation equivalence is defined as follows: (1) the two APIs have equivalent parameter lists; (2) the two APIs have equivalent return statements; (3) the method bodies of the two APIs implement the same functionality. Unlike the embodiments of this invention, Terragni et al. used GUMTREE to check whether two APIs are completely identical, which would miss many equivalent APIs. Therefore, the metric of the embodiments of this invention can better reflect the performance of the model.

[0083] For ease of description later, in this embodiment of the invention, the accuracy of the parameter list, the accuracy of the return statement, and the accuracy of the method implementation are denoted as P-accuracy, R-accuracy, and M-accuracy, respectively, and PR-accuracy is used to represent the proportion of APIs that have equivalent parameter lists or equivalent return statements.

[0084] 3. Experimental Results

[0085] Since the metrics of this embodiment are slightly different from the original metrics of the APIzator tool, this embodiment re-evaluates the APIzator tool using the modified metrics. Table 1 shows the original evaluation results of the APIzator tool (APIzator_ori), the evaluation results of the APIzator tool in this embodiment (APIzator_our), and the evaluation results of the Google Chrome plugin (Code2API) of this invention.

[0086] Table 1: Accuracy of each method

[0087]

[0088] The following analysis results can be obtained from Table 1:

[0089] (i) The Code2API of this invention outperforms the APIzator tool across all accuracy metrics. Specifically, Code2API achieves 66.0%, 65.0%, and 43.5% accuracy in parameter lists, return statements, and method implementations, respectively, which are 9.5%, 7.5%, and 11.0% higher than the corresponding metrics of APIzator. The reasons why Code2API significantly improves the accuracy of APIs across all aspects are as follows: (I) Mind chain reasoning enables Code2API to create APIs in a way similar to that of a developer; (II) Few-shot context learning enables Code2API to fully understand the APIzation task; (III) Code2API is not bound by limited rules and can generate APIs for a wide range of code segments, thereby improving accuracy.

[0090] (ii) The revised metric in this embodiment is reasonable. The evaluation results of the APIzator tool in this embodiment show almost no deviation from the evaluation results in the original paper, with only a slight difference in M-accuracy. This indicates that the revised metric in this embodiment is reasonable. Since the equivalent APIs generated by the APIzator tool are almost identical, the revision in this embodiment has little impact on the results of the APIzator tool. However, the equivalent APIs generated by large language models often have some formal differences, such as different variable names. If the original metric is used, the capabilities of Code2API will be greatly underestimated.

[0091] Therefore, compared to the APIzator tool, the Google Chrome extension (Code2API) in this embodiment can generate more accurate parameter lists, return statements, and method implementations. The accuracy rates of these three metrics are 66.0%, 65.0%, and 43.5%, respectively, which are 9.5%, 7.5%, and 11.0% higher than the APIzator tool.

[0092] Question 2. Evaluation of method name quality and overall API quality

[0093] 1. Experimental setup

[0094] As described in the background section, a key drawback of the APIzator tool is that the method names it generates are almost always inaccurate or even meaningless. The original authors neglected the importance of descriptive method names and did not evaluate the quality of method names in their study. Therefore, in addressing this problem, the objective of this embodiment of the invention is to evaluate APIzator. C API H API A Method name quality. Since automatic evaluation struggles to determine the semantic correctness of method names, this embodiment of the invention conducted a user survey to address this research problem. This embodiment of the invention recruited 18 volunteers to participate in the user survey. This embodiment of the invention reused the evaluation dataset from Problem 1, which contains 200 API triples: <API C API H API A Then, in this embodiment of the invention, the 200 API triples are divided into six subsets (each subset contains 33-34 API triples). Each subset is then independently evaluated by three volunteers. Specifically, each volunteer receives one Stack Overflow answer post (including the question title and answer post link) and three APIs, which are evaluated in two aspects: method name quality and overall API quality (see below). It is worth noting that participants do not know which API was generated by which method.

[0095] 2. Evaluation metrics

[0096] Method name quality: In the user study of this invention embodiment, each volunteer was asked to read the method name, method implementation, and related posts, and then rate the method name. The score range was 1 to 4, and the meaning of each score is as follows:

[0097] A score of 1 indicates that the method name and method implementation are unrelated.

[0098] A score of 2 indicates a low relevance between the method name and the method body.

[0099] A score of 3 indicates a high degree of relevance between the method name and the method body.

[0100] A score of 4 indicates that the method name and method body are a perfect match, and the method name fully expresses the intent of the method.

[0101] Overall API Quality: In the user study of this invention embodiment, for a given API triplet... <API C API H API A After each participant scores each API method name, this embodiment of the invention requires each participant to select the optimal API from three candidate APIs based on their own development experience. Since each triple is independently evaluated by three volunteers, the optimal API is determined by voting. If three different choices are obtained, further discussion with the corresponding three evaluators is required to reach a consensus.

[0102] 3. Experimental Results

[0103] Through research, this embodiment of the invention collected human evaluation results for 600 API triples from 18 evaluators. Each triple corresponds to three method name scores and one best API. Since each triple was evaluated by three different evaluators, this embodiment of the invention merged the human evaluation results of six subsets and calculated the Cohen's Kappa coefficients between the three sets of user ratings, which were 0.67, 0.7, and 0.71, all greater than 0.6, indicating significant consistency among different evaluators. Finally, this embodiment of the invention also performed a Mann-Whitney U rank test to verify whether there are significant differences between the Google Chrome plugin (Code2API) and other methods. Table 2 shows all the evaluation results.

[0104] Table 2: Method names and overall API quality of each method

[0105]

[0106] The following analysis results can be obtained from Table 2:

[0107] (i) The method names of the Google Chrome extension (Code2API) in this embodiment are of significantly higher quality than those of the APIzator tool, and slightly better than those of human developers. Specifically, API C API H API A The percentages of method names rated 4 points were 74.3%, 60.0%, and 10.0%, respectively, indicating that the vast majority of method names generated by Code2API match the functionality of the method body. Method names generated by the APIzator tool were mainly rated 2 or 3 points (67.3%), meaning that most method names generated by APIzator are inaccurate. Furthermore, the percentage of 4-point method names generated by the Google Chrome extension (Code2API) (74.3%) is slightly better than that of human developers (60.0%), indicating that the quality of method names generated by Code2API even surpasses that of human developers. The reasons for Code2API generating high-quality method names are as follows: (I) This embodiment of the invention provides complete contextual information of the answer code segment to create the method name; (II) The large language model has excellent language understanding capabilities and can summarize contextual information; (III) The large language model has excellent code understanding capabilities and can understand the semantics of the code segment itself; (IV) The hint project in this embodiment of the invention successfully utilizes the corresponding language and code knowledge of the large model.

[0108] (ii) Statistically, the method name quality of the Google Chrome extension (Code2API) in this embodiment is significantly better than that of APIzator and human developers. Since the average method name score of Code2API is 3.65, slightly higher than the 3.39 of human developers, this embodiment of the invention performed a Mann-Whitney U rank test to verify whether Code2API is significantly superior to human developers. Furthermore, this embodiment of the invention also performed the same test on Code2API and APIzator. The results are shown in the p-value column of Table 2. The p-values ​​of both tests are much less than 0.01. Therefore, this embodiment of the invention concludes that the method name quality of Code2API is significantly better than that of APIzator and human developers.

[0109] (iii) The overall API quality of the Google Chrome extension (Code2API) in this embodiment is far superior to APIzator and slightly superior to that of human developers. Of the 200 triples, 50.5% correspond to the optimal API. C 48.0% of the triples correspond to the optimal API, which is API. H The optimal API corresponding to 1.5% of triples is API. A This indicates that the APIC The overall quality is far superior to API. A Slightly better than API H This also demonstrates that the present invention has reached a level comparable to that of human developers in the API zation task of Java code segments.

[0110] Therefore, it can be seen that the quality of method names generated by Code2API is far superior to APIzator, and slightly superior to those generated by human developers. More than half (50.5%) of the APIs generated by Code2API are intended to be used by developers.

[0111] Question 3. Ablation experiments for the project.

[0112] 1. Experimental setup

[0113] This invention employs two key techniques in constructing hints: thought chain reasoning and few-shot context learning. In this study, an ablation analysis was performed to verify their effectiveness. Specifically, the invention compared the Google Chrome extension (Code2API) with three incomplete versions: (I) few-shot removed: This invention does not use few-shot context learning in constructing hints; (II) thought chain removed: This invention does not use thought chain reasoning in constructing hints; (III) both removed: This invention does not use either few-shot context learning or thought chain reasoning in constructing hints. The invention repeatedly uses the evaluation metrics described in Problem 1 for comparison.

[0114] 2. Experimental Results

[0115] Table 3: Accuracy of different prompts

[0116]

[0117] The evaluation results are shown in Table 3. It can be seen that:

[0118] (i) Regardless of which prompting technique is removed from the embodiments of the present invention, the overall performance of the model in the embodiments of the present invention will be compromised. This illustrates the importance and necessity of using thought chain reasoning and few-shot context learning in prompting engineering.

[0119] (ii) Mind chain reasoning and few-shot context learning improve the accuracy of generated APIs in a complementary manner. Comparing two experiments with the removal of few-shot context learning and the removal of mind chain reasoning, the removal of few-shot context learning performs better in R-accuracy, while the removal of mind chain reasoning performs better in P-accuracy, indicating that the two are complementary.

[0120] (iii) Directly using large language models cannot solve the APIzation problem; designing prompts suitable for specific tasks is key to applying large language models. After simultaneously removing thought chain reasoning and few-shot context learning, Code2API's performance drops sharply, far below that of the APIzator tool. This further confirms the effectiveness of leveraging thought chain reasoning and few-shot context learning to extract relevant knowledge and logical reasoning from large language models to complete APIzation tasks in a manner similar to human developers.

[0121] Therefore, it can be seen that thought chain reasoning and few-shot context learning improve the accuracy of generated APIs in a complementary manner, and removing either one will impair the overall performance of the model in this embodiment of the invention.

[0122] Question 4. Generalization Research

[0123] 1. Experimental setup

[0124] As mentioned in the background section, one of the drawbacks of the APIzator tool is that its design relies on specific rules and external tools, making it difficult to extend to other programming languages. Compared to the APIzator tool, the Google Chrome extension (Code2API) proposed in this embodiment is based on hint engineering, is extremely lightweight and flexible, and therefore, the goal of this invention is to investigate the universality of Code2API. Specifically, this invention further evaluates whether Code2API can be easily extended to the APIzation task of Python code segments and achieve performance comparable to Java.

[0125] To conduct this generalized study, this embodiment of the invention collected 5000 code snippets from Stack Overflow Python answer posts that met specific criteria (e.g., question titles were of the "How to" type, at least 2 points for accepted answers, only one code snippet in the answer post, and the Stack Overflow page views were in the top 20,000, etc.) following the APIzator data collection process. Then, this embodiment selected the top 100 Python code snippets (based on Stack Overflow page views) and manually created APIs as real labels for these snippets. Next, this embodiment slightly modified the thought chain reasoning and few-shot context learning (e.g., Python functions did not need to be encapsulated in a class) to enable the Google Chrome extension (Code2API) to complete the APIzation task for the Python code snippets. Finally, this embodiment applied Code2API to the 100 Python code snippets with manually created APIs, generating reusable APIs for these snippets.

[0126] 2. Experimental Results

[0127] The embodiments of the present invention use the same evaluation metrics as in Problem 1, and compare their performance with the Java evaluation results, as shown in Table 4.

[0128] Table 4: Accuracy of this embodiment on different programming languages

[0129]

[0130] Based on the experimental results shown in Table 4, the embodiments of the present invention can be analyzed to obtain:

[0131] By switching the suggestions from processing Java code snippets to processing Python code snippets, the Google Chrome extension (Code2API) in this example performed better across all evaluation metrics. This is because Python variables do not require explicit data type declarations; therefore, Code2API can more easily infer the parameters and return statements of Python methods. Code2API's consistently good performance on Python datasets confirms that it can be easily generalized to other programming languages ​​without any performance penalty.

[0132] As can be seen, Code2API can complete APIation tasks for code segments in different programming languages ​​by simply modifying the hint content, without sacrificing performance.

[0133] It should also be noted that the method for automatically generating reusable APIs based on code snippets in the above embodiments can essentially be executed by a computer program or module. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a system for automatically generating reusable APIs based on code snippets, corresponding to the method for automatically generating reusable APIs based on code snippets provided in the above embodiments, such as... Figure 3 As shown, it includes:

[0134] The information acquisition module is used to acquire web page data to be processed and extract the information contained in the web page data. The information includes the question title, several question and answer posts corresponding to the question title, and code snippets corresponding to each question and answer post.

[0135] The prompt word acquisition module is used to construct prompt words for a large language model. The prompt words include role-specific prompt words, thought chain reasoning prompt words, and few-shot context learning prompt words.

[0136] The text description API method acquisition module is used to input the constructed prompt words and each code snippet into the large language model and output an API method with a text description.

[0137] The API method retrieval module is used to clean API methods with text descriptions using regular expressions, removing the text descriptions from the API methods to obtain the API method corresponding to each code snippet.

[0138] Similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer electronic device corresponding to the method for automatically generating reusable APIs based on code snippets provided in the above embodiments, which includes a memory and a processor;

[0139] The memory is used to store computer programs;

[0140] The processor is configured to implement, when executing the computer program, a method for automatically generating reusable APIs based on code snippets as described in the above embodiments.

[0141] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0142] Therefore, based on the same inventive concept, another preferred embodiment of the present invention also provides a computer-readable storage medium corresponding to the method for automatically generating reusable APIs based on code snippets provided in the above embodiments. The storage medium stores a computer program that, when executed by a processor, can implement the method for automatically generating reusable APIs based on code snippets in the above embodiments.

[0143] It is understood that the aforementioned storage media may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Furthermore, the storage media may also be various media capable of storing program code, such as USB flash drives, external hard drives, magnetic disks, or optical discs.

[0144] It is understood that the processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0145] It should also be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the embodiments provided in this application, the division of steps or modules in the system and method is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules or steps may be combined or integrated together, and a module or step may also be split.

[0146] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A method for automatically generating reusable APIs based on code snippets, characterized in that, Includes the following steps: S1. Obtain the webpage data to be processed and extract the information contained in the webpage data, including the question title, several question and answer posts corresponding to the question title, and the code snippet corresponding to each question and answer post; S2. Construct prompt words for a large language model, wherein the prompt words include role-specific prompt words, thought chain reasoning prompt words, and few-shot context learning prompt words; S3. Input the constructed prompts and each code snippet into the large language model, and output an API method with a text description; S4. Use regular expressions to clean the API methods with text descriptions, remove the text descriptions from the API methods with text descriptions, and get the API method corresponding to each code snippet; The thought chain reasoning prompts include thought chain reasoning that uses the thought process of professional developers as a large language model. The few-shot context learning prompts include sample input, sample output, test input, and format constraints. The sample input and sample output are used to inform the large language model of the expected output format; the test input indicates that the large language model is currently converting the code segment and the code segment context. Format constraints are used to inform large language models of the input and output formats for API-based tasks.

2. The method for automatically generating reusable APIs based on code snippets as described in claim 1, characterized in that, The large language model adopts the GPT-3.5-turbo model.

3. The method for automatically generating reusable APIs based on code snippets as described in claim 1, characterized in that, The role-assigning prompts include assigning roles to the large language model and informing the large language model of the task to be solved.

4. A system for automatically generating reusable APIs based on code snippets, characterized in that, This method is used to implement the method for automatically generating reusable APIs based on code snippets as described in any one of claims 1 to 3.

5. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, implements the method for automatically generating reusable APIs based on code snippets as described in any one of claims 1 to 3.

6. A computer electronic device, characterized in that, Including memory and processor; The memory is used to store computer programs; The processor is configured to, when executing the computer program, implement the method for automatically generating reusable APIs based on code snippets as described in any one of claims 1 to 3.

Images