A library migration recommendation method based on search enhancement generation

Abstract

The application discloses a library migration recommendation method based on retrieval enhancement generation, which firstly adopts a large model technology to extract the migration purpose of the submission information in the code repository, avoids the performance degradation problem caused by manual extraction, and improves the efficiency and accuracy of the migration purpose extraction; then the migration purpose and the migration purpose category of the developer are introduced, so that the recommendation process not only considers the functional characteristics of the to-be-migrated library itself, but also combines the migration purpose of the developer, ensuring that the recommended library can realize replacement in the specified scene; finally, the retrieval enhancement technology is used in combination with the large language model to generate the library migration recommendation list, and the learning of the historical migration mode is further introduced, so that the library migration recommendation process can refer to the historical migration rules, and meanwhile, it is ensured that the generated library migration recommendation list is not limited to the collected data and has generalization. The application enables the developer to quickly obtain the target library that can be migrated, and significantly improves the quality of software maintenance.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of artificial intelligence and software engineering, and specifically relates to a library migration recommendation method based on retrieval enhancement generation. Background Technology

[0002] Library migration is a critical activity in modern software development, which involves the introduction of numerous third-party maintained libraries. However, over time, some libraries may become unmaintained, develop defects and vulnerabilities, and these deteriorating libraries can have unpredictable adverse effects on projects that use them. Therefore, developers may replace deteriorating libraries with newer, more efficient, or easier-to-use libraries; this process is commonly referred to as library migration.

[0003] With the rapid expansion and continuous evolution of the modern software ecosystem, timely replacement of third-party libraries is a crucial step in ensuring project security and stability. For developers, when faced with complex challenges such as security vulnerabilities and dependency conflicts, it is essential to quickly integrate new alternative libraries to guarantee the sustainable evolution of the project. However, quickly selecting suitable alternative libraries often heavily relies on extensive migration experience and deep technical expertise, especially for developers with limited experience. This leads to the need for library migration recommendations.

[0004] Most existing library migration recommendation methods rely on manually defined evaluation metrics for screening. These methods primarily focus on the migration frequency, application domain, migration mechanism, and technological motivations of historical library migrations. They calculate scores for candidate libraries using predefined formulas and recommend target libraries to those awaiting migration. However, these solutions cannot achieve automated, intelligent library migration recommendations. Due to the diversity of data, developers still need to spend considerable time and effort designing comprehensive evaluation metrics and calculation formulas. For library characteristics that are difficult to quantify, this approach is not only inefficient but also prone to inconsistent quality in library migration recommendations.

[0005] Furthermore, existing research has demonstrated that large language models (LLMs) possess the ability to migrate code between specified libraries. For example, the paper [Almeida, Aylton, Laerte Xavier, and Marco Tulio Valente. "Automatic Library Migration Using Large Language Models: First Results." Proceedings of the 18th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement. 2024] utilizes a framework based on large language models (LLMs) and designs various suggestion methods such as Zero-Shot, One-Shot, and Chain of Thoughts to achieve code migration between different versions using GPT-4. This method allows for relatively fast code migration between different versions of the same library, effectively improving code maintainability. However, the application scope of such methods is limited to different versions of the same library, as they cannot recommend target libraries for migration between different libraries, thus restricting code migration between different libraries.

[0006] The challenge of recommending libraries for migration lies in the need for a detailed understanding of the characteristics of the library to be migrated, the developer's purpose for migration, and the history of previous migrations. This involves natural language descriptions that lack structural information, requiring the recommender to understand the library to be migrated and its migration history. Therefore, simply relying on existing large language models for library migration is insufficient to handle the complexity of library migration scenarios, and the recommended libraries may lack applicability to the specific scenario.

[0007] Existing library migration recommendation methods primarily rely on learning from historical migration records. However, their effectiveness is limited to considering only the relationships between libraries, failing to adequately account for the specificities of each migration behavior and thus unable to provide dynamic recommendations based on different migration actions. Currently, superior methods, such as those in the literature [He, Hao, et al. "A multi-metric ranking approach for library migration recommendations." 2021 IEEE International Conference on Software Analysis, Evolution and Reengineering (Saner). IEEE, 2021], collect historical library migration records, calculate support scores across four dimensions to obtain a final ranking score, and then determine the target libraries for recommendation. However, such methods also have significant limitations, as the four specified dimensions may not be comprehensive. Existing research has demonstrated that the scenarios in which human developers migrate libraries are diverse and categorizable. For example, the paper [Gu, Haiqiao, Hao He, and Minghui Zhou. "Self-admitted librarymigrations in Java, JavaScript, and Python packaging ecosystems: A comparative study." 2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 2023] categorizes library migrations into three main types: migration caused by the library to be migrated, migration caused by the attributes of the target library, and migration caused by the project itself, which are further subdivided into 13 subcategories.

[0008] Therefore, how to design a library migration recommendation method that combines the purpose of library migration and historical migration patterns to better apply complex migration scenarios and promote the sustainable development of software libraries has become an urgent problem to be solved. Summary of the Invention

[0009] In view of the above, the present invention provides a library migration recommendation method based on retrieval enhancement generation. This method can effectively learn historical migration patterns while considering the semantic information of the migration destination, providing developers with an effective recommendation tool to deal with complex library migration scenarios.

[0010] A library migration recommendation method based on retrieval enhancement includes the following steps:

[0011] (1) Collect historical migration records and store them in a structured data format to build a dataset for the code repository;

[0012] (2) Input the submission information and migration purpose classification rules corresponding to the library migration into the large language model as prompt words to generate the migration purpose and its category;

[0013] (3) Search and sort similar library migration records in the search dataset;

[0014] (4) Input the name of the library to be migrated, the purpose of migration, the category of the purpose of migration, and the retrieved library migration records as prompt words into the large language model to automatically generate a list of recommended code repositories.

[0015] Furthermore, the specific implementation method of step (1) is as follows:

[0016] 1.1 Construct API (Application Programming Interface) requests based on the specified programming language, number of stars, number of contributors, contributor type, and last update time. Use time-slice recursion to bypass API access restrictions and obtain a large number of code repositories and their metadata.

[0017] 1.2 Construct an API request based on the file name to which the code repository migration belongs, and use pagination query to bypass API access restrictions to obtain the historical commits of the code repository and the corresponding library migration commit information;

[0018] 1.3 Each historical commit is evaluated by manually creating library migration extraction rules. Historical commits that meet the library migration extraction rules are retained, and regular expressions are used to match and obtain the library migration records in each historical commit.

[0019] 1.4 Construct a dataset of the code repository based on the obtained code repository and its metadata, historical commits and library migration records.

[0020] Furthermore, the library migration extraction rules include:

[0021] ① In the historical commits to which the database migration belongs, the difference between the number of databases added and the number of databases deleted should be less than or equal to 1;

[0022] ② In the historical commits to which the database migration belongs, the sum of the number of databases added and the number of databases deleted should be greater than 2.

[0023] Furthermore, the specific implementation of step (2) is as follows:

[0024] 2.1 Using the library migration submission information as context, construct prompt words and pass them to the GPT-4o API to request the extraction of the migration purpose. The GPT-4o API is a multimodal artificial intelligence model interface provided by OpenAI. This interface will extract and output the migration purpose in the submission information based on the pre-trained large language model and the provided task instructions.

[0025] 2.2 Use the migration purpose and its classification rules as input data to the GPT-4o API to request the generation of migration purpose categories. The GPT-4o API will generate migration purpose categories based on the provided input data.

[0026] 2.3 By constructing fuzzy matching and ignoring special characters and spaces, the target library name appearing in the migration destination is masked to obtain the masked migration destination.

[0027] Furthermore, the specific implementation method of step (3) is as follows:

[0028] 3.1 Divide the dataset in the code repository into a retrieval database and a test dataset in an 8:2 ratio;

[0029] 3.2 Construct a retrieval triple containing the name of the database to be migrated, the purpose of migration, and the category of the purpose of migration, and use this structured information as the retrieval query statement;

[0030] 3.3 Calculate and sort the similarity between the search query and each database migration record in the search database, and extract the top few database migration records with the highest similarity.

[0031] Furthermore, the specific implementation of step (4) is as follows:

[0032] 4.1 Use the name of the database to be migrated, the purpose of migration, the category of the purpose of migration, and the retrieved database migration records as prompt words;

[0033] 4.2 Construct an API request and send it to the server where the large language model is located. This API request includes authentication information, prompt words, and other parameter settings that may affect the generated results.

[0034] 4.3 The server parses the received API requests, inputs the prompt words into the large language model, and uses the retrieved library migration records as context information to automatically generate a recommended list of code repositories.

[0035] A computer device includes a memory and a processor, wherein the memory stores a computer program and the processor executes the computer program to implement the above-described library migration recommendation method based on retrieval enhancement generation.

[0036] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described library migration recommendation method based on retrieval enhancement generation.

[0037] This invention proposes a library migration recommendation method that combines retrieval enhancement generation and large model technology for goal-oriented library migration recommendations, significantly improving performance. First, this invention employs large model technology to extract migration purposes from commit information in the code repository, avoiding the performance degradation caused by manual extraction and improving the efficiency and accuracy of migration purpose extraction. Second, by incorporating the developer's migration purpose and migration purpose category, this invention ensures that the recommendation process considers not only the functional characteristics of the library to be migrated but also the developer's migration purpose, guaranteeing that the recommended library can replace the target library in the specified scenario. Finally, this invention utilizes retrieval enhancement technology combined with a large language model to generate a library migration recommendation list, further incorporating the learning of historical migration patterns. This allows the library migration recommendation process to reference historical migration rules while ensuring that the generated library migration recommendation list is not limited to the collected data and has generalization capabilities. This technical effect enables developers to quickly obtain target libraries that can be migrated, significantly improving the quality of software maintenance, especially in the maintenance and development of large code projects, and has significant practical application value. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the library migration recommendation method based on retrieval enhancement according to the present invention.

[0039] Figure 2 This is a schematic diagram illustrating the extraction of prompt words for migration purposes in this invention.

[0040] Figure 3 This is a schematic diagram of the prompt words in the migration purpose classification of this invention.

[0041] Figure 4 This is a schematic diagram of the prompt words in the generation of the library migration recommendation list of this invention. Detailed Implementation

[0042] To describe the present invention in more detail, the technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0043] like Figure 1 As shown, the library migration recommendation method based on retrieval enhancement of the present invention includes the following steps:

[0044] Data collection and preprocessing.

[0045] Step S11. Repository Acquisition and Filtering: Construct a GitHub API request to acquire repository code, and filter the repository according to the following requirements: ① More than 10 stars; ② Python is used as the programming language; ③ The last push operation of the repository must be later than January 1, 2020; ④ The number of contributors must be greater than 2 and bot contributions are excluded; Use time-slice recursive partitioning to bypass API access restrictions and acquire a large number of repository code and their metadata as part of the dataset.

[0046] Step S12. Obtaining historical commits and commit information: Based on the collected code repository data, construct an API request to obtain historical commits. The obtained historical commits require configuration changes to the requirement.txt file. Use pagination query to bypass the access restrictions of the API request and obtain historical commits and corresponding commit information that meet the conditions.

[0047] Step S13. Extraction of Library Migration Records: The obtained historical commits are evaluated using manually selected library migration rules. Committances that conform to the library migration rules are retained, and regular expressions are used to match and extract the library migration records from each historical commit. The library migration extraction rules include two rule judgments:

[0048] ① In the historical commits to which the database migration belongs, the difference between the number of databases added and the number of databases deleted should be less than or equal to 1;

[0049] ② In the historical commits to which the database migration belongs, the sum of the number of databases added and the number of databases deleted should be greater than 2.

[0050] The generation and classification of migration objectives.

[0051] Step S21. Extract the migration purpose from the commit message: To better reflect the developer's original purpose when migrating the library and preserve its semantic information, the migration purpose is extracted from the natural language form of the commit message. This is achieved by using the commit message as a contextual prompt, such as... Figure 2 As shown, the constructed prompt words are passed to the GPT-4o API, requesting the extraction of the transfer purpose. The GPT-4o API will output the transfer purpose in the submission information based on its pre-trained language model and the provided task prompts.

[0052] Step S22. Generate migration destination categories: Using the defined library migration destination classification rules, generate migration destination categories. This involves requesting the generation of migration destination categories by using the migration destination and the classification rules as input data to the GPT-4o API. The GPT-4o API will then process the generated categories according to the specified rules. Figure 3 The context information shown in the prompts generates the migration destination category.

[0053] This implementation follows the migration purpose classification rules proposed in "A large-scale empirical study on Java library migrations: prevalence, trends, and rationales." These rules define four main categories: source library issues, target library advantages, project-specific reasons, and other issues. Migration intentions under the source library issues category typically stem from defects in the source library, such as lack of maintenance, outdated versions, security vulnerabilities, or program errors and defects. Intentions under the target library advantages category highlight the superiority offered by the target library, including better ease of use, more granular features, better performance, smaller size / complexity, higher popularity, stronger stability and maturity, and a more active community ecosystem. Project-specific intentions are related to the unique needs of the project, such as easier integration, streamlined project requirements, ensuring license compatibility, and organizational factors.

[0054] Step S23. Masking the migration destination: In order to prevent the name of the target library or its variants from appearing in the text information of the migration destination, the name of the target library in the migration destination is masked to obtain the masked migration destination. That is, by constructing fuzzy matching, ignoring variants such as special characters and spaces, the name of the target library appearing in the migration destination is masked as [mask].

[0055] Retrieve library migration records.

[0056] Step S31. Construct the retrieval database: Divide the complete dataset into a retrieval database and a test dataset in an 8:2 ratio.

[0057] Step S32. Construct the search query statement: Construct a search triplet containing the name of the database to be migrated, the migration purpose, and the migration purpose category, and use the structured information as the search query statement.

[0058] Step S33. Retrieve the most similar historical database migration records: In order to focus on the keyword information in the name of the database to be migrated and the migration purpose, the BM25 score is used to calculate the similarity between the retrieved triplet and each data in the retrieved vector database, and the three most similar historical database migration records are obtained by sorting.

[0059] A list of recommended libraries is generated.

[0060] Step S41. Construct suggestion words: Use the name of the database to be migrated, the purpose of migration, the category of the purpose of migration, and the retrieved historical database migration records as suggestion words, such as... Figure 4 As shown.

[0061] Step S42. Send API request: Construct an API request and send it to the server where the large language model is located. The API request includes authentication information, prompt words, and other parameter settings that may affect the generated results.

[0062] Step S43. Large Model API Call: The server parses the received API request, inputs the prompt words into the large language model, and uses the retrieved historical library migration records as the context of the model input to generate the recommendation library list. This prompt list sorts the recommendation libraries in priority order, which can help less experienced developers quickly obtain library migration solutions when maintaining projects.

[0063] The specific calling process of the large model API in this embodiment is as follows:

[0064] ① Formatted prompts: The designed prompts, including model role specifications, task descriptions, retrieval examples, and input and output format restrictions, are formatted into appropriate input formats according to the requirements of the large model API. The formatted content must ensure the integrity of the information so that the large model can understand the task and output correctly.

[0065] ② Constructing API requests: Use Python to construct API requests. The requests should include the API key, the selected large model version, the formatted prompt words, and the parameter settings for calling the large model (such as the maximum length of the generated text, temperature parameters, etc.).

[0066] ③ Send Request: Send the constructed API request to the server where the large model resides.

[0067] ④ Receiving and processing responses: After receiving the response data returned by the large model, the response content is parsed according to the preset format, the generated library migration recommendation list is extracted, and the 10 generated library names arranged in priority order are saved to ensure that the format is correct and the content is complete.

[0068] In generative recommendation tasks, this implementation uses the Precision@k (precision of the first k results) evaluation metric. This metric measures the quality of the recommendation results by calculating the ratio of correctly predicted positive samples in the first k recommended results to the total number of correctly predicted positive samples. Typically, k is set to 1, 3, 5, or 10 to hierarchically represent the model's performance at different recommendation lengths. To evaluate the model's ability to rank and understand optimal solutions, this implementation uses the MRR (mean reciprocal rank) evaluation metric. This metric focuses on the position of the correct answer, taking the reciprocal of the correct answer's ranking in the given results as the accuracy, and then averaging it across all questions. This is suitable for scenarios where each query has only one correct answer, emphasizing the importance of prioritizing the display of correct answers.

[0069] In this embodiment, the effectiveness of the present invention is evaluated on a constructed test set, where k is selected as 1, 3, 5, and 10; Claude-3.7-Sonnet and GPT-4 are selected as recommendation models, and the final comparison results are shown in Table 1:

[0070] Table 1

[0071]

[0072] As can be seen from the table, when using Claude-3.7-Sonnet and GPT-4 as recommendation models, the recommendation performance of Claude-3.7-Sonnet is slightly higher than that of GPT-4 (Claude-3.7-Sonnet's Precision@1 is 0.566 and MRR is 0.652, while GPT-45's Top-1 is 0.516 and MRR is 0.598). However, the Precision@10 of both models exceeds 0.7 and the MRR exceeds 0.5. This indicates that the present invention can demonstrate superior recommendation accuracy under different recommendation models, and that the present invention can prioritize and recommend the best results to developers in the recommendation list.

[0073] The above description of the embodiments is provided to enable those skilled in the art to understand and apply the present invention. Those skilled in the art can readily make various modifications to the above embodiments and apply the general principles described herein to other embodiments without creative effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made to the present invention by those skilled in the art based on the disclosure thereof should be within the scope of protection of the present invention.

Claims

1. A library migration recommendation method based on retrieval enhancement generation, characterized in that, Includes the following steps: (1) Collect historical migration records and store them in a structured data format to build a dataset for the code repository; (2) The submission information, migration purpose, and classification rules corresponding to the library migration are used as prompt words and input into the large language model to generate the migration purpose and its category. The specific implementation method is as follows: 2.1 Using the library migration submission information as context, construct prompt words and pass them to the GPT-4o API to request the extraction of the migration purpose. The GPT-4o API is a multimodal artificial intelligence model interface provided by OpenAI. This interface will extract and output the migration purpose in the submission information based on the pre-trained large language model and the provided task instructions. 2.2 Use the migration purpose and its classification rules as input data to the GPT-4o API to request the generation of migration purpose categories. The GPT-4o API will generate migration purpose categories based on the provided input data. 2.3 By constructing fuzzy matching and ignoring special characters and spaces, the target library name appearing in the migration destination is masked to obtain the masked migration destination; (3) Search and sort similar library migration records in the search dataset; (4) Input the name of the library to be migrated, the purpose of migration, the category of the purpose of migration, and the retrieved library migration records as prompt words into the large language model to automatically generate a list of recommended code repositories.

2. The library migration recommendation method based on retrieval enhancement as described in claim 1, characterized in that: The specific implementation method of step (1) is as follows: 1.1 Construct API requests based on the specified programming language, number of stars, number of contributors, contributor type, and last update time. Use time-slice recursive partitioning to bypass API access restrictions and obtain a large number of code repositories and their metadata. 1.2 Construct an API request based on the file name to which the code repository migration belongs, and use pagination query to bypass API access restrictions to obtain the historical commits of the code repository and the corresponding library migration commit information; 1.3 Each historical commit is evaluated by manually creating library migration extraction rules. Historical commits that meet the library migration extraction rules are retained, and regular expressions are used to match and obtain the library migration records in each historical commit. 1.4 Construct a dataset of the code repository based on the obtained code repository and its metadata, historical commits and library migration records.

3. The library migration recommendation method based on retrieval enhancement generation according to claim 2, characterized in that: The library migration extraction rules include: ① In the historical commits to which the database migration belongs, the difference between the number of databases added and the number of databases deleted should be less than or equal to 1; ② In the historical commits to which the database migration belongs, the sum of the number of databases added and the number of databases deleted should be greater than 2.

4. The library migration recommendation method based on retrieval enhancement generation according to claim 1, characterized in that: The specific implementation method of step (3) is as follows: 3.1 Divide the dataset in the code repository into a retrieval database and a test dataset in an 8:2 ratio; 3.2 Construct a retrieval triple containing the name of the database to be migrated, the purpose of migration, and the category of the purpose of migration, and use this structured information as the retrieval query statement; 3.3 Calculate and sort the similarity between the search query and each database migration record in the search database, and extract the top few database migration records with the highest similarity.

5. The library migration recommendation method based on retrieval enhancement generation according to claim 1, characterized in that: The specific implementation method of step (4) is as follows: 4.1 Use the name of the database to be migrated, the purpose of migration, the category of the purpose of migration, and the retrieved database migration records as prompt words; 4.2 Construct an API request and send it to the server where the large language model is located. This API request includes authentication information, prompt words, and other parameter settings that affect the generated results. 4.3 The server parses the received API requests, inputs the prompt words into the large language model, and uses the retrieved library migration records as context information to automatically generate a recommended list of code repositories.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: The processor is used to execute the computer program to implement the library migration recommendation method based on retrieval enhancement as described in any one of claims 1 to 5.

7. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, it implements the library migration recommendation method based on retrieval enhancement as described in any one of claims 1 to 5.

Images