Data construction method, code question-answering method, task platform and code question-answering system

Abstract

Provided in the embodiments of the present disclosure are a data construction method, a code question-answering method, a task platform and a code question-answering system. The data construction method comprises: acquiring a first code snippet and an initial sample question for the first code snippet; using a text processing model to rewrite the initial sample question by using directory structure information of a sample code repository in a code library as context information, so as to obtain a sample question; on the basis of the first code snippet and the sample question, recalling a related second code snippet from the code library; and using the text processing model to generate, by using the second code snippet as context information, a sample answer corresponding to the sample question, wherein the sample question and the sample answer are sample data used for training a code question-answering model, and the code question-answering model is applicable to the code repository. The data construction efficiency is improved, and the data construction cost is reduced; and a model obtained by means of training is adapted to a code repository, thereby improving the user experience.

Description

Data construction, code question answering methods, task platform and code question answering system

[0001] This disclosure claims priority to Chinese Patent Application No. 202411649202.9, filed with the China Patent Office on November 18, 2024, entitled “Data Construction, Code Question Answering Method, Task Platform and Code Question Answering System”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the technical field of deep learning, and in particular to a data construction, code question answering method, task platform, and code question answering system. Background Technology

[0003] With the rapid development of software and hardware, the size and complexity of code repositories are constantly increasing, posing a challenge for developers to efficiently utilize these massive code resources. In repository-level code repositories, developers often need to quickly find relevant information from a vast amount of code snippets to solve code repository-related problems or perform secondary development. Therefore, the necessity of building a free-response question-and-answer model for code repositories is becoming increasingly important.

[0004] Currently, when building repository-level code question answering models, general question answering models cannot effectively handle noisy code snippets introduced during the code snippet recall stage, resulting in inaccurate responses. Therefore, data construction for repository-level code question answering models has become crucial.

[0005] However, constructing sample data for training code question-answering models often relies on manual annotation, which demands high levels of expertise from personnel, is costly, and inefficient. While fully automated methods (e.g., large language models and fixed rule templates) can generate sample questions that are weakly correlated with the code repository because they cannot be mapped to the specific details of the code repository, this reduces the quality of the generated sample questions and answers, resulting in poor performance of the trained code question-answering model. Therefore, there is an urgent need for an efficient and high-quality automated method for constructing sample data. Summary of the Invention

[0006] In view of this, embodiments of this disclosure provide a data construction method. One or more embodiments of this disclosure simultaneously relate to a question-answering model training method, a code question-answering method, a task platform, a code question-answering system, a data construction apparatus, a question-answering model training apparatus, a code question-answering apparatus, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art. In one embodiment of this disclosure, a data construction method is provided, comprising:

[0007] The problem is to obtain the first code snippet and the initial sample problem for the first code snippet;

[0008] Using a text processing model, based on the directory structure information of the sample code repository in the code library, the initial sample problem is rewritten to obtain the sample problem;

[0009] Based on the first code snippet and the sample problem, the relevant second code snippet is recalled from the code repository;

[0010] Using a text processing model, sample answers to sample questions are generated based on a second code snippet. The sample questions and sample answers are sample data used to train the code question answering model, which is applicable to code repositories.

[0011] In one embodiment of this disclosure, a text processing model is used to rewrite the initial sample question based on the directory structure information of the sample code repository in the code library, obtaining a highly relevant sample question specific to the sample code repository details. Based on the first code fragment and the highly relevant sample question specific to the sample code repository details, a relevant second code fragment is retrieved from the code library, improving the accuracy of code fragment retrieval. Using the text processing model, based on the high-quality second code fragment, an accurate sample answer corresponding to the sample question is generated, obtaining highly accurate sample questions and sample answers. This achieves automatic construction of high-quality sample data, which not only improves data construction efficiency and reduces data construction costs, but also makes the trained code question answering model more suitable for the code repository in the code library, enabling repository-level, higher-quality code question answering processing adapted to the code repository, thus improving the user experience. Attached Figure Description

[0012] Figure 1 is a flowchart of a data construction method provided in an embodiment of this disclosure;

[0013] Figure 2 is a flowchart illustrating a data construction method according to an embodiment of this disclosure;

[0014] Figure 3 is a flowchart of a training method for a question-answering model provided in an embodiment of this disclosure;

[0015] Figure 4 is a flowchart of a code question-and-answer method provided in an embodiment of this disclosure;

[0016] Figure 5 is a flowchart of a data construction method for training an intelligent coding assistant provided in an embodiment of this disclosure;

[0017] Figure 6 is a schematic diagram of the structure of a task platform provided in an embodiment of this disclosure;

[0018] Figure 7 is a schematic diagram of the structure of a code question-and-answer system provided in an embodiment of this disclosure;

[0019] Figure 8 is a structural block diagram of a computing device provided in an embodiment of this disclosure. Detailed Implementation

[0020] Numerous specific details are set forth in the following description to provide a full understanding of this disclosure. However, this disclosure can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this disclosure. Therefore, this disclosure is not limited to the specific implementations disclosed below.

[0021] The terminology used in one or more embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this disclosure. The singular forms “a,” “the,” and “the” as used in one or more embodiments of this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this disclosure refers to and includes any or all possible combinations of one or more associated listed items.

[0022] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this disclosure, and similarly, second may also be referred to as first. Depending on the context, the word “if” as used herein may be interpreted as “when”, “in response to a determination”, or “when…”.

[0023] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this disclosure are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0024] In one or more embodiments of this disclosure, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability, such as large-scale language models (LLMs) and multimodal pre-training models.

[0025] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as natural language processing tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios for large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0026] First, the terms and concepts involved in one or more embodiments of this disclosure will be explained.

[0027] Intelligent Coding Assistant: Utilizing large-scale language models and artificial intelligence technology, it provides functions such as code completion, code generation, code error detection and correction, and code optimization to help developers reduce repetitive work and achieve significant improvements in programming efficiency and code quality.

[0028] A code chunk is a code snippet within a codebase, typically a code file, a code function, a code class, etc.

[0029] Repository-level code Q&A: Question and answer pairs for the code repository, including questions and answers about the code repository's structure, functions, logic, etc. Common forms include explanatory Q&A (such as explaining the running logic of a functional module) and search-based Q&A (such as introducing the entry class of a specific logic).

[0030] Prompt: A paradigm of information that uses natural language to guide or motivate a model to perform a task.

[0031] Development Issues: Development issues in the codebase are usually questions raised by developers in the codebase, including issue descriptions, issue tags, and issue solutions.

[0032] Pull Request: A pull request in the codebase, usually submitted by a developer, which includes a pull request description, pull request code, pull request tests, etc.

[0033] LangChain framework: A framework for developing language model applications. It simplifies the process of building, deploying, and managing applications based on large language models, and supports various tasks such as question answering and text generation.

[0034] BM25 Recall Algorithm: An information retrieval algorithm used to evaluate the relevance of query terms to documents. It calculates a score by considering factors such as term frequency and document length, and is widely used in search engines to improve the quality of search results.

[0035] This disclosure provides a data construction method, and also relates to a question-answering model training method, a code question-answering method, a task platform, a code question-answering system, a data construction device, a question-answering model training device, a code question-answering device, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0036] Referring to Figure 1, Figure 1 shows a flowchart of a data construction method provided in an embodiment of this disclosure, including the following specific steps:

[0037] Step 102: Obtain the first code snippet and the initial sample problem for the first code snippet.

[0038] This disclosure applies to applications, platforms, or systems that have sample data construction capabilities for training models.

[0039] The first code snippet serves as a reference for retrieving the second code snippet from the codebase. The first code snippet can be a code file, a code function, a code class, or a code variable; no specific limitation is imposed here. For example, a Python code function: `def add(a,b): return a+b`.

[0040] The initial sample question is a natural language question posed to the first code snippet. It is a low-quality, weakly relevant sample question and is not specific to the details of the sample code repository. It can be an explanatory question, such as "How does the add function work?", or a search question, such as "Where can I find the definition of the add function?".

[0041] To obtain the first code snippet, one option is to retrieve the first code snippet from the code repository, and another option is to randomly generate the first code snippet. No specific method is specified here.

[0042] To obtain initial sample questions for the first code snippet, one option is to generate initial sample questions for the first code snippet based on a preset question template. Another option is to generate initial sample questions for the first code snippet based on natural language processing technology. Yet another option is to manually write initial sample questions for the first code snippet. No specific method is specified here.

[0043] For example, on a data processing platform with sample data construction capabilities, 20 first code snippets are retrieved from the code repository:

[0044] The problem of generating initial samples based on natural language processing technology:

[0045] How does the codebase process and query Prometheus metric data?

[0046] In one optional embodiment of this disclosure, step 102, obtaining the initial sample question for the first code segment, includes the following specific steps: using a text processing model to generate the initial sample question for the first code segment.

[0047] Using a text processing model, initial sample questions are generated for the first code snippet. One possible approach is to use the text processing model to generate initial sample questions and corresponding reference answers for the first code snippet. The reference answers are natural language responses generated for the initial sample questions; they are low-quality, weakly relevant sample answers and are not specific to the details of the sample code repository. For example, "The add function is an addition function" could be an explanatory question-and-answer format or a search question-and-answer format, such as "The definition of the add function can be found in a file in the code repository. You can locate the specific location of the function by searching the code repository."

[0048] Correspondingly, one possible approach to step 104 is to retrieve relevant second code snippets from the codebase based on the first code snippet, the sample question, and the reference answer. This refines the retrieval index, improving the accuracy of the retrieved second code snippets.

[0049] For example, the prompt for inputting a large language model is as follows:

[0050] "I need your help to build reasonable user question and answer data pairs."

[0051] 20 first code snippets:

[0052] Based on the above, please raise a reasonable user question and provide a response to it.

[0053] Require:

[0054] 1. The returned answer must be in Markdown format;

[0055] 2. When the answer involves specific files, try to use hyperlinks in the format [name](file: / / path);

[0056] Return format:

[0057] ##question

[0058] Mock question here

[0059] ##reply

[0060] Answer here

[0061] ##Documents involved in the answer

[0062] If a file (such as md, json, xml, yml, etc.) is reasonably inferred to be helpful in answering "user questions", return two or more file names. The file paths here are separated by commas and end with 'a'.

[0063] eg:README.md,tabby ssh / src / index.ts;

[0064] If the answer involves specific code snippets, I would appreciate it if you could return an additional "Source of Answer" field, in the following format:

[0065] ##Answer-related fragments

[0066] The code snippets here use commas to separate IDs and end with 'a';

[0067] The initial sample question for the first code snippet, "How does the codebase process and query Prometheus metric data?", and the corresponding reference answer, "The main functionality for processing and querying Prometheus metric data is implemented through the 'mock_pull_metric' function. This function is asynchronous; it takes a 'PrometheusMetric' object as an argument and returns the data associated with that metric. Specifically, the 'mock_pull_metric' function retrieves the given data from the 'prometheus_data' dictionary."

[0068] In this embodiment of the disclosure, by utilizing a text processing model, initial sample questions for the first code snippet are quickly generated, which significantly improves data construction efficiency, reduces the workload of manually writing sample data, and provides a basis for rewriting sample questions obtained in subsequent rewriting.

[0069] In step 102, the first code fragment is obtained, which provides an index for the subsequent recall of the second code fragment. The initial sample problem for the first code fragment is obtained, which provides a basis for the subsequent rewriting of the sample problem.

[0070] Step 104: Using a text processing model, based on the directory structure information of the sample code repository in the code library, rewrite the initial sample problem to obtain the sample problem.

[0071] A text processing model is a deep learning model capable of processing natural language text. It can understand and generate natural language text and perform text processing tasks such as text classification, named entity recognition, sentence rewriting, and question generation. In this embodiment, the text processing model is used to rewrite an initial sample question and generate sample responses. The text processing model can be a Transformer model, BERT model, diffusion model, generative adversarial model, large-scale language model, GPT model, etc., and is not limited thereto.

[0072] A code repository is a collection of software code repositories (projects). Code repositories are typically hosted on code platforms, allowing developers to view, use, and contribute code. A sample code repository is a code repository within the code repository that serves as a sample reference for building sample data. Sample code repositories have specific details. For example, a Python code repository named "streams explorer" within the code repository. The directory structure information of a sample code repository describes the target structure of the sample code repository.

[0073] The sample questions are natural language questions that rewrite the first code snippet. They are high-quality, highly relevant sample questions specific to code repository details. Compared to the initial sample questions, the sample questions are more specific and accurate. Because they are specific to code repository details, they can be explanation-based questions, such as "How does the add function work in the backend / utils.py file?", or search-based questions, such as "In the streams explorer code repository, in which file is the definition of the add function located?".

[0074] Using a text processing model, the initial sample question is rewritten based on the directory structure information of the sample code repository in the codebase to obtain the sample question. One possible approach is to use the text processing model to rewrite the initial sample question using the directory structure information of the sample code repository as context information. Context information is provided to the text processing model to perform text processing tasks, helping it to better understand and generate text. Furthermore, another possible approach is to use the text processing model to rewrite the initial sample question based on keyword information from the initial sample question, using the directory structure information of the sample code repository as context information, to obtain the sample question.

[0075] Optionally, the initial sample problem is often in Chinese or other languages, while the code snippets are usually written in English characters. Therefore, rewriting the initial sample problem to obtain the sample problem requires language translation.

[0076] For example, the prompt message for inputting a large language model is as follows:

[0077] "You are the person in charge of a code repository."

[0078] When users query this repository, you need to complete the following tasks to help the algorithm retrieve relevant code content:

[0079] 1. Expand user queries appropriately based on the repository file directory structure and translate them into English.

[0080] 2. Generate up to 8 keywords for retrieval.

[0081] 3. Return in JSON format.

[0082] Sample code repository name: streams explorer

[0083] The directory structure of the sample code repository is as follows:

[0084] Your reply should follow the format below:

[0085] Sample question: "Where is the base64 encoded code?"

[0086] Based on the keyword information ("Prometheus") in the initial sample question, and using the above directory structure information as context, the initial sample question ("How does the codebase process and query Prometheus metric data?") is rewritten to obtain the following sample question:

[0087] How does this codebase process and query Prometheus metric data?

[0088] By using a text processing model and based on the directory structure information of the sample code repository in the code library, the initial sample question is rewritten to obtain a sample question that is specific to the details of the code repository and is strongly related to the code repository. This provides a high-quality retrieval basis for the subsequent recall of the second code fragment.

[0089] Step 106: Based on the first code snippet and the sample problem, recall the relevant second code snippet from the code repository.

[0090] The second code snippet is a code snippet retrieved from the codebase that is relevant to the first code snippet and the sample question. The second code snippet is obtained based on the first code snippet and a strongly relevant sample question specific to the sample code repository. The second code snippet is highly relevant to the first code snippet and the sample question, and typically answers the sample question more accurately. For example, the first code snippet is:

[0091] def add(a,b): return a+b

[0092] The sample question is "How does the add function work in the backend / utils.py file?"

[0093] The second code snippet is:

[0094] The second code snippet not only includes the definition of the add function, but also detailed comments and some additional logic (such as type checking), which allows for a more accurate answer to the sample question.

[0095] For example, based on 20 first code snippets and a sample question (“How does this codebase process and query Prometheus metric data?”), 33 relevant second code snippets were retrieved from the codebase:

[0096] Based on the first code snippet and a strongly related sample question specific to the sample code repository, a relevant second code snippet was retrieved from the code repository, improving the accuracy of code snippet retrieval and providing contextual data support for the subsequent generation of sample responses.

[0097] Step 108: Using a text processing model, generate sample answers to sample questions based on the second code snippet. The sample questions and sample answers are sample data used to train the code question-answering model, which is applicable to code repositories.

[0098] The sample response is a natural language answer generated for the sample question. It is a highly relevant, high-quality sample response specific to the sample code repository. The sample response references the second code snippet and provides detailed explanations and relevant information. The sample responses are generated based on the content of the second code snippet, combined with the directory structure information of the sample code repository, to produce specific and accurate answers. For explanatory questions, the sample responses explain the functionality and implementation details of the code. For example, the sample question is "How does the add function work in the backend / utils.py file?", and the sample response is "The 'add' function in the 'backend / utils.py' file is a simple addition function used to add two numbers. The function first checks whether the input parameters 'a' and 'b' are integers or floating-point numbers. If the input is not a valid number type, the function will throw a 'TypeError' exception. Then, the function performs the addition operation and returns the result." For search-related questions, the sample responses show the specific location of the relevant code and other contextual information. For example, the sample question is "In the streams explorer code repository, in which file is the definition of the add function located?", and the sample response is "In the 'streams explorer' code repository, the definition of the 'add' function is located in the 'backend / utils.py' file. The specific path is as follows: streams explorer / backend / init.py core / k8s_app.py metrics.py" The definitions of the `add` function are in the files `init.py`, `streams_explorer.py`, and `utils.py`.

[0099] Code question answering models are deep learning models specifically designed to answer questions about code repositories. They are trained using sample questions and responses as sample data and are capable of understanding and generating natural language responses to code repositories. Code question answering models aim to solve problems developers encounter when dealing with large-scale code repositories, such as code understanding, feature lookup, and location retrieval. Code question answering models can be Transformer models, BERT models, diffusion models, generative adversarial models, large-scale language models, GPT models, etc., and are not limited here.

[0100] It should be noted that the quality of the sample questions and sample answers, which are used as sample data for training the code question answering model, directly affects the performance of the trained code question answering model.

[0101] Using a text processing model, based on the second code snippet, sample answers to sample questions are generated. One possible approach is to use the text processing model, with the second code snippet as context information, to generate sample answers to sample questions. Context information is relevant information provided to the text processing model for performing text processing tasks, helping the model better understand and generate text.

[0102] For example, the prompt message for inputting a large language model is as follows:

[0103] "You are a question-and-answer assistant for code repositories."

[0104] I will provide you with a second code snippet that might be helpful in answering the question, retrieved from the sample question. You will need to answer the user's question based on the given information.

[0105] I will <retrieval>< / retrieval> Given a region, retrieve a second code snippet that might be helpful for question answering. Finally, in <query>< / query> Given a sample problem within a region,

[0106] Let's begin.

[0107] Please take all search results into consideration when responding to user questions;

[0108] Please reply in Chinese;

[0109] The final result is output in Markdown format.

[0110] The content format requirements are as follows; please do not directly output the required text in your answer:

[0111] 1. Please respond to user questions, but avoid excessive elaboration and do not output a directory tree;

[0112] 2. If the answer involves specific code, do not include the code snippet ID. Users are unaware of the code snippet being recalled. A good response format is to provide an appropriately abbreviated version of the code, followed by an explanation of the code.

[0113] 3. If the answer to the question does not require specific code, such as asking about architecture, avoid including example code in the answer;

[0114] Rendering format requirements:

[0115] 1. When the answer content involves specific files, strictly ensure that you use the hyperlink format [filename](file: / / filepath), for example, [user.java](file: / / extensions / firefox / user.java);

[0116] 2. Strictly ensure that the reference file is represented using the format [filename](file: / / filepath), where filename is the filename without the file path, and filepath includes the complete file path!

[0117] 3. If referencing an entire path, ensure it is represented in the format [directory_name](file: / / filepath), such as [firefox](file: / / extensions / firefox / ).

[0118] Using 33 second code snippets ([backend / streams_explorer / core / services / metric_providers.py, backend / tests / test_metricprovider.py;]) as context information, sample answers are generated for the sample question ("How does this codebase process and query Prometheus metric data?").

[0119] "The process of this codebase for processing and querying Prometheus metric data is mainly divided into the following parts:"

[0120] In summary, this codebase efficiently processes and queries Prometheus metrics data by defining the `PrometheusMetric` enumeration class, using asynchronous Hypertext Transfer Protocol (HTTP) requests to query Prometheus data, periodically refreshing the data, and implementing a caching mechanism.

[0121] In this embodiment, a text processing model is used to rewrite the initial sample question based on the directory structure information of the sample code repository in the code library, obtaining a highly relevant sample question specific to the sample code repository details. Based on the first code fragment and the highly relevant sample question specific to the sample code repository details, a relevant second code fragment is retrieved from the code library, improving the accuracy of code fragment retrieval. Using the text processing model, based on the high-quality second code fragment, an accurate sample answer corresponding to the sample question is generated, obtaining highly accurate sample questions and sample answers. This achieves automatic construction of high-quality sample data, which not only improves data construction efficiency and reduces data construction costs, but also makes the trained code question answering model more suitable for the code repository in the code library. It can complete repository-level, higher-quality code question answering processing adapted to the code repository, improving the user experience.

[0122] In one optional embodiment of this disclosure, obtaining the first code fragment in step 102 includes the following specific steps: obtaining metadata information of the code library; and retrieving the relevant first code fragment from the code library based on the metadata information.

[0123] Due to the large number of files and the complexity of the code in the codebase, directly generating sample questions would result in a lack of strong correlation between the generated sample questions and those in the codebase. Therefore, a pseudo-random strategy can be used in the first stage to recall the first code fragment, and then generate sample questions based on this. In the second stage, a high-quality retrieval and recall can be performed, and a high-quality sample response can be generated based on the high-quality second code fragment recalled.

[0124] Metadata information in a codebase is supplementary information related to the codebase. It is typically not included in the actual code files but provides important context and background about the codebase. Metadata helps developers quickly understand the overall state of the codebase and provides necessary input for automation tools such as issue trackers and continuous integration systems. Examples include development issues and pull requests for codebase features.

[0125] One possible way to obtain metadata information from a code repository is to obtain the metadata, parse the metadata, and obtain metadata information. For example, a metadata acquisition tool can be used to obtain the metadata of the code repository, parse out a large amount of metadata information such as development issues and feature iteration requests, and initially recall the relevant first code snippets.

[0126] For example, a metadata acquisition tool is used to acquire the metadata of the code repository, parse the metadata to obtain development issues and feature iteration requests, and based on the development issues and feature iteration requests, retrieve 20 relevant first code snippets from the code repository.

[0127] In this embodiment, the recall strategy based on metadata information not only improves the relevance of sample questions to the actual codebase content, but also ensures that the subsequently constructed sample data is closer to the real development scenario. Obtaining the first code fragment provides an index basis for the subsequent recall of the second code fragment. Addressing the issues of user questions not corresponding to codebase details and low code retrieval accuracy in single-step codebase recall strategies, a multi-step codebase retrieval strategy can generate highly relevant, high-quality sample questions specific to codebase details using a large language model in the first round of code fragment retrieval. Simultaneously, in the second round of code fragment retrieval, various information, including the first code fragment and the rewritten high-quality sample questions, further improves the accuracy of question-related code fragment retrieval, thereby generating more accurate sample responses and effectively improving the data quality of the sample data.

[0128] In one optional embodiment of this disclosure, retrieving a relevant first code segment from a code repository based on metadata information includes the following specific steps: using the code repository's retrieval engine, retrieving the relevant first code segment from the code repository based on metadata information.

[0129] A codebase retrieval engine is a tool for searching and retrieving relevant code snippets within a codebase. Retrieval engines typically combine multiple technologies and algorithms, such as full-text search, semantic analysis, and similarity calculation, to improve search accuracy and efficiency. Within a codebase, a retrieval engine can help developers quickly locate specific feature implementations, bug fixes, configuration settings, etc. For example, a retrieval engine built on the LangChain framework and the BM25 retrieval algorithm.

[0130] Search keywords are information entered into the search engine when performing a search, describing the code snippets you want to retrieve. Search keywords can be single words, combinations of words, phrases, or sentences. The search engine will retrieve code snippets from its codebase based on these keywords and return relevant snippets. The choice of search keywords directly affects the accuracy and relevance of search results. For example, searching for the keyword "Prometheus metrics" to find code snippets related to Prometheus metrics might include code snippets implementing functions such as defining metrics, collecting metric data, and processing metric data.

[0131] One possible approach is to use the code repository's search engine to retrieve relevant first code snippets based on metadata information.

[0132] For example, using the codebase's search engine, the aforementioned development issues and feature iteration requests were used as search keywords to retrieve 20 relevant first code snippets from the codebase.

[0133] In this implementation, by utilizing the code repository's search engine and based on metadata information, relevant first code snippets can be retrieved efficiently and accurately from the massive code repository. This further improves the relevance between the sample problem and the actual code repository content, and also ensures that the sample data constructed subsequently is closer to the real development scenario. The retrieval of the first code snippet provides an index basis for the subsequent retrieval of the second code snippet.

[0134] In one optional embodiment of this disclosure, step 102, obtaining the initial sample question for the first code segment, includes the following specific steps: using a text processing model, based on the directory structure information of the code library, to generate the initial sample question for the first code segment.

[0135] Using a text processing model and based on the directory structure information of the codebase, an initial sample question is generated for the first code snippet. One possible approach is to use the text processing model with the directory structure information of the codebase as context information to generate the initial sample question for the first code snippet. The context information is relevant to the text processing model for performing text processing tasks, helping the model better understand and generate text. Furthermore, another possible approach is to use the text processing model with the directory structure information of the codebase as context information to generate the initial sample question for the first code snippet and the corresponding reference answer.

[0136] For example, the prompt for inputting a large language model is as follows:

[0137] "I need your help to build reasonable user question-and-answer data pairs for a given codebase."

[0138] The code directory structure is as follows:

[0139] Based on the above, please propose a reasonable user question and provide the correct answer to that question.

[0140] Require:

[0141] 1. **User questions must address the structure, functionality, and logic of the current codebase, and must be related to the entered directory;**

[0142] 2. The returned answers must be in Markdown format;

[0143] 3. When the answer involves specific files, try to use hyperlinks in the format [name](file: / / path);

[0144] Return format:

[0145] ##User Issues

[0146] Mock question here

[0147] ##Answer to the Question

[0148] Answer here

[0149] ##Documents involved in the answer

[0150] If a file (such as md, json, xml, yml, etc.) is reasonably inferred to be helpful in answering "user questions", return two or more file names. The file paths here are separated by commas and end with 'a'.

[0151] eg:README.md,tabby ssh / src / index.ts;

[0152] If the answer involves specific code snippets, I would appreciate it if you could return an additional "Source of Answer" field, in the following format:

[0153] ##Answer-related fragments

[0154] The code snippets here use commas to separate IDs and end with 'a';

[0155] Based on the directory structure information of the code repository ( <catalog>{catalog} < / catalog>Using this as contextual information, generate the initial sample question for the first code snippet: "How does the codebase process and query Prometheus metric data?" and the corresponding reference answer: "The main functionality for processing and querying Prometheus metric data is implemented through the 'mock_pull_metric' function. This function is asynchronous; it takes a 'PrometheusMetric' object as an argument and returns the data associated with that metric. Specifically, the 'mock_pull_metric' function retrieves the given data from the 'prometheus_data' dictionary."

[0156] In this embodiment of the disclosure, by utilizing a text processing model based on the directory structure information of the code repository, a weakly related initial sample question that is relatively specific to the details of the sample code repository is quickly generated, which significantly improves the efficiency of data construction, reduces the workload of manually writing sample data, and provides a basis for rewriting the sample question obtained in the future.

[0157] In one optional embodiment of this disclosure, step 104 includes the following specific steps: using a text processing model to extract keyword information from the initial sample question, and rewriting the initial sample question based on the keyword information and the directory structure information of the sample code repository in the code library to obtain the sample question.

[0158] Keyword information is the core content extracted from the initial sample question, representing its core essence. It is fundamental for accurately retrieving the second code snippet and for rewriting the initial sample question. Keyword information helps the text processing model more accurately understand the initial sample question and generate highly relevant sample questions specific to the sample code repository details. Keyword information typically includes nouns, verbs, and technical terms. For example, in the initial sample question "How does the code repository process and query Prometheus metric data?", the keywords include "Prometheus", "metrics_handling", "metrics_querying", "backend", "metrics.py", "core", "metric_providers", and "linking_services".

[0159] Based on keyword information and the directory structure information of the sample code repository in the code library, the initial sample question is rewritten to obtain the sample question. One possible approach is to rewrite the initial sample question based on keyword information and using the directory structure information of the sample code repository in the code library as context information. Context information is provided to the text processing model for performing text processing tasks, helping the text processing model better understand and generate text.

[0160] For example, the prompt message for inputting a large language model is as follows:

[0161] "You are the person in charge of a code repository."

[0162] When users query this repository, you need to complete the following tasks to help the algorithm retrieve relevant code content:

[0163] 1. Expand user queries appropriately based on the repository file directory structure and translate them into English.

[0164] 2. Generate up to 8 keywords for retrieval.

[0165] 3. Return in JSON format.

[0166] Sample code repository name: streams explorer

[0167] The directory structure of the sample code repository is as follows:

[0168] Your reply should follow the format below:

[0169] Sample question: "Where is the base64 encoded code?"

[0170] Keyword information: ["base64_encoding", "base64", "encode"]}

[0171] Extract keyword information ("Prometheus", "metrics_handling", "metrics_querying", "backend", "metrics.py", "core", "metric_providers", "linking_services") from the initial sample question. Based on this keyword information and using the directory structure information of the sample code repository in the codebase as context information, rewrite the initial sample question ("How does the codebase process and query Prometheus metric data?") to obtain the following sample question:

[0172] How does this codebase process and query Prometheus metric data?

[0173] Keyword information: ["Prometheus","metrics_handling","metrics_querying","backend","metrics.py","core","metric_providers","linking_services"].

[0174] In this embodiment of the disclosure, a text processing model is used to extract keyword information from the initial sample question. Based on the keyword information and the directory structure information of the sample code repository in the code library, the initial sample question is rewritten to obtain a more specific, sample code repository-specific, and strongly related sample question, and to provide a higher quality retrieval basis for the subsequent recall of the second code fragment.

[0175] In one optional embodiment of this disclosure, step 106 includes the following specific steps: based on the first code snippet, keyword information, and sample question, recalling the relevant second code snippet from the code library.

[0176] After obtaining highly relevant sample questions specific to the sample code repository, due to the inherent differences between natural language and code language, it may still be impossible to accurately retrieve code fragments, requiring further refined searching.

[0177] For example, based on 20 first code snippets, keyword information ("Prometheus","metrics_handling","metrics_querying","backend","metrics.py","core","metric_providers","linking_services"), and sample questions ("How does this codebase process and query Prometheus metric data?"), 33 relevant second code snippets were retrieved from the codebase.

[0178] Based on the first code snippet, keyword information, and strongly relevant sample questions specific to the sample code repository, relevant second code snippets were retrieved from the code repository, further improving the accuracy of code snippet retrieval and providing contextual data support for the subsequent generation of sample responses.

[0179] In one optional embodiment of this disclosure, the process of recalling a relevant second code segment from a code repository based on a first code segment, keyword information, and a sample question includes the following specific steps: using the code repository's search engine, the relevant second code segment is recalled from the code repository based on the first code segment, keyword information, and a sample question.

[0180] It should be noted that the retrieval engine used in this embodiment can be consistent with the retrieval engine used in subsequent code question answering to ensure that the distribution of noisy code fragments retrieved is consistent with the distribution of noisy code fragments in the actual code question answering. For example, a retrieval engine built on the LangChain framework and the BM25 recall algorithm can be uniformly adopted. This consistency helps to maintain consistent performance and effectiveness in training and practical applications.

[0181] Using the code repository's search engine, based on the first code snippet, keyword information, and sample question, relevant second code snippets are retrieved from the code repository. Using the code repository's search engine, the first code snippet, keyword information, and sample question are used as search keywords to retrieve relevant second code snippets from the code repository.

[0182] For example, using the code repository's search engine, the 33 relevant second code snippets were retrieved from the code repository by using the first code snippet, keyword information, and sample question as search keywords.

[0183] In this embodiment of the disclosure, by utilizing the retrieval engine of the code library, based on the first code fragment, keyword information, and sample question, the relevant second code fragment can be retrieved from the code library efficiently and accurately, achieving high-quality retrieval and further improving the accuracy of code fragment retrieval and retrieval, providing contextual information data support for the subsequent generation of sample responses.

[0184] In one optional embodiment of this disclosure, step 108 includes the following specific steps: using a text processing model, based on the directory structure information of the second code snippet and the sample code repository, to generate a sample answer corresponding to the sample question.

[0185] The directory structure information of the sample code repository describes the target structure of the sample code repository.

[0186] Using a text processing model, based on the directory structure information of the second code snippet and the sample code repository, sample answers to sample questions are generated. One possible approach is to use the text processing model with the directory structure information of the second code snippet and the sample code repository as context information to generate sample answers to sample questions. Context information is relevant information provided to the text processing model for performing text processing tasks, helping the model better understand and generate text.

[0187] For example, the prompt for inputting a large language model is as follows:

[0188] "You are a question-and-answer assistant for code repositories."

[0189] I will provide you with a user question for a sample code repository, along with the repository's directory structure (catalog) and environment dependencies as contextual information, as well as a second code snippet retrieved from the user question that might be helpful in answering it. You need to answer the user's question based on the given information.

[0190] I will <catalog>< / catalog> Given the directory structure of the sample code repository in the region, <requirements>< / requirements> The environment dependencies of a given sample code repository within the region, <retrieval>< / retrieval> Given a region, retrieve a second code snippet that might be helpful for question answering. Finally, in <query>< / query> Given a user question within the region,

[0191] Let's begin.

[0192] Please take into account the sample code repository directory, environment dependencies, and search results when responding to user questions;

[0193] Please reply in Chinese;

[0194] The final result is output in Markdown format.

[0195] The content format requirements are as follows; please do not directly output the required text in your answer:

[0196] 1. Please respond to user questions, but avoid excessive elaboration and do not output a directory tree;

[0197] 2. If the answer involves specific code, do not include the code snippet ID. Users are unaware of the code snippet being recalled. A good response format is to provide an appropriately abbreviated version of the code, followed by an explanation of the code.

[0198] 3. If the answer to the question does not require specific code, such as asking about architecture, avoid including example code in the answer;

[0199] Rendering format requirements:

[0200] 1. When the answer content involves specific files, strictly ensure that you use the hyperlink format [filename](file: / / filepath), for example, [user.java](file: / / extensions / firefox / user.java);

[0201] 2. Strictly ensure that the reference file is represented using the format [filename](file: / / filepath), where filename is the filename without the file path, and filepath includes the complete file path!

[0202] 3. If referencing an entire path, ensure it is represented in the format [directory_name](file: / / filepath), such as [firefox](file: / / extensions / firefox / ).

[0203] Using 33 second code snippets ([backend / streams_explorer / core / services / metric_providers.py, backend / tests / test_metricprovider.py;]) and the directory structure information of the sample code repository ( <catalog> Directory Structure:{catelog}< / catalog> Using this as contextual information, generate sample answers to the sample question ("How does this codebase process and query Prometheus metric data?").

[0204] "The process of this codebase for processing and querying Prometheus metric data is mainly divided into the following parts:"

[0205] In summary, this codebase efficiently processes and queries Prometheus metrics data by defining the `PrometheusMetric` enumeration class, using asynchronous Hypertext Transfer Protocol (HTTP) requests to query Prometheus data, periodically refreshing the data, and implementing a caching mechanism.

[0206] In this embodiment of the disclosure, a text processing model is used to generate accurate sample answers to sample questions based on the directory structure information of the second code snippet and the sample code repository, which are specific to the details of the sample code repository. This results in highly accurate sample questions and sample answers, and enables the automatic construction of high-quality sample data.

[0207] Figure 2 shows a flowchart illustrating a data construction method according to an embodiment of this disclosure, as shown in Figure 2:

[0208] For the code repository, based on the repository's metadata, relevant first code snippets are retrieved. A large language model generates initial sample questions and answers for the first code snippets. The large language model extracts keyword information from the initial sample questions and rewrites them to obtain sample questions. Based on the first code snippets, keyword information, and sample questions, relevant second code snippets are retrieved from the code repository. Reference code snippets are obtained from the code repository. The large language model generates sample answers corresponding to the sample questions. Sample data is constructed to train a code question-answering model suitable for the code repository.

[0209] As shown in Figure 2, the data construction method for repository-level code question answering based on multi-step code repository retrieval can construct high-quality code question answering sample data specific to code repositories by leveraging a general large language model and multi-stage repository code fragment retrieval and retrieval. This not only reduces the cost of manual annotation and saves the cost of annotating sample data, but also solves the problems of low relevance of sample questions and omission of retrieved code fragments in single-step retrieval strategies, thereby improving the quality of repository-level code question answering data and ensuring the diversity of sample data and the quality of code question answering.

[0210] Through the above-described data construction method, high-quality sample data for training the code question-answering model was obtained. Correspondingly, Figure 3 shows a flowchart of a question-answering model training method provided in an embodiment of this disclosure, including the following specific steps:

[0211] Step 302: Obtain sample questions and sample answers, wherein the sample questions and sample answers are constructed according to the data construction method described above.

[0212] The embodiments disclosed herein are applicable to applications, platforms, or systems with model training capabilities.

[0213] The embodiments disclosed herein belong to the same concept as the data construction method described above. For details not described in detail in step 302, please refer to the description of the technical solution of the data construction method described above, and they will not be repeated here.

[0214] Obtaining sample questions and sample answers provides sample data support for training the code question-answering model.

[0215] Step 304: Using the initial question-answering model, generate predicted responses to the sample questions.

[0216] The initial question-answering model is a pre-trained general-purpose model that already possesses certain text understanding and generation capabilities, but it has not yet been trained for code-based question answering in code repositories. The initial question-answering model can be a Transformer model, BERT model, diffusion model, generative adversarial model, large-scale language model, GPT model, etc., and is not limited here.

[0217] The predicted response is a natural language answer generated by the initial question-answering model based on the input sample question. The predicted response may differ from the actual sample response. By comparing the predicted response with the sample response, the performance of the initial question-answering model can be measured, and the model can be trained in a targeted manner to gradually improve its accuracy.

[0218] For example, the sample question is: "How does this codebase process and query Prometheus metric data?". Inputting this sample question into a general question-answering model generates a predicted answer: "The `mock_pull_metric` function is used to process and query Prometheus metric data. This function retrieves data related to a given PrometheusMetric object from the `prometheus_data` dictionary."

[0219] Step 306: Based on the predicted response and sample response, train the initial question-answering model to obtain a code question-answering model suitable for code repositories in the code library.

[0220] Based on predicted and sample responses, an initial question-answering model is trained to obtain a code question-answering model suitable for the code repository in the codebase. One possible approach is to calculate the loss value based on the predicted and sample responses, adjust the model parameters of the initial question-answering model based on the loss value, and obtain a code question-answering model suitable for the code repository in the codebase when the training termination condition is met. The loss value can be cross-entropy loss, mean squared error loss, Kullback-Leibler (KL) divergence, etc., and the training termination condition can be a preset number of iterations, a loss threshold, or when performance on the validation set no longer improves.

[0221] For example, the sample response is: "The process of processing and querying Prometheus metric data in this codebase is mainly divided into the following parts: ... In general, this codebase efficiently processes and queries Prometheus metric data by defining the PrometheusMetric enumeration class, using asynchronous hypertext transfer protocol (HTTP) to request and query Prometheus data, periodically refreshing the data, and implementing a caching mechanism." The predicted response is: "The mock_pull_metric function is used to process and query Prometheus metric data. This function retrieves data related to a given PrometheusMetric object from the prometheus_data dictionary." Based on the predicted response and the sample response, the cross-entropy loss value is calculated. Based on the cross-entropy loss value, the model parameters of the general question answering model are adjusted. Under the condition of satisfying the preset number of iterations, a code question answering model suitable for the code repository "streams explorer" in the codebase is obtained.

[0222] In this embodiment of the disclosure, high-quality sample data pre-constructed using a data construction method is used to train a general initial question-answering model, thereby obtaining a code question-answering model suitable for code repositories in the code library. This model can perform repository-level, higher-quality code question-answering processing adapted to the code repository, thus improving the user experience.

[0223] Through the above-described training method for the question-answering model, a code question-answering model suitable for code repositories in the code library was trained. Correspondingly, Figure 4 shows a flowchart of a code question-answering method provided in an embodiment of this disclosure, including the following specific steps:

[0224] Step 402: Obtain user questions for the target code.

[0225] This disclosure applies to applications, platforms, or systems that feature code-based question-and-answer functionality. Examples include integrated development environments (IDEs) that deploy APIs (Application Programming Interfaces) with code-based question-and-answer models, code hosting platforms, and code documentation generation and maintenance platforms.

[0226] The target code is the specific code that the user wants to understand or query; it can be a code function, code class, code module, or other code unit. The target code may be located in a code file within a codebase. The user may want to understand the functionality, usage, implementation details, etc., of the target code by asking questions. The user's question is a natural language question posed to the target code.

[0227] Exemplarily, the user question for the target code (mock_pull_metric function) sent by the receiving terminal is "How does the mock_pull_metric function process Prometheus metric data?"

[0228] Obtaining the user question for the target code provides a retrieval basis for subsequent recall of target code snippets and data support for subsequent generation of model responses.

[0229] Step 404: Based on the user question, recall relevant target code snippets from the code repositories in the code library.

[0230] The target code snippets are relevant code segments recalled from the code repositories in the code library. The target code snippets are closely related to the user's question and can be code files, code functions, code classes, or code variables, which are not limited here.

[0231] Based on the user question, to recall relevant target code snippets from the code repositories in the code library, an optional method is: using the retrieval engine of the database, with the user question as the retrieval keyword, recall relevant target code snippets from the code repositories in the code library.

[0232] Exemplarily, using the retrieval engine of the code library, with the user question as the retrieval keyword, recall 10 relevant target code snippets from the'streams explorer' code repository:

[0233] [backend / tests / test_metricprovider.py]:

[0234] <Snippet 1>< / Snippet 1>……<Snippet 10>< / Snippet 10>

[0235] Based on the user question, recalling relevant target code snippets from the code repositories in the code library provides context information for subsequent code Q&A.

[0236] Step 406: Using a pre-trained code Q&A model, generate a model response corresponding to the user question based on the target code snippets, where the code Q&A model is trained according to the above training method of the Q&A model.

[0237] The model response is a natural language response generated for the user question, which is a specific, strongly relevant, and high-quality response regarding the details of the code repository. The model response refers to the target code snippets and provides detailed explanations and relevant information. The model response generates a specific and accurate response based on the content of the target code snippets. For explanatory Q&A, the model response explains the function and implementation details of the code.

[0238] This embodiment of the invention is based on the same concept as the training method of the question-answering model described above. For details not described in detail in step 406, please refer to the description of the technical solution of the training method of the question-answering model described above, and they will not be repeated here.

[0239] For example, the prompt for inputting a large language model is:

[0240] "You are a question-and-answer assistant for code repositories."

[0241] I will provide you with a second code snippet that might be helpful in answering the question, retrieved from the sample question. You will need to answer the user's question based on the given information.

[0242] I will <retrieval>< / retrieval> Given a region, retrieve a second code snippet that might be helpful for question answering. Finally, in <query>< / query> Given a sample problem within a region,

[0243] Let's begin.

[0244] Please take all search results into consideration when responding to user questions;

[0245] Please reply in Chinese;

[0246] The final result is output in Markdown format.

[0247] The content format requirements are as follows; please do not directly output the required text in your answer:

[0248] 1. Please respond to user questions, but avoid excessive elaboration and do not output a directory tree;

[0249] 2. If the answer involves specific code, do not include the code snippet ID. Users are unaware of the code snippet being recalled. A good response format is to provide an appropriately abbreviated version of the code, followed by an explanation of the code.

[0250] 3. If the answer to the question does not require specific code, such as asking about architecture, avoid including example code in the answer;

[0251] Rendering format requirements:

[0252] 1. When the answer content involves specific files, strictly ensure that you use the hyperlink format [filename](file: / / filepath), for example, [user.java](file: / / extensions / firefox / user.java);

[0253] 2. Strictly ensure that the reference file is represented using the format [filename](file: / / filepath), where filename is the filename without the file path, and filepath includes the complete file path!

[0254] 3. If referencing an entire path, ensure it is represented in the format [directory_name](file: / / filepath), such as [firefox](file: / / extensions / firefox / ).

[0255] Using 10 target code snippets as context, generate a user question ("How does the mock_pull_metric function process Prometheus metric data?").

[0256] The corresponding model response:

[0257] The `mock_pull_metric` function is an asynchronous function used to process and query Prometheus metric data. It takes a `PrometheusMetric` object as an argument and returns the data associated with that metric.

[0258] Specifically, the `mock_pull_metric` function retrieves the metric data corresponding to the given `PrometheusMetric` object from the `prometheus_data` dictionary. The function definition is as follows:

[0259] async def mock_pull_metric(_,metric:PrometheusMetric):

[0260] return prometheus_data[metric.metric]

[0261] parameter:

[0262] _: Placeholder parameter, usually not used.

[0263] metric: A PrometheusMetric object containing the name of the metric to be queried.

[0264] Return value:

[0265] Returns the value in the prometheus_data dictionary corresponding to the key metric.metric.

[0266] The PrometheusMetric class is defined as follows:

[0267] property:

[0268] name: The name of the indicator.

[0269] metric: The full name of the metric, in the format {name}_metric.

[0270] In this way, the `mock_pull_metric` function can simulate the process of retrieving metric data from Prometheus and return the corresponding data.

[0271] In this embodiment of the disclosure, a code question-answering model is trained using high-quality sample data pre-built through a data construction method. The model generates model answers corresponding to user questions based on target code snippets, thereby completing repository-level, higher-quality code question-answering processing adapted to the code repository and improving the user experience.

[0272] The following description, in conjunction with Figure 5, uses the application of the data construction method provided in this disclosure in training an intelligent coding assistant as an example to further illustrate the data construction method described above. Figure 5 shows a flowchart of a data construction method for training an intelligent coding assistant according to an embodiment of this disclosure, including the following specific steps:

[0273] Step 502: Obtain development issues and feature iteration requests from open-source code repositories.

[0274] Step 504: Using the retrieval engine of the open-source database built with the LangChain framework and the BM25 recall algorithm, retrieve relevant first code snippets from the open-source code repository using development issues and feature iteration requests as search keywords.

[0275] Step 506: Using the large language model and the directory structure information of the open-source code repository as context information, generate initial sample questions and reference answers for the first code snippet, under the constraint that the length of the first code snippet meets the input limit of the large language model.

[0276] Step 508: Using a large language model, extract keyword information from the initial sample question, and based on the keyword information, use the directory structure information of the sample code repository in the open source code library as context information to rewrite the initial sample question to obtain the sample question.

[0277] Step 510: Using the retrieval engine of the open-source database built with the LangChain framework and the BM25 recall algorithm, retrieve relevant second code snippets from the open-source code repository using the first code snippet, keyword information, sample question, and reference answer as search keywords.

[0278] Step 512: Using the large language model, under the constraint that the length of the second code snippet meets the input limit of the large language model, the sample answer corresponding to the sample question is generated using the directory structure information of the second code snippet and the sample code repository as context information. The sample question and sample answer are sample data used to train the intelligent coding assistant. The intelligent coding assistant is applicable to the code repository in the open source code library.

[0279] In this embodiment, compared to the problems of high data annotation costs, small data scale, and difficulty in guaranteeing data quality in manual data annotation schemes, a multi-step code repository retrieval strategy is used to gradually generate code repository-related sample question-answer pairs, which can efficiently and with high quality obtain a large amount of repository-level code question-answer data. Addressing the issues of sample questions not corresponding to code repository details and low code retrieval accuracy in single-step code repository retrieval strategies, the multi-step code repository retrieval strategy can generate initial sample questions specific to code repository details using a large language model in the first round of code snippet retrieval. Simultaneously, in the second round of code snippet retrieval, various information such as keyword information of the generated questions and directory structure information of the code repository are used to overcome the inherent differences between code language and natural language, further improving the accuracy of question-related code snippet retrieval and thus generating more accurate sample answers. This effectively improves the data quality of the sample data constructed for training the intelligent coding assistant.

[0280] Corresponding to the above method embodiments, this disclosure also provides a task platform embodiment. Figure 6 shows a schematic diagram of the structure of a task platform provided in one embodiment of this disclosure. As shown in Figure 6, the task platform 600 includes a sample data construction interface 602 and a first response unit 604;

[0281] The sample data construction interface 602 is used to receive sample data construction requests sent by the terminal, wherein the sample data construction requests carry metadata information of the code library;

[0282] The first response unit 604 is configured to: recall a relevant first code segment from the code repository based on metadata information; generate an initial sample question for the first code segment using a text processing model; rewrite the initial sample question based on the directory structure information of the sample code repository in the code repository using the text processing model to obtain a sample question; recall a relevant second code segment from the code repository based on the first code segment and the sample question; and generate a sample answer corresponding to the sample question based on the second code segment using the text processing model. The sample question and the sample answer are sample data used to train a code question-answering model, which is applicable to code repositories.

[0283] Task Platform 600 is a comprehensive virtual platform for automating the construction of high-quality sample data and training code question-answering models. It receives requests from terminals, processes these requests, and generates corresponding sample data or trained models.

[0284] The sample data construction interface 602 is a module in the task platform 600, responsible for receiving sample data construction requests from the terminal and passing these requests to the first response unit 604 for processing. The first response unit 604 is the core processing module in the task platform 600, responsible for performing a series of operations based on metadata information, including code snippet retrieval, sample question generation, sample question rewriting, sample response generation, and model training.

[0285] In this embodiment, the first response unit uses a text processing model to rewrite the initial sample question based on the directory structure information of the sample code repository in the code library, thereby obtaining a highly relevant sample question specific to the details of the sample code repository. Based on the first code fragment and the highly relevant sample question specific to the details of the sample code repository, a relevant second code fragment is retrieved from the code library, improving the accuracy of code fragment retrieval. Using the text processing model, based on the high-quality second code fragment, an accurate sample answer corresponding to the sample question is generated, obtaining highly accurate sample questions and sample answers. This achieves automatic construction of high-quality sample data, which not only improves data construction efficiency but also reduces data construction costs.

[0286] Optionally, the task platform 600 also includes a model training interface;

[0287] The model training interface is used to receive model training requests sent by the terminal.

[0288] The first response unit 604 is also used to respond to the model training request, generate a predicted answer to the sample question using the initial question-answering model, and train the initial question-answering model based on the predicted answer and the sample answer to obtain a code question-answering model suitable for the code repository.

[0289] The model training interface is a module in the task platform 600, which is responsible for receiving model training requests from the terminal and passing these requests to the first response unit 604 for processing.

[0290] In this embodiment of the disclosure, the first response unit trains a general initial question-and-answer model based on high-quality sample questions and sample answers to obtain a code question-and-answer model suitable for code repositories in the code library. This model can perform repository-level, higher-quality code question-and-answer processing adapted to the code repository, thereby improving the user experience.

[0291] The above is an illustrative scheme of a task platform according to this embodiment. It should be noted that the technical solution of this task platform belongs to the same concept as the technical solutions of the data construction method and the question-answering model training method described above. For details not described in detail in the technical solution of the task platform, please refer to the description of the technical solutions of the data construction method or the question-answering model training method described above.

[0292] Corresponding to the above method embodiments, this disclosure also provides a code question-and-answer system embodiment. Figure 7 shows a schematic diagram of the structure of a code question-and-answer system provided in one embodiment of this disclosure. As shown in Figure 7, the code question-and-answer system 700 includes a question-and-answer interface 702 and a second response unit 704;

[0293] The Q&A interface 702 is used to receive code Q&A requests sent by the terminal, wherein the code Q&A requests carry user questions about the target code;

[0294] The second response unit 704 is used to recall relevant target code snippets from the code repository in the code library based on user questions, and generate model answers corresponding to user questions based on target code snippets using a pre-trained code question answering model. The code question answering model is trained according to the training method of the question answering model mentioned above.

[0295] The Q&A interface 702 is also used to send the model's response back to the terminal.

[0296] The Code Question Answering System 700 is an intelligent question-answering system for receiving code-related questions from users. It can generate detailed answers based on a pre-trained code question answering model. The system can retrieve relevant code snippets from a code repository and use these snippets as contextual information to generate accurate responses.

[0297] The question-and-answer interface 702 is a module in the code question-and-answer system 700, which is responsible for receiving code question-and-answer requests from the terminal and feeding back the generated model answers to the terminal.

[0298] The second response unit 704 is the core processing module in the code question answering system 700. It is responsible for recalling relevant target code snippets from the code library based on user questions and generating model answers corresponding to user questions using a pre-trained code question answering model.

[0299] In this embodiment of the disclosure, the second response unit retrieves relevant target code snippets from the code repository based on the user's question, and uses a pre-trained code question-and-answer model with the target code snippets as context information to generate a model answer corresponding to the user's question. This completes repository-level, higher-quality code question-and-answer processing adapted to the code repository, thereby improving the user experience.

[0300] The above is an illustrative scheme of a code question-answering system according to this embodiment. It should be noted that the technical solution of this code question-answering system and the technical solution of the code question-answering method described above belong to the same concept. For details not described in detail in the technical solution of the code question-answering system, please refer to the description of the technical solution of the code question-answering method described above.

[0301] Corresponding to the above method embodiments, this disclosure also provides a data construction apparatus embodiment, the apparatus comprising:

[0302] The first acquisition module is configured to acquire the first code snippet and the initial sample problem for the first code snippet;

[0303] The first rewriting module is configured to use a text processing model to rewrite the initial sample problem based on the directory structure information of the sample code repository in the code library, thereby obtaining the sample problem.

[0304] The first recall module is configured to recall relevant second code snippets from the codebase based on the first code snippet and the sample problem;

[0305] The first generation module is configured to use the text processing model to generate a sample answer corresponding to the sample question based on the second code snippet, wherein the sample question and the sample answer are sample data used to train the code question answering model, and the code question answering model is applicable to code repositories.

[0306] Optionally, the first acquisition module is further configured to: acquire metadata information of the code repository; and, based on the metadata information, retrieve the relevant first code segment from the code repository.

[0307] Optionally, the first acquisition module is further configured to: use the code repository's retrieval engine to retrieve relevant first code snippets from the code repository based on metadata information.

[0308] Optionally, the first acquisition module is further configured to: use a text processing model to generate an initial sample question for the first code snippet based on the directory structure information of the code repository.

[0309] Optionally, the first rewriting module is further configured to: use a text processing model to extract keyword information from the initial sample question, and rewrite the initial sample question based on the keyword information and the directory structure information of the sample code repository in the code library to obtain the sample question.

[0310] Optionally, the first recall module is further configured to: recall relevant second code snippets from the code repository based on the first code snippet, keyword information, and sample question.

[0311] Optionally, the first recall module is further configured to: use the code repository's retrieval engine to recall relevant second code snippets from the code repository based on the first code snippet, keyword information, and sample questions.

[0312] Optionally, the first generation module is further configured to: use a text processing model to generate sample answers corresponding to sample questions based on the directory structure information of the second code snippet and the sample code repository.

[0313] The above is an illustrative scheme of a data construction apparatus according to this embodiment. It should be noted that the technical solution of this data construction apparatus and the technical solution of the data construction method described above belong to the same concept. For details not described in detail in the technical solution of the data construction apparatus, please refer to the description of the technical solution of the data construction method described above.

[0314] Corresponding to the above method embodiments, this disclosure also provides an embodiment of a training device for a question-answering model, the device comprising:

[0315] The second acquisition module is configured to acquire sample questions and sample responses, wherein the sample questions and sample responses are constructed based on the aforementioned data construction device;

[0316] The second generation module is configured to use the initial question-answering model to generate predicted answers to the sample questions.

[0317] The second training module is configured to train the initial question-answering model based on predicted and sample responses, thereby obtaining a code question-answering model suitable for code repositories in the codebase.

[0318] The above is a schematic scheme of a question-answering model training device according to this embodiment. It should be noted that the technical solution of this question-answering model training device and the technical solution of the question-answering model training method described above belong to the same concept. For details not described in detail in the technical solution of the question-answering model training device, please refer to the description of the technical solution of the question-answering model training method described above.

[0319] Corresponding to the above method embodiments, this disclosure also provides an embodiment of a code question-and-answer device, which includes:

[0320] The third acquisition module is configured to acquire user questions targeting the target code;

[0321] The third recall module is configured to recall relevant target code snippets from the code repository based on user questions;

[0322] The third generation module is configured to use a pre-trained code question-answering model to generate a model answer to a user's question, with the target code snippet as context information. The code question-answering model is trained using the training device for the question-answering model.

[0323] The above is an illustrative scheme of a code question-and-answer device according to this embodiment. It should be noted that the technical solution of this code question-and-answer device and the technical solution of the code question-and-answer method described above belong to the same concept. For details not described in detail in the technical solution of the code question-and-answer device, please refer to the description of the technical solution of the code question-and-answer method described above.

[0324] Figure 8 shows a structural block diagram of a computing device according to an embodiment of the present disclosure. The computing device 800 includes:

[0325] Memory 810 and processor 820;

[0326] The memory 810 is used to store computer programs / instructions, and the processor 820 is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor 820, they implement the steps of the above-mentioned data construction method, question-answering model training method, or code question-answering method.

[0327] In one or more embodiments of this disclosure, the computing device can be understood as an integrated smart terminal, including but not limited to a server, desktop computer, PC (Personal Computer), all-in-one model machine, mobile phone, tablet computer or other portable smart terminal, etc., and the computing device may have the model described in the above embodiments of this application pre-installed.

[0328] Specifically, this computing device can pre-install various types of models, including but not limited to models in natural language processing, visual processing, speech processing, code processing, and multimodal task processing, thus providing diverse model selection. In different product forms, this computing device can support one or more model usage methods, including but not limited to model training, model invocation, model fine-tuning, model deployment, model inference, and application. In some product forms, this computing device also supports model management, including but not limited to multi-type model management (supporting the management of discriminative, generative, and other model types), model version control (supporting the control of different model versions), and model evaluation (evaluating model performance and effectiveness based on model evaluation tools). In other product forms, this computing device can also create applications based on models, providing API calling capabilities, allowing models to be called into created applications through API interfaces, and providing application management tools for application management and monitoring.

[0329] Furthermore, the computing device may also include data management (supporting the creation and management of model tuning datasets), a training center (providing abundant training resources to help users learn and master AI technology), and basic control capabilities (providing enterprise-level basic control capabilities to ensure the security and efficient operation of the system). Through the above functions, it provides a comprehensive and integrated device for AI development, training, deployment, and application.

[0330] Furthermore, the components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 and the memory 810 can be connected via a bus.

[0331] The computing device 800 may also include an access device that enables the computing device 800 to communicate with a database storing data via one or more networks. Examples of such networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Worldwide Interoperability for Microwave Access (WiMAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0332] In one embodiment of this disclosure, the aforementioned components of the computing device 800, as well as other components not shown in FIG. 8, may be interconnected, for example, via a bus. It should be understood that the computing device block diagram shown in FIG. 8 is merely for illustrative purposes and is not intended to limit the scope of this disclosure. Those skilled in the art can add or replace other components as needed.

[0333] The computing device 800 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 800 can also be a mobile or stationary server.

[0334] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device belongs to the same concept as the technical solutions of the data construction method, the question-answering model training method, and the code question-answering method described above. For details not described in detail in the technical solution of the computing device, please refer to the descriptions of the technical solutions of the data construction method, the question-answering model training method, or the code question-answering method described above.

[0335] An embodiment of this disclosure also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described data construction method, question-answering model training method, or code question-answering method.

[0336] An embodiment of this disclosure also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the above-described data construction method, question-answering model training method, or code question-answering method.

[0337] The above is an illustrative embodiment of a computer-readable storage medium and a computer program product according to this invention. It should be noted that the technical solutions of this computer-readable storage medium and computer program product belong to the same concept as the technical solutions of the methods described above. Details not described in detail in the technical solutions of the computer-readable storage medium and computer program product can be found in the descriptions of the technical solutions of the methods described above.

[0338] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0339] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0340] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this disclosure are not limited to the described order of actions, because according to the embodiments of this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this disclosure.

[0341] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0342] The preferred embodiments disclosed above are merely illustrative of this disclosure. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments of this disclosure. These embodiments are selected and specifically described in this disclosure to better explain the principles and practical applications of the embodiments of this disclosure, thereby enabling those skilled in the art to better understand and utilize this disclosure. This disclosure is limited only by the claims and their full scope and equivalents.

Claims

1. A data construction method, comprising: The problem of obtaining the first code snippet and the initial sample problem for the first code snippet; Using a text processing model, the initial sample problem is rewritten based on the directory structure information of the sample code repository in the code library to obtain the sample problem; Based on the first code snippet and the sample problem, a relevant second code snippet is recalled from the code repository; Using the text processing model, a sample answer corresponding to the sample question is generated based on the second code snippet. The sample question and the sample answer are sample data used to train the code question answering model, which is applicable to code repositories.

2. The method according to claim 1, wherein obtaining the first code fragment includes: Retrieve metadata information from the code repository; Based on the metadata information, the relevant first code segment is retrieved from the code repository.

3. The method according to claim 2, wherein retrieving the relevant first code segment from the code repository based on the metadata information comprises: Using the retrieval engine of the code repository, relevant first code snippets are retrieved from the code repository based on the metadata information.

4. The method according to any one of claims 1-3, wherein obtaining the initial sample problem for the first code segment includes: Using a text processing model, based on the directory structure information of the code repository, an initial sample question is generated for the first code snippet.

5. The method according to any one of claims 1-3, characterized in that, The step of obtaining an initial sample for the first code snippet includes: Using a text processing model, generate an initial sample question and a corresponding reference answer for the first code snippet; The reference response is a natural language response generated for the initial sample question, used to provide supplementary retrieval basis for the subsequent recall of the second code fragment.

6. The method according to any one of claims 1-5, wherein the step of rewriting the initial sample problem based on the directory structure information of the sample code repository in the code library using a text processing model to obtain the sample problem includes: Using a text processing model, keyword information is extracted from the initial sample question. Based on the keyword information and the directory structure information of the sample code repository in the code library, the initial sample question is rewritten to obtain the sample question.

7. The method according to claim 6, wherein recalling a relevant second code segment from the code repository based on the first code segment and the sample problem comprises: Based on the first code snippet, the keyword information, and the sample question, a relevant second code snippet is recalled from the code library.

8. The method according to claim 7, wherein recalling a relevant second code segment from the code library based on the first code segment, the keyword information, and the sample question comprises: Using the retrieval engine of the code library, based on the first code fragment, the keyword information, and the sample question, a relevant second code fragment is retrieved from the code library.

9. The method according to any one of claims 1-8, wherein generating a sample answer corresponding to the sample question based on the second code fragment using the text processing model comprises: Using the text processing model, based on the directory structure information of the second code snippet and the sample code repository, a sample answer corresponding to the sample question is generated.

10. The method according to claim 1, characterized in that, The step of recalling a relevant second code snippet from the codebase based on the first code snippet and the sample problem includes: The first code snippet and the sample problem are fused to obtain fused features; Based on the fusion features, the relevant second code snippet is recalled from the code library.

11. The method according to any one of claims 1-11, characterized in that, Before generating the sample answer corresponding to the sample question based on the second code snippet using the text processing model, the method further includes: The second code snippet is deduplicated and / or sorted. The sorting process is based on at least one of the following: relevance to the sample problem, completeness of the code snippet, or complexity of the code snippet.

12. The method according to any one of claims 1-12, characterized in that, The method further includes: The quality of the generated sample questions and sample responses is assessed. Based on the evaluation results, high-quality sample data were selected for training the code question-answering model.

13. A method for training a question-answering model, comprising: Obtain sample questions and sample responses, wherein the sample questions and sample responses are constructed using the data construction method according to any one of claims 1-8; Using the initial question-answering model, generate predicted answers to the sample questions; Based on the predicted response and the sample response, the initial question-answering model is trained to obtain a code question-answering model suitable for code repositories in the code library.

14. A code question-and-answer method, comprising: Obtain user questions related to the target code; Based on the user's question, the relevant target code snippets are retrieved from the code repository in the code library; Using a pre-trained code question-answering model, a model answer corresponding to the user's question is generated based on the target code fragment, wherein the code question-answering model is trained according to the training method of the question-answering model according to claim 13.

15. A task platform, comprising a sample data construction interface and a first response unit; The data construction interface is used to receive sample data construction requests sent by the terminal, wherein... The sample data construction request carries metadata information from the codebase; The first response unit is configured to: recall a relevant first code segment from the code repository based on the metadata information; generate an initial sample question for the first code segment using a text processing model; rewrite the initial sample question based on the directory structure information of the sample code repository in the code repository using the text processing model to obtain a sample question; recall a relevant second code segment from the code repository based on the first code segment and the sample question; and generate a sample answer corresponding to the sample question based on the second code segment using the text processing model. The sample question and the sample answer are sample data used to train a code question-answering model, and the code question-answering model is applicable to code repositories.

16. The task platform according to claim 15 further includes a model training interface; The model training interface is used to receive model training requests sent by the terminal. The first response unit is further configured to respond to the model training request by generating a predicted response to the sample question using the initial question-answering model, and training the initial question-answering model based on the predicted response and the sample response to obtain a code question-answering model suitable for the code repository.

17. A code question-answering system, comprising a question-answering interface and a second response unit; The question-and-answer interface is used to receive code-related question-and-answer requests sent by the terminal. The code Q&A request carries user questions regarding the target code; The second response unit is configured to retrieve relevant target code snippets from the code repository in the code library based on the user question, and generate a model answer corresponding to the user question based on the target code snippets using a pre-trained code question answering model, wherein the code question answering model is trained according to the question answering model training method of claim 13; The question-and-answer interface is also used to send the model's response back to the terminal.

18. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 14.

19. A computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 14.

20. A computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 14.

Images