Data construction, code question answering methods, task platform and code question answering system

Abstract

The embodiments of this specification provide data construction, code question and answer methods, task platforms, and code question and answer systems, wherein the data construction method includes: obtaining a first code snippet and an initial sample question for the first code snippet; using a text processing model, using the directory structure information of a sample code warehouse in a code library as context information, rewriting the initial sample question to obtain a sample question; based on the first code snippet and the sample question, recalling a related second code snippet from the code library; using a text processing model, using the second code snippet as context information, generating a sample answer corresponding to the sample question, wherein the sample question and the sample answer are sample data for training a code question and answer model, and the code question and answer model is applicable to a code warehouse. The efficiency of data construction is improved, the cost of data construction is reduced, and the trained model is adapted to the code warehouse, which improves the user experience.

Description

Technical Field

[0001] The embodiments of this specification relate to the technical field of deep learning, and more particularly to a data construction, code question and answer method, task platform and code question and answer system. Background Art

[0002] With the rapid development of software and hardware development, the size and complexity of code bases are increasing, and developers are faced with the challenge of how to efficiently use these huge code resources. In a repository-level code base, developers often need to quickly find relevant information in a large number of code snippets to solve code base-related problems or perform secondary development. The necessity of building a free question-and-answer model for code repositories is increasing.

[0003] Currently, when building a warehouse-level code question-answering model, the general question-answering model cannot effectively handle the noisy code snippets introduced in the code snippet recall stage, which in turn produces answers with insufficient accuracy. Therefore, data construction for the warehouse-level code question-answering model becomes crucial.

[0004] However, constructing sample data for training code question-answering models often relies on manual annotation, which requires high professional quality of personnel, is costly and inefficient. In addition, fully automatic construction methods (for example, large language models and fixed rule templates) cannot correspond to the details of the code repository, and the generated sample questions are weakly correlated with the code repository, which reduces the quality of the constructed sample questions and sample answers, resulting in poor performance of the trained code question-answering model. Therefore, there is an urgent need for an efficient and high-quality automatic construction method for sample data. Summary of the invention

[0005] In view of this, an embodiment of this specification provides a data construction method. One or more embodiments of this specification also involve a training method for a question-answering model, a code question-answering method, a task platform, a code question-answering system, a data construction device, a training device for a question-answering model, a code question-answering device, a computing device, a computer-readable storage medium, and a computer program product to solve the technical defects existing in the prior art.

[0006] In one embodiment of the embodiments of this specification, a data construction method is provided, including:

[0007] Obtaining a first code snippet and an initial sample question for the first code snippet;

[0008] Using the text processing model, based on the directory structure information of the sample code repository in the code base, the initial sample question is rewritten to obtain the sample question;

[0009] Recalling a related second code snippet from the code base based on the first code snippet and the sample question;

[0010] A text processing model is used to generate sample answers corresponding to sample questions based on the second code snippet, wherein the sample questions and sample answers are sample data for training a code question-and-answer model, and the code question-and-answer model is applicable to a code repository.

[0011] In one embodiment of the present specification, a text processing model is used to rewrite the initial sample questions based on the directory structure information of the sample code repository in the code library to obtain sample questions that are specific to the details of the sample code repository and are strongly related. Based on the first code snippet and the sample questions that are specific to the details of the sample code repository and are strongly related, a related second code snippet is recalled from the code library, thereby improving the accuracy of code snippet retrieval and recall. Based on the high-quality second code snippet, a text processing model is used to generate accurate sample answers corresponding to the sample questions, thereby obtaining highly accurate sample questions and sample answers, thereby achieving automatic construction of high-quality sample data. This not only improves data construction efficiency and reduces data construction costs, but also, the high-quality sample data makes the trained code question and answer model more suitable for the code repository in the code library, and can complete warehouse-level, higher-quality code question and answer processing that is suitable for the code repository, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a flow chart of a data construction method provided by an embodiment of this specification;

[0013] Figure 2 It is a flowchart of a data construction method provided by an embodiment of this specification;

[0014] Figure 3 is a flowchart of a method for training a question-answering model provided by an embodiment of this specification;

[0015] Figure 4 is a flow chart of a code question and answer method provided by an embodiment of this specification;

[0016] Figure 5 is a processing flow chart of a data construction method for training an intelligent coding assistant provided by an embodiment of this specification;

[0017] Figure 6 It is a structural diagram of a task platform provided by an embodiment of this specification;

[0018] Figure 7 It is a structural diagram of a code question and answer system provided by an embodiment of this specification;

[0019] Figure 8 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0020] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0021] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0022] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0023] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0024] In one or more embodiments of this specification, a large model refers to a deep learning model with large-scale model parameters, which usually contains hundreds of millions, tens of billions, hundreds of billions, trillions, or even more than 10 trillion model parameters. A large model can also be called a foundation model / foundation model. The large model is pre-trained with large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks, and the model has good generalization ability, such as a large-scale language model (LLM, Large Language Model), a multimodal pre-training model, etc.

[0025] When the big model is used in practice, only a small number of samples are needed to fine-tune the pre-trained model and it can be applied to different tasks. The big model can be widely used in natural language processing (NLP), computer vision and other fields. Specifically, it can be applied to computer vision tasks such as visual question answering (VQA), image description (IC, Image Caption), image generation, as well as natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. The main application scenarios of the big model include digital assistants, intelligent robots, search, online education, office software, e-commerce, intelligent design, etc.

[0026] First, the terms involved in one or more embodiments of this specification are explained.

[0027] Intelligent Coding Assistant: Using large-scale language models and artificial intelligence technology, it provides functions such as code completion, code generation, code error detection and correction, and code optimization to help developers reduce repetitive work and achieve significant improvements in programming efficiency and code quality.

[0028] Code Chunk: A code snippet in a code base, usually a code file, a code function, a code class, etc. in the code base.

[0029] Repository-level code Q&A: Questions and answers about the code base, including questions and answers about the structure, function, logic, etc. of the code base. Common forms include explanation-type Q&A (such as explaining the operating logic of functional modules) and search-type Q&A (such as introducing the entry class of specified logic).

[0030] Prompt: A type of information that uses natural language to guide or inspire the model to perform tasks.

[0031] Development Issues: Development issues in the code base, usually issues raised by developers in the code base, including problem description, problem label, problem solution, etc.

[0032] Pull Request: A pull request in a code base, usually a pull request made by a developer in the code base, including a pull description, pull code, and pull test.

[0033] LangChain Framework: A framework for developing language model applications that simplifies the process of building, deploying and managing applications based on large language models, supporting a variety of tasks such as question answering, text generation, etc.

[0034] BM25 recall algorithm: An information retrieval algorithm used to evaluate the relevance of query terms to documents. It calculates the score by considering factors such as word frequency and document length, and is widely used in search engines to improve the quality of search results.

[0035] In this specification, a data construction method is provided. This specification also involves a question and answer model training method, a code question and answer method, a task platform, a code question and answer system, a data construction device, a question and answer model training device, a code question and answer device, a computing device, a computer-readable storage medium and a computer program product, which are described in detail one by one in the following embodiments.

[0036] See also Figure 1 , Figure 1 A flowchart of a data construction method provided by an embodiment of this specification is shown, including the following specific steps:

[0037] Step 102: Obtain a first code snippet and an initial sample question for the first code snippet.

[0038] The embodiments of this specification are applied to an application, platform or system having a function of constructing sample data for training a model.

[0039] The first code snippet is a reference code snippet used to recall the second code snippet from the code library. The first code snippet can be a code file, a code function, a code class, or a code variable, which is not limited here. For example, a Python code function: def add(a, b): return a + b.

[0040] The initial sample question is a natural language question raised for the first code snippet, which is a low-quality and weakly relevant sample question. The initial sample question is not specific to the details of the sample code repository and can be an explanatory Q&A, such as "How does the add function work?", or a search Q&A, such as "Where can the definition of the add function be found?".

[0041] To obtain the first code snippet, one optional way is to recall the first code snippet from the code library, and another optional way is to randomly generate the first code snippet, which is not limited here.

[0042] To obtain the initial sample question for the first code snippet, one optional way is to generate the initial sample question for the first code snippet based on a preset question template, another optional way is to generate the initial sample question for the first code snippet based on natural language processing techniques, and another optional way is to manually write the initial sample question for the first code snippet, which is not limited here.

[0043] Exemplarily, on a data processing platform with sample data construction capabilities, recall 20 first code snippets from the code library:

[0044] <Snippet 1 name=backend / tests / test_metricprovider.py> File: backend / tests / test_metricprovider.py.

[0045] async def mock_pull_metric(_, metric: PrometheusMetric): # This is an asynchronous function for simulating pulling metric data from Prometheus.

[0046] return prometheus_data[metric.metric] # Retrieve the data for the corresponding metric from the prometheus_data dictionary.

[0047] < / Snippet 1>

[0048] <Snippet 2 name=backend / tests / test_metricprovider.py> File: backend / tests / test_metricprovider.py.

[0049] async def mock_pull_metric(_, metric: PrometheusMetric): # This is an asynchronous function used to simulate pulling metric data from Prometheus.

[0050] return prometheus_data[metric.metric] # Retrieve the data for the corresponding metric from the prometheus_data dictionary.

[0051] < / fragment 2>

[0052] ……

[0053] <fragment 20 name=backend / streams_explorer / core / k8s_app.py> # File: backend / streams_explorer / core / k8s_app.py.

[0054] def parse_extra_topics(extra_topics: str) -> List[str]: # Define a function to parse the extra topics string.

[0055] # remove trailing commas # Remove trailing commas (if any) from the string.

[0056] extra_topics = extra_topics[:-1] if extra_topics[-1] == ',' else extra_topics # Split the string by commas into multiple substrings in the format "key=value", and we only keep the "value" part.

[0057] return list(

[0058] map(

[0059] lambda topic: topic.split("=")[1], # Split the substring and take the part after the equal sign.

[0060] extra_topics.split(","), # Split the string by commas. ) )

[0063] < / fragment 20>.

[0064] Based on natural language processing technology, generate initial sample questions:

[0065] “How does the codebase process and query Prometheus metrics data?”

[0066] In an optional embodiment of the present specification, obtaining initial sample questions for the first code snippet in step 102 includes the following specific steps: generating initial sample questions for the first code snippet using a text processing model.

[0067] Using the text processing model, an initial sample question for the first code snippet is generated. An optional method is to use the text processing model to generate an initial sample question for the first code snippet and a reference answer corresponding to the initial sample question. The reference answer is a natural language answer generated for the initial sample question, which is a low-quality, weakly relevant sample answer. The reference answer is not specific to the sample code repository details, for example, "the add function is an addition function", which can be an explanation question or a search question, for example, "the definition of the add function can be found in a file in the code repository. You can locate the specific location of the function by searching the code repository."

[0068] Correspondingly, an optional manner of step 104 is: based on the first code snippet, the sample question and the reference answer, recall the relevant second code snippet from the code library. By refining the retrieval index, the accuracy of the recalled second code snippet is improved.

[0069] For example, the prompt for inputting a large language model is as follows:

[0070] “I need your help to construct reasonable user question-answer data pairs.

[0071] 20 first code snippets:

[0072] <snippet 1 name=backend / tests / test_metricprovider.py>### File: backend / tests / test_metricprovider.py.

[0073] async def mock_pull_metric(_, metric: PrometheusMetric): # This is an asynchronous function used to simulate pulling indicator data from Prometheus.

[0074] return prometheus_data[metric.metric]# Get the data of the corresponding indicator from the prometheus_data dictionary.

[0075] < / Fragment 1>

[0076] <Fragment 2 name=backend / tests / test_metricprovider.py> File: backend / tests / test_metricprovider.py.

[0077] async def mock_pull_metric(_, metric: PrometheusMetric): # This is an asynchronous function for mocking the pulling of metric data from Prometheus.

[0078] return prometheus_data[metric.metric] # Get the data corresponding to the metric from the prometheus_data dictionary.

[0079] < / Fragment 2>

[0080] ……

[0081] <Fragment 20 name=backend / streams_explorer / core / k8s_app.py># File: backend / streams_explorer / core / k8s_app.py.

[0082] def parse_extra_topics(extra_topics: str) -> List[str]: # Define a function to parse the extra topics string.

[0083] # remove trailing commas # Remove trailing commas (if any) from the string.

[0084] extra_topics = extra_topics[:-1] if extra_topics[-1] == ',' else extra_topics # Split the string by commas into multiple substrings, each in the format "key=value", and we only keep the "value" part.

[0085] return list(

[0086] map(

[0087] lambda topic: topic.split("=")[1], # Split the substring and take the part after the equal sign.

[0088] extra_topics.split(","), # Split the string by commas. ) )

[0091] < / Fragment 20>.

[0092] Please ask a reasonable user question based on the above content and give an answer to that question.

[0093] Requirements:

[0094] 1. The returned answer needs to maintain the Markdown format;

[0095] 2. When the content of the answer involves specific files, try to use the [name](file: / / path) format hyperlink to represent;

[0096] Return format:

[0097] ## Question

[0098] mock question here

[0099] ## Answer

[0100] answer here

[0101] ## Files related to the answer

[0102] Reasonably speculate on files (such as md, json, xml, yml, etc.) that may be helpful for answering the "user question", return two or more file names, and here the file paths are separated by commas and end with a;

[0103] e.g.: README.md, tabby ssh / src / index.ts;

[0104] If the answer involves specific code snippets, I hope you can additionally return an answer source field in the following form:

[0105] ## Code snippets related to the answer

[0106] Here the code snippet ids are separated by commas and end with a;

[0107] Generate the initial sample question for the first code snippet "How does the codebase process and query Prometheus metric data?" and the reference answer corresponding to the initial sample question "The main functionality for processing and querying Prometheus metric data is implemented by the 'mock_pull_metric' function. This function is an asynchronous function that receives a 'PrometheusMetric' object as a parameter and returns the data associated with the metric. Specifically, the 'mock_pull_metric' function gets the given metric from the 'prometheus_data' dictionary."

[0108] In the embodiment of this specification, by utilizing a text processing model, initial sample questions for the first code snippet are quickly generated, which significantly improves data construction efficiency, reduces the workload of manually writing sample data, and provides a rewriting basis for subsequent rewriting to obtain sample questions.

[0109] In step 102, the first code snippet is obtained to provide an index basis for the subsequent recall of the second code snippet, and the initial sample question for the first code snippet is obtained to provide a rewriting basis for the subsequent rewriting of the sample question.

[0110] Step 104: Using the text processing model, based on the directory structure information of the sample code repository in the code base, the initial sample question is rewritten to obtain a sample question.

[0111] The text processing model is a deep learning model that has the function of processing natural language text, can understand and generate natural language text, and can perform text processing tasks such as text classification, named entity recognition, sentence rewriting, question generation, etc. In the embodiment of this specification, the text processing model is used to rewrite the initial sample question and generate sample answers. The text processing model can be a Transformer model, a BERT model, a diffusion model, a generative adversarial model, a large-scale language model, a GPT model, etc., which are not limited here.

[0112] A code repository is a collection of software code repositories (projects). Code repositories are usually hosted on a code platform for developers to view, use, and contribute code. A sample code repository is a code repository in the code repository that is used as a sample reference for building sample data. A sample code repository has specific details. For example, a Python code repository named streams explorer in the code repository. The directory structure information of a sample code repository is information that describes the target structure of the sample code repository.

[0113] The sample question is a natural language question rewritten for the first code snippet. It is a highly relevant, high-quality sample question that is specific to the details of the code repository. Compared with the initial sample question, the sample question is more specific and accurate. The sample question is specific to the details of the code repository and can be an explanation question, such as "In the backend / utils.py file, how does the add function work?" or a search question, such as "In the streams explorer code repository, in which file is the definition of the add function located?".

[0114] Using the text processing model, based on the directory structure information of the sample code warehouse in the code library, the initial sample question is rewritten to obtain the sample question. An optional way is: using the text processing model, using the directory structure information of the sample code warehouse in the code library as context information, the initial sample question is rewritten to obtain the sample question. Context information is relevant information provided to the text processing model for performing text processing tasks, which helps the text processing model to better understand and generate text. Furthermore, an optional way is: using the text processing model, based on the keyword information in the initial sample question, using the directory structure information of the sample code warehouse in the code library as context information, the initial sample question is rewritten to obtain the sample question.

[0115] Optionally, the initial sample questions are often in Chinese or other languages, while the code snippets are usually written in English characters. Therefore, the initial sample questions are rewritten to obtain sample questions, and language translation is also required.

[0116] Exemplarily, the prompt information for inputting a large language model is as follows:

[0117] “You are the owner of a code repository.

[0118] When users ask you about using this repository, you need to complete the following tasks to help the algorithm retrieve relevant code content:

[0119] 1. Expand the user's query appropriately according to the warehouse file directory structure and convert it into English.

[0120] 2. Generate up to 8 keywords for search.

[0121] 3. Return in JSON format.

[0122] Sample code repository name: streams explorer

[0123] The directory structure of the sample code repository is as follows:

[0124] / : [.pre commit config.yaml, README.md, azure pipelines.yml, dockercompose.yaml]

[0125] backend / :[__init__.py, create_openapi_docs.py, main.py, settings.yaml]

[0126] backend / streams_explorer / :[__init__.py, application.py, default.py, defaultlinker.py, ex

[0127] …

[0128] hooks / :[hook_generate fetchers.sh]

[0129] Your response should follow the following format:

[0130] '''{Sample question: "Where is the base64 encoded code?"}"

[0131] Based on the keyword information ("Prometheus") in the initial sample question, the initial sample question ("How does the codebase process and query Prometheus indicator data?") is rewritten with the above directory structure information as context information to obtain the sample question:

[0132] “How does this codebase process and query Prometheus metrics data?”

[0133] Using the text processing model, the initial sample questions are rewritten based on the directory structure information of the sample code repository in the code base to obtain sample questions that are specific to the details of the code repository and are strongly relevant, providing high-quality retrieval basis for the subsequent recall of the second code snippet.

[0134] Step 106: Based on the first code snippet and the sample question, recall the relevant second code snippet from the code library.

[0135] The second code snippet is a code snippet that is recalled from the code repository and is related to the first code snippet and the sample question. The second code snippet is retrieved based on the first code snippet and a strongly related sample question that is specific to the sample code repository details. The second code snippet is highly related to the first code snippet and the sample question and can usually answer the sample question more accurately. For example, the first code snippet is:

[0136] def add(a, b): return a + b

[0137] A sample question is "In the backend / utils.py file, how does the add function work?"

[0138] The second code snippet is:

[0139] # backend / utils.py

[0140] def add(a, b):

[0141] """This is a simple addition function that adds two numbers.

[0142] parameter:

[0143] a (int, float): first addend

[0144] b (int, float): the second addend

[0145] return:

[0146] int, float: the sum of two numbers

[0147] """# Check if the input is a number

[0148] if not (isinstance(a, (int, float)) and isinstance(b, (int, float))):

[0149] raise TypeError("Input must be an integer or floating point number")

[0150] # Perform addition operation

[0151] result = a + b

[0152] # Return results

[0153] return result

[0154] The second code snippet not only contains the definition of the add function, but also contains detailed comments and some additional logic (such as type checking) so that the sample question can be answered more accurately.

[0155] Exemplarily, based on 20 first code snippets and a sample question (“How does this codebase process and query Prometheus metric data?”), 33 relevant second code snippets are recalled from the codebase:

[0156] [backend / streams_explorer / core / services / metric_providers.py, backend / tests / test_metricprovider.py;]:

[0157] <Fragment 1 name=README.md>

[0158] Docker Compose: Provides a docker-compose.yaml file for configuring and running Docker containers.

[0159] Kubernetes: Provides a Helm Chart configuration file values.yaml for deploying applications on Kubernetes.

[0160] Standalone: Provides a settings.yaml file for configuration when running the application independently.

[0161] Configuration method:

[0162] It can be configured using the double underscore notation in the config section of the helm chart / values.yaml file, e.g., K8S__consumer_group_annotation: consumerGroup. You can also paste the content of the backend / settings.yaml file into the settings section of the helm chart / values.yaml file. Alternatively, it can be configured using environment variables with the prefix SE, e.g., SE_K8S__deployment__cluster=false.

[0163] < / Fragment 1>

[0164] <Fragment 2 name=README.md>

[0165] Plugin customization: Describes how to create custom linkers, metric providers, and extractors.

[0166] LinkingService: Implementing the LinkingService class can create custom linkers.

[0167] MetricProvider: Implementing the MetricProvider class allows for the creation of custom metric providers. The default MetricProvider supports Prometheus.

[0168] Extractor: Implementing the Extractor class allows for the creation of custom extractors. Several default extractor plugins are provided.

[0169] < / fragment 2>

[0170] ……

[0171] <fragment 33 name=backend / streams_explorer / core / services / metric_providers.py>

[0172] def __init__(self, metric: str, query: str, key: str, value_transformer: Callable): ## Class definition: class MetricProvider, __init__ method: Initializes an instance of the MetricProvider class.

[0173] self.metric: str = metric # metric: str: The name of the metric.

[0174] self.query: str = query # query: str: The query statement used to retrieve metric data from a data source (such as Prometheus).

[0175] self._k: str = key # key: str: The key used to identify a specific part of the metric.

[0176] self._v: Callable = value_transformer # value_transformer: Callable: The value transformer used to process the retrieved metric value.

[0177] < / fragment 33>.

[0178] Based on the first code snippet and strongly related sample questions specific to the sample code repository details, relevant second code snippets were recalled from the codebase, improving the accuracy of code snippet retrieval and recall, and providing data support for context information for subsequent generation of sample responses.

[0179] Step 108: Generate sample answers corresponding to sample questions based on the second code snippet using the text processing model, wherein the sample questions and sample answers are sample data for training the code question-answering model, and the code question-answering model is applicable to the code repository.

[0180] The sample answer is a natural language answer generated for the sample question, which is a highly relevant, high-quality sample answer that is specific to the sample code repository. The sample answer refers to the second code snippet and provides detailed explanations and related information. The sample reply is a specific and accurate reply based on the content of the second code snippet and the directory structure information of the sample code repository. For explanation-type questions and answers, the sample reply explains the function and implementation details of the code. For example, the sample question is "In the backend / utils.py file, how does the add function work?" The sample reply is "The 'add' function in the 'backend / utils.py' file is a simple addition function that is used to add two numbers. The function first checks whether the input parameters 'a' and 'b' are integers or floating-point numbers. If the input is not a valid numeric type, the function throws a 'TypeError' exception. Then, the function performs the addition operation and returns the result." For search-type questions and answers, the sample reply shows the specific location of the relevant code and other contextual information. For example, the sample question is "In the streams explorer code repository, in which file is the definition of the add function located?" The sample reply is "In the 'streams explorer' code repository, the definition of the 'add' function is located in the 'backend / utils.py' file. The specific path is as follows: streamsexplorer / backend / init.pycore / k8s_app.pymetrics.pyinit.pystreams_explorer.pyutils.py#add The definition of the function is in this file".

[0181] The code question answering model is a deep learning model specifically used to answer questions about the code repository in the code base. The code question answering model is trained based on sample questions and sample answers as sample data, and can understand and generate natural language answers to the code repository. The code question answering model is designed to solve the problems encountered by developers when dealing with large-scale code repositories, such as code understanding, function query, location search, etc. The code question answering model can be a Transformer model, a BERT model, a diffusion model, a generative adversarial model, a large-scale language model, a GPT model, etc., without limitation here.

[0182] It should be noted that sample questions and sample answers are sample data used to train the code question-answering model, and their quality directly affects the performance of the trained code question-answering model.

[0183] Using the text processing model, based on the second code snippet, to generate a sample answer corresponding to the sample question. An optional way is: using the text processing model, using the second code snippet as context information, to generate a sample answer corresponding to the sample question. Context information is relevant information provided to the text processing model for performing text processing tasks, helping the text processing model to better understand and generate text.

[0184] Exemplarily, the prompt information for inputting a large language model is as follows:

[0185] “You are a question-and-answer assistant for code repositories.

[0186] I will give you a second code snippet that I retrieved based on the sample question that may be helpful in answering the question. You need to answer the question asked by the user based on the given information.

[0187] I will be <retrieval>< / retrieval> The second code snippet retrieved in the area may be helpful for question answering. <query>< / query> Given a sample problem in the region,

[0188] Start now.

[0189] ## Search Results

[0190] [backend / streams_explorer / core / services / metric_providers.py, backend / tests / test_metricprovider.py;]:

[0191] <Segment 1 name=README.md>

[0192] Docker Compose: Provides a docker-compose.yaml file for configuring and running Docker containers.

[0193] Kubernetes: Provides a Helm Chart configuration file values.yaml for deploying applications on Kubernetes.

[0194] standalone: ​​provides a settings.yaml file for configuration when running the application independently.

[0195] Configuration method:

[0196] It can be configured using the double-underscore notation in the config section of the helm chart / values.yaml file, e.g., K8S__consumer_group_annotation: consumerGroup. You can also paste the content of the backend / settings.yaml file into the settings section of the helm chart / values.yaml file. Alternatively, it can be configured using environment variables with the prefix SE, e.g., SE_K8S__deployment__cluster=false.

[0197] < / Fragment 1>

[0198] <Fragment 2 name=README.md>

[0199] Plugin customization: Describes how to create custom linkers, metric providers, and extractors.

[0200] LinkingService: Implementing the LinkingService class can create custom linkers.

[0201] MetricProvider: Implementing the MetricProvider class can create custom metric providers. The default MetricProvider supports Prometheus.

[0202] Extractor: Implementing the Extractor class can create custom extractors. Several default extractor plugins are provided.

[0203] < / Fragment 2>

[0204] ……

[0205] <Fragment 33 name=backend / streams_explorer / core / services / metric_providers.py>

[0206] def __init__(self, metric: str, query: str, key: str, value_transformer: Callable): ## Class definition: class MetricProvider, __init__ method: Initializes an instance of the MetricProvider class.

[0207] self.metric: str = metric #metric: str: The name of the metric.

[0208] self.query: str = query #query: str: The query statement used to retrieve metric data from a data source (such as Prometheus).

[0209] self._k: str = key #key: str: The key used to identify a specific part of the metric.

[0210] self._v: Callable = value_transformer #value_transformer: Callable: The value transformer used to transform the retrieved metric value.

[0211] < / Fragment 33>.

[0212]

[0213] ## User question

[0214] <query> {query}{refine_query}< / query>

[0215] Please comprehensively consider the retrieval results and reply to the user's question;

[0216] Please reply in Chinese;

[0217] The final returned result should be in Markdown format.

[0218] The content format requirements are as follows. Please do not directly output the required text in the answer:

[0219] 1. Please reply to the question raised by the user. Do not make excessive extensions in the content and do not output the table of contents tree;

[0220] 2. If the answer content involves specific code, do not show the fragment id. The user is not aware of the recalled code fragments. A good form of reply is to give appropriately abbreviated code and then give an explanation of the code;

[0221] 3. If the answer to the question does not require specific code, such as asking about architecture, avoid outputting sample code in the answer;

[0222] Rendering format requirements:

[0223] 1. When the answer involves a specific file, make sure to use the [filename](file: / / filepath) hyperlink format, such as [user.java](file: / / extensions / firefox / user.java);

[0224] 2. Strictly ensure that the reference file is represented in the format of [filename](file: / / filepath), where filename is the file name without the file path, and filepath contains the full file path!

[0225] 3. If you use the entire path, make sure to use the [directory_name](file: / / filepath) format, such as [firefox](file: / / extensions / firefox / )"

[0226] Using the 33 second code snippets ([backend / streams_explorer / core / services / metric_providers.py, backend / tests / test_metricprovider.py;]) as context information, generate sample responses to the sample question (“How does this codebase process and query Prometheus metrics data?”):

[0227] “The process of processing and querying Prometheus indicator data in this code base is mainly divided into the following parts:

[0228] 1. Define Prometheus Metrics

[0229] An enumeration class named 'PrometheusMetric' is defined in [metric_providers.py](file: / / backend / streams_explorer / core / services / metric_providers.py). This class contains the name, query statement, key and value conversion function of each metric.

[0230] '''python

[0231] class PrometheusMetric(Enum):

[0232] def __init__(self, metric: str, query: str, key: str, value_transformer: Callable): ##Class definition: class MetricProvider, __init__ method: Initialize an instance of the MetricProvider class.

[0233] self.metric: str = metric #metric: str: metric name.

[0234] self.query: str = query #query: str: Query statement used to obtain indicator data from a data source (such as Prometheus).

[0235] self._k: str = key #key: str: Key, used to identify a specific part of the indicator.

[0236] self._v: Callable = value_transformer #value_transformer: Callable: Value converter, used to convert the obtained indicator value.

[0237] '''

[0238] 2. **Query Prometheus data**

[0239] This is mainly achieved through the 'PrometheusMetricProvider' class, which is a Prometheus Metric provider as its name suggests. This class contains multiple asynchronous methods for interacting with the Prometheus API, such as '_pull_metric' and '_query'.

[0240] The '_query' method uses the httpx library to create an asynchronous Hypertext Transfer Protocol (HTTP) client, send query requests to Prometheus and process the responses:

[0241] '''python

[0242] async def _query(self, query: str) ->list:

[0243] async with httpx.AsyncClient(base_url=self._api_base) as client:# Create an asynchronous HTTP client.

[0244] try:

[0245] r = await client.get(" / query", params={"query": query}) # Send a GET request to the Prometheus API.

[0246] if r.status_code == httpx.codes.OK: # Check if the response status code is 200 (OK).

[0247] data = r.json()# Parse the response JSON data.

[0248] if data and "data" in data and "result" in data["data"]: # Check whether the data structure is correct.

[0249] return data["data"]["result"]# Returns the query result.

[0250] except httpx.ReadTimeout:#Handle read timeout exception.

[0251] logger.warning("Prometheus query '{}' timed out", query)# Record warning logs.

[0252] raise PrometheusException# If the request fails, throw a PrometheusException exception''

[0253] The '_pull_metric' method asynchronously pulls data for a specific metric:

[0254] '''python

[0255] async def _pull_metric(self, metric: PrometheusMetric) ->list:

[0256] try:

[0257] return await self._query(metric.query)# Call the _query method to execute the query

[0258] except PrometheusException as e:# Handle PrometheusException exception

[0259] logger.error(f"Error pulling {metric}: {e}")# Log error

[0260] return []# If an exception occurs, return an empty list'''

[0261] 3. **Refresh data regularly**

[0262] The 'PrometheusMetricProvider' class also defines the 'refresh_data' method, which refreshes and processes the data of Metrics at regular intervals. Use 'asyncio.gather' to execute multiple tasks in parallel to improve efficiency:

[0263] '''python

[0264] async def refresh_data(self):

[0265] logger.debug("Pulling metrics from Prometheus")# Record debug logs, indicating that metrics pulling has started.

[0266] tasks = []

[0267] for metric in PrometheusMetric:# Iterate over all PrometheusMetric.

[0268] tasks.append(asyncio.ensure_future(self._process_metric(metric)))# Create and add tasks.

[0269] await asyncio.gather(*tasks)# Execute all tasks in parallel to improve efficiency. '''

[0270] 4. **Cache mechanism**

[0271] In order to reduce frequent queries to Prometheus, this class implements a simple caching mechanism, and will only re-pull data when the cache times out:

[0272] '''python

[0273] async def get(self) ->List[Metric]:

[0274] now = datetime.utcnow()# Get the current time.

[0275] cache_age = now - self._last_refresh # Calculate the age of the cache.

[0276] if cache_age>self._cache_ttl: # If the cache times out.

[0277] self._last_refresh = now # Update the last refresh time.

[0278] await self.update()# Update the cache.

[0279] else:

[0280] logger.debug("Serving cached metrics (age {}s)", cache_age.seconds)#Record debug logs to indicate the return of cached data.

[0281] return self._metrics# Returns the Metrics list.

[0282] '''

[0283] Overall, this codebase efficiently processes and queries Prometheus's metrics data by defining the PrometheusMetric enumeration class, using asynchronous Hypertext Transfer Protocol (HTTP) requests to query Prometheus data, refreshing data periodically, and implementing a caching mechanism. "

[0284] In the embodiments of the present specification, a text processing model is utilized to rewrite the initial sample questions based on the directory structure information of the sample code repository in the code base to obtain sample questions that are specific to the details of the sample code repository and are strongly correlated; based on the first code snippet and the sample questions that are specific to the details of the sample code repository and are strongly correlated, a related second code snippet is recalled from the code base, thereby improving the accuracy of code snippet retrieval and recall; based on the high-quality second code snippet, a text processing model is utilized to generate accurate sample answers corresponding to the sample questions to obtain highly accurate sample questions and sample answers, thereby achieving automatic construction of high-quality sample data, which not only improves data construction efficiency and reduces data construction costs, but also, the high-quality sample data makes the trained code question-and-answer model more suitable for the code repository in the code base, and can complete warehouse-level, higher-quality code question-and-answer processing that is suitable for the code repository, thereby improving the user experience.

[0285] In an optional embodiment of the present specification, obtaining the first code snippet in step 102 includes the following specific steps: obtaining metadata information of the code library; and recalling the relevant first code snippet from the code library based on the metadata information.

[0286] Due to the large number of files and complex code content in the code base, directly generating sample questions will result in the generated sample questions not having a strong correlation with the code base. In the first stage, a pseudo-random strategy can be used to recall the first code snippet, generate sample questions on this basis, and then perform high-quality retrieval and recall in the second stage, generating high-quality sample answers based on the recalled high-quality second code snippet.

[0287] The metadata information of a code base is additional information related to the code base. Metadata information is usually not included in the actual code files, but provides important context and background about the code base. Metadata information helps developers quickly understand the overall situation of the code base and also provides necessary input for automated tools (such as issue trackers, continuous integration systems, etc.). For example, the development issues (Issues) of the code base and the feature iteration requests (Pull Requests) of the code base.

[0288] Obtain metadata information of the code base. An optional method is to obtain metadata of the code base, parse the metadata, and obtain metadata information. For example, a metadata acquisition tool is used to obtain metadata of the code base, parse a large amount of metadata information such as development issues and function iteration requests, and preliminarily recall the relevant first code snippet.

[0289] Exemplarily, a metadata acquisition tool is used to acquire metadata of a code base, parse the metadata, obtain development issues and function iteration requests, and based on the development issues and function iteration requests, retrieve 20 relevant first code snippets from the code base.

[0290] In the embodiments of this specification, the recall strategy based on metadata information not only improves the relevance of sample questions to the actual code base content, but also ensures that the sample data constructed subsequently is closer to the actual development scenario, obtains the first code snippet, and provides an index basis for the subsequent recall of the second code snippet. In view of the problem that the user questions constructed in the single-step code base recall strategy cannot correspond to the code base details, and the code retrieval recall accuracy is not high, through the multi-step code base retrieval recall strategy, it is possible to use a large language model to generate highly relevant and high-quality sample questions specific to the code base details in the first round of code snippet retrieval. At the same time, in the second round of code snippet retrieval, the first code snippet and the rewritten high-quality sample questions and other information are used to further improve the accuracy of the code snippet retrieval recall related to the problem, thereby generating more accurate sample replies, effectively improving the data quality of the sample data.

[0291] In an optional embodiment of the present specification, recalling the relevant first code snippet from the code library based on the metadata information includes the following specific steps: utilizing the retrieval engine of the code library to recall the relevant first code snippet from the code library based on the metadata information.

[0292] The code base search engine is a tool for searching and recalling relevant code snippets in the code base. The search engine usually combines multiple technologies and algorithms, such as full-text search, semantic analysis, similarity calculation, etc., to improve the accuracy and efficiency of the search. In the code base, the search engine can help developers quickly locate specific function implementations, bug fixes, configuration settings, etc. For example, a search engine built based on the LangChain framework and the BM25 recall algorithm.

[0293] The search keyword is the information entered into the search engine when searching, which is used to describe the code snippets you want to recall. The search keyword can be a single word, a combination of multiple words, a phrase, or a sentence. The search engine will search for code snippets in the code base based on these keywords and return relevant code snippets. The selection of search keywords directly affects the accuracy and relevance of search results. For example, the search keyword "Prometheus metrics" is used to find code snippets related to Prometheus indicators, which may include code snippets that implement functions such as defining indicators, collecting indicator data, and processing indicator data.

[0294] Using the search engine of the code library, based on the metadata information, the relevant first code snippet is recalled from the code library. An optional method is: using the search engine of the code library, using the metadata information as a search keyword, to recall the relevant first code snippet from the code library.

[0295] Exemplarily, the search engine of the code library is used to retrieve 20 relevant first code snippets from the code library using the above development issues and function iteration requests as search keywords.

[0296] During the implementation of this specification, the code base retrieval engine is utilized, and based on metadata information, the relevant first code snippet can be efficiently and accurately recalled from the huge code base, further improving the relevance of the sample problem with the actual code base content, and further ensuring that the subsequently constructed sample data is closer to the actual development scenario, obtaining the first code snippet, and providing an index basis for the subsequent recall of the second code snippet.

[0297] In an optional embodiment of the present specification, obtaining initial sample questions for the first code snippet in step 102 includes the following specific steps: using a text processing model to generate initial sample questions for the first code snippet based on directory structure information of the code library.

[0298] Using the text processing model, based on the directory structure information of the code base, an initial sample question for the first code snippet is generated. An optional method is: using the text processing model, using the directory structure information of the code base as context information, to generate the initial sample question for the first code snippet. Context information is relevant information provided to the text processing model for performing text processing tasks, helping the text processing model to better understand and generate text. Furthermore, an optional method is: using the text processing model, using the directory structure information of the code base as context information, to generate the initial sample question for the first code snippet and the reference answer corresponding to the initial sample question.

[0299] For example, the prompt for inputting a large language model is as follows:

[0300] “I need your help in building reasonable user question-answer data pairs for a given code base.

[0301] The code directory structure is as follows:

[0302] <catalog>

[0303] / : [.pre commit config.yaml, README.md, azure pipelines.yml, dockercompose.yaml]

[0304] ……

[0305] hooks / : [hook_generate fetchers.sh]

[0306] < / catalog>

[0307] 20 first code snippets:

[0308] <snippet 1 name=backend / tests / test_metricprovider.py>### File: backend / tests / test_metricprovider.py.

[0309] async def mock_pull_metric(_, metric:PrometheusMetric): # This is an asynchronous function used to mock pulling metric data from Prometheus.

[0310] return prometheus_data[metric.metric] # Retrieve the data corresponding to the metric from the prometheus_data dictionary.

[0311] < / fragment 1>

[0312] <fragment 2 name=backend / tests / test_metricprovider.py> File: backend / tests / test_metricprovider.py.

[0313] async def mock_pull_metric(_, metric:PrometheusMetric): # This is an asynchronous function used to mock pulling metric data from Prometheus.

[0314] return prometheus_data[metric.metric] # Retrieve the data corresponding to the metric from the prometheus_data dictionary.

[0315] < / fragment 2>

[0316] ……

[0317] <fragment 20 name=backend / streams_explorer / core / k8s_app.py> # File: backend / streams_explorer / core / k8s_app.py.

[0318] def parse_extra_topics(extra_topics:str)->List[str]: # Define a function to parse the extra topics string.

[0319] # remove trailing commas # Remove trailing commas (if any) from the string.

[0320] extra_topics = extra_topics[:1] if extra_topics[-1] == "," else extra_topics # Split the string into multiple substrings separated by commas, each substring in the format "key=value", and we only keep the "value" part.

[0321] return list(

[0322] map(

[0323] lambda topic: topic.split("=")[1], # Split the substring and take the part after the equal sign.

[0324] extra_topics.split(","), # Split the string by commas. ) )

[0327] < / fragment 20>.

[0328] Please ask a reasonable user question based on the above content and give the correct answer to the question.

[0329] Requirements:

[0330] 1. **The user's question must be about the structure, function, logic, etc. of the current codebase and related to the input directory;**

[0331] 2. The returned answer needs to be in Markdown format;

[0332] 3. When the content of the answer involves specific files, try to use the [name](file: / / path) format hyperlink to represent;

[0333] Return format:

[0334] ## User question

[0335] mock question here

[0336] ## Question answer

[0337] answer here

[0338] ## Files involved in the answer

[0339] Reasonably speculate on the files (such as md, json, xml, yml, etc.) that may be helpful for answering the "user question", and return two or more file names. Here, the file paths are separated by commas and end with a;

[0340] eg: README.md, tabby ssh / src / index.ts;

[0341] If the answer involves a specific code snippet, I hope you can return an additional answer source field in the following form:

[0342] ## The answer involves fragments

[0343] Here the code snippet ids are separated by commas and end with a;"

[0344] Based on the directory structure information of the code library ( <catalog> {catalog}< / catalog> ) as context information, generate the initial sample question for the first code snippet "How does the codebase process and query Prometheus metric data?" and the reference answer corresponding to the initial sample question "The main functionality for processing and querying Prometheus metric data is implemented by the 'mock_pull_metric' function. This function is an asynchronous function that receives a 'PrometheusMetric' object as a parameter and returns data related to the metric. Specifically, the 'mock_pull_metric' function gets the given metric from the 'prometheus_data' dictionary."

[0345] In the embodiments of this specification, by utilizing a text processing model, based on the directory structure information of the code library, initial sample questions that are relatively specific to the details of the sample code repository and weakly correlated are quickly generated, thereby significantly improving data construction efficiency, reducing the workload of manually writing sample data, and providing a rewriting basis for subsequent rewriting to obtain sample questions.

[0346] In an optional embodiment of the present specification, step 104 includes the following specific steps: using a text processing model to extract keyword information from an initial sample question, and rewriting the initial sample question based on the keyword information and directory structure information of a sample code repository in a code library to obtain a sample question.

[0347] Keyword information is information extracted from the initial sample question that characterizes the core content of the initial sample question. Keyword information is the basis for accurate retrieval of the second code snippet and the basis for rewriting the initial sample question, helping the text processing model to understand the initial sample question more accurately and generate sample questions that are specific to the details of the sample code repository and are strongly related. Keyword information usually includes nouns, verbs, technical terms, etc. For example, in the initial sample question "How does the codebase handle and query Prometheus indicator data?", the keyword information includes "Prometheus", "metrics_handling", "metrics_querying", "backend", "metrics.py", "core", "metric_providers", "linking_services".

[0348] Based on the keyword information and the directory structure information of the sample code warehouse in the code library, the initial sample question is rewritten to obtain the sample question. An optional method is: based on the keyword information, the directory structure information of the sample code warehouse in the code library is used as context information to rewrite the initial sample question to obtain the sample question. Context information is relevant information provided to the text processing model for performing text processing tasks, which helps the text processing model better understand and generate text.

[0349] Exemplarily, the prompt information for inputting a large language model is as follows:

[0350] “You are the owner of a code repository.

[0351] When users ask you about using this repository, you need to complete the following tasks to help the algorithm retrieve relevant code content:

[0352] 1. Expand the user's query appropriately according to the warehouse file directory structure and convert it into English.

[0353] 2. Generate up to 8 keywords for search.

[0354] 3. Return in JSON format.

[0355] Sample code repository name: streams explorer

[0356] The directory structure of the sample code repository is as follows:

[0357] / : [.pre commit config.yaml, README.md, azure pipelines.yml, dockercompose.yaml]

[0358] backend / : [__init__.py, create_openapi_docs.py, main.py, settings.yaml]

[0359] backend / streams_explorer / :[__init__.py, application.py, default.py, defaultlinker.py, ex

[0360] …

[0361] hooks / :[hook_generate fetchers.sh]

[0362] Your response should follow the following format:

[0363] '''{Sample question: "Where is the base64 encoded code?"

[0364] Keyword information: ["base64_encoding", "base64", "encode"]}"

[0365] Extract keyword information ("Prometheus", "metrics_handling", "metrics_querying", "backend", "metrics.py", "core", "metric_providers", "linking_services") from the initial sample question, and based on the keyword information, use the directory structure information of the sample code repository in the code library as context information to rewrite the initial sample question ("How does the code library handle and query Prometheus indicator data?") to obtain the sample question:

[0366] “How does this codebase process and query Prometheus metrics data?”

[0367] Keyword information: ["Prometheus", "metrics_handling", "metrics_querying", "backend", "metrics.py", "core", "metric_providers", "linking_services"].

[0368] In the embodiments of the present specification, a text processing model is used to extract keyword information from an initial sample question, and the initial sample question is rewritten based on the keyword information and the directory structure information of the sample code repository in the code library to obtain a more specific, strongly relevant sample question that is specific to the details of the sample code repository, and provides a higher quality retrieval basis for the subsequent recall of the second code snippet.

[0369] In an optional embodiment of the present specification, step 106 includes the following specific steps: based on the first code snippet, keyword information and sample questions, recalling a related second code snippet from a code library.

[0370] After obtaining sample questions that are specific to the details of the sample code repository and are strongly relevant, it may still be impossible to retrieve and recall accurate code snippets due to the inherent differences between natural language and code language, and further refined retrieval is required.

[0371] Exemplarily, based on the 20 first code snippets, keyword information ("Prometheus", "metrics_handling", "metrics_querying", "backend", "metrics.py", "core", "metric_providers", "linking_services") and sample questions ("How does this code base handle and query Prometheus metric data?"), 33 relevant second code snippets are recalled from the code base.

[0372] Based on the first code snippet, keyword information, and strongly relevant sample questions specific to the details of the sample code repository, the relevant second code snippet is recalled from the code repository, further improving the accuracy of code snippet retrieval and recall, and providing data support for contextual information for the subsequent generation of sample answers.

[0373] In an optional embodiment of the present specification, based on the first code snippet, keyword information and sample questions, recalling the relevant second code snippet from the code library includes the following specific steps: using the retrieval engine of the code library to recall the relevant second code snippet from the code library based on the first code snippet, keyword information and sample questions.

[0374] It should be noted that the search engine used in the embodiments of this specification can be consistent with the search engine used in subsequent code questions and answers to ensure that the noise code snippets retrieved and recalled are consistent with the distribution of noise code snippets in the actual code questions and answers. For example, a search engine based on the LangChain framework and the BM25 recall algorithm is uniformly used. This consistency helps to maintain consistent performance and effects in training and actual applications.

[0375] Utilizing the retrieval engine of the code library, based on the first code snippet, keyword information and sample questions, recalling the related second code snippet from the code library, utilizing the retrieval engine of the code library, taking the first code snippet, keyword information and sample questions as retrieval keywords, recalling the related second code snippet from the code library.

[0376] Exemplarily, the search engine of the code library is used to retrieve 33 related second code snippets from the code library using the first code snippet, keyword information and sample questions as search keywords.

[0377] In the embodiments of the present specification, the retrieval engine of the code library is utilized to efficiently and accurately recall the related second code snippet from the code library based on the first code snippet, keyword information and sample questions, thereby achieving high-quality retrieval and recall, further improving the accuracy of code snippet retrieval and recall, and providing data support for contextual information for the subsequent generation of sample replies.

[0378] In an optional embodiment of the present specification, step 108 includes the following specific steps: using a text processing model to generate a sample answer corresponding to the sample question based on the second code snippet and the directory structure information of the sample code repository.

[0379] The directory structure information of the sample code repository is information describing the target structure of the sample code repository.

[0380] Using the text processing model, based on the second code snippet and the directory structure information of the sample code warehouse, a sample answer corresponding to the sample question is generated. An optional method is: using the text processing model, using the second code snippet and the directory structure information of the sample code warehouse as context information, to generate a sample answer corresponding to the sample question. Context information is relevant information provided to the text processing model for performing text processing tasks, helping the text processing model to better understand and generate text.

[0381] For example, the prompt for inputting a large language model is as follows:

[0382] “You are a question-and-answer assistant for code repositories.

[0383] I will give you a user question about the sample code repository, as well as the sample code repository's directory structure catalog and the sample code repository's environment dependencies as context information, and a second code snippet retrieved based on the user's question that may be helpful in answering the question. You need to answer the user's question based on the given information.

[0384] I will be <catalog>< / catalog> Given the directory structure of the sample code repository in the area <requirements>< / requirements>The environment dependencies for a given sample code repository in a region, <retrieval>< / retrieval> The second code snippet retrieved in the area may be helpful for question answering. <query>< / query> Given a user problem in the region,

[0385] Start now.

[0386] ##Sample code repository directory

[0387] <catalog> Directory Structure:{catelog}< / catalog>

[0388] ## Environment Dependencies

[0389] <requirements> {requirements}< / requirements>

[0390] ## Search Results

[0391] [backend / streams_explorer / core / services / metric_providers.py, backend / tests / test_metricprovider.py;]

[0392] <Segment 1 name=README.md>

[0393] Docker Compose: Provides a docker-compose.yaml file for configuring and running Docker containers.

[0394] Kubernetes: Provides a Helm Chart configuration file values.yaml for deploying applications on Kubernetes.

[0395] standalone: ​​provides a settings.yaml file for configuration when running the application independently.

[0396] Configuration method:

[0397] It can be configured using the double-underscore notation in the config section of the helm chart / values.yaml file, for example, K8S__consumer_group_annotation: consumerGroup. You can also paste the content of the backend / settings.yaml file into the settings section of the helm chart / values.yaml file. Alternatively, it can be configured using environment variables with the prefix SE, such as SE_K8S__deployment__cluster=false.

[0398] < / fragment 1>

[0399] <fragment 2 name=README.md>

[0400] Plugin customization: Describes how to create custom linkers, metric providers, and extractors.

[0401] LinkingService: Implementing the LinkingService class can create custom linkers.

[0402] MetricProvider: Implementing the MetricProvider class can create custom metric providers. The default MetricProvider supports Prometheus.

[0403] Extractor: Implementing the Extractor class can create custom extractors. Several default extractor plugins are provided.

[0404] < / fragment 2>

[0405] ……

[0406] <fragment 33 name=backend / streams_explorer / core / services / metric_providers.py>

[0407] def __init__(self, metric: str, query: str, key: str, value_transformer: Callable): ## Class definition: class MetricProvider, __init__ method: Initializes an instance of the MetricProvider class.

[0408] self.metric: str = metric #metric: str: The name of the metric.

[0409] self.query: str = query #query: str: The query statement used to retrieve metric data from a data source (such as Prometheus).

[0410] self._k: str = key #key: str: The key used to identify a specific part of the metric.

[0411] self._v: Callable = value_transformer #value_transformer: Callable: The value transformer used to transform the retrieved metric value.

[0412] < / fragment 33>.

[0413]

[0414] ## User's question

[0415] <query> {query}{refine_query}< / query>

[0416] Please consider the sample code repository directory, environment dependencies, and retrieval results to reply to the user's question;

[0417] Please reply in Chinese;

[0418] The final returned result should be output in Markdown format.

[0419] The content format requirements are as follows. Please do not directly output the required text in the answer:

[0420] 1. Please reply to the question raised by the user. Do not make excessive extensions in the content and do not output the directory tree;

[0421] 2. If the answer content involves specific code, do not show the fragment id. The user is not aware of the recalled code fragments. A good reply form is to give appropriately abbreviated code and then give an explanation of the code;

[0422] 3. If the question answer does not need to involve specific code, such as asking about the architecture, avoid outputting example code and other content in the answer;

[0423] Rendering format requirements:

[0424] 1. When the answer involves a specific file, make sure to use the [filename](file: / / filepath) hyperlink format, such as [user.java](file: / / extensions / firefox / user.java);

[0425] 2. Strictly ensure that the reference file is represented in the format of [filename](file: / / filepath), where filename is the file name without the file path, and filepath contains the full file path!

[0426] 3. If you use the entire path reference, make sure to use the [directory_name](file: / / filepath) format, such as [firefox](file: / / extensions / firefox / )"

[0427] With 33 second code snippets ([backend / streams_explorer / core / services / metric_providers.py, backend / tests / test_metricprovider.py;]) and the directory structure information of the sample code repository ( <catalog> Directory Structure:{catelog}< / catalog> ) as context information, and generate a sample answer to a sample question (“How does this codebase process and query Prometheus metrics data?”):

[0428] “The process of processing and querying Prometheus indicator data in this code base is mainly divided into the following parts:

[0429] 1. Define Prometheus Metrics

[0430] An enumeration class named 'PrometheusMetric' is defined in [metric_providers.py](file: / / backend / streams_explorer / core / services / metric_providers.py). This class contains the name, query statement, key and value conversion function of each metric.

[0431] '''python

[0432] class PrometheusMetric(Enum):

[0433] def __init__(self, metric: str, query: str, key: str, value_transformer: Callable): ##Class definition: class MetricProvider, __init__ method: Initialize an instance of the MetricProvider class.

[0434] self.metric: str = metric #metric: str: metric name.

[0435] self.query: str = query #query: str: Query statement used to obtain indicator data from a data source (such as Prometheus).

[0436] self._k: str = key #key: str: Key, used to identify a specific part of the indicator.

[0437] self._v: Callable = value_transformer #value_transformer: Callable: Value converter, used to convert the obtained indicator value.

[0438] '''

[0439] 2. **Query Prometheus data**

[0440] This is mainly achieved through the 'PrometheusMetricProvider' class, which is a Prometheus Metric provider as its name suggests. This class contains multiple asynchronous methods for interacting with the Prometheus API, such as '_pull_metric' and '_query'.

[0441] The '_query' method uses the httpx library to create an asynchronous Hypertext Transfer Protocol (HTTP) client, send query requests to Prometheus and process the responses:

[0442] '''python

[0443] async def _query(self, query: str) ->list:

[0444] async with httpx.AsyncClient(base_url=self._api_base) as client:# Create an asynchronous HTTP client

[0445] try:

[0446] r = await client.get(" / query", params={"query": query}) # Send a GET request to the Prometheus API

[0447] if r.status_code == httpx.codes.OK:# Check if the response status code is 200 (OK)

[0448] data = r.json()# Parse the response JSON data

[0449] if data and "data" in data and "result" in data["data"]:# Check if the data structure is correct

[0450] return data["data"]["result"]# Returns the query result

[0451] except httpx.ReadTimeout:#Handle read timeout exception

[0452] logger.warning("Prometheus query '{}' timed out", query)# Record warning log

[0453] raise PrometheusException# If the request fails, throw a PrometheusException exception''

[0454] The '_pull_metric' method asynchronously pulls data for a specific metric:

[0455] '''python

[0456] async def _pull_metric(self, metric: PrometheusMetric) ->list:

[0457] try:

[0458] return await self._query(metric.query)# Call the _query method to execute the query

[0459] except PrometheusException as e:# Handle PrometheusException exception

[0460] logger.error(f"Error pulling {metric}: {e}")# Log error

[0461] return []# If an exception occurs, return an empty list'''

[0462] 3. **Refresh data regularly**

[0463] The 'PrometheusMetricProvider' class also defines the 'refresh_data' method, which refreshes and processes the data of Metrics at regular intervals. Use 'asyncio.gather' to execute multiple tasks in parallel to improve efficiency:

[0464] '''python

[0465] async def refresh_data(self):

[0466] logger.debug("Pulling metrics from Prometheus")# Record debug logs to indicate that metrics have started to be pulled

[0467] tasks = []

[0468] for metric in PrometheusMetric:# Traverse all PrometheusMetric

[0469] tasks.append(asyncio.ensure_future(self._process_metric(metric)))# Create and add tasks

[0470] await asyncio.gather(*tasks)# Execute all tasks in parallel to improve efficiency'''

[0471] 4. **Cache mechanism**

[0472] In order to reduce frequent queries to Prometheus, this class implements a simple caching mechanism, and will only re-pull data when the cache times out:

[0473] '''python

[0474] async def get(self) ->List[Metric]:

[0475] now = datetime.utcnow()# Get the current time

[0476] cache_age = now - self._last_refresh # Calculate the age of the cache

[0477] if cache_age>self._cache_ttl:# If the cache times out

[0478] self._last_refresh = now# Update the last refresh time

[0479] await self.update()# Update cache

[0480] else:

[0481] logger.debug("Serving cached metrics (age {}s)", cache_age.seconds)#Record debug logs to indicate cached data is returned

[0482] return self._metrics# Returns the Metrics list'''

[0483] Overall, this codebase efficiently processes and queries Prometheus's metrics data by defining the PrometheusMetric enumeration class, using asynchronous Hypertext Transfer Protocol (HTTP) requests to query Prometheus data, refreshing data periodically, and implementing a caching mechanism. "

[0484] In the embodiment of the present specification, a text processing model is utilized to generate sample answers corresponding to accurate sample questions that are specific to the details of the sample code repository based on the directory structure information of the second code snippet and the sample code repository, thereby obtaining highly accurate sample questions and sample answers, thereby achieving automatic construction of high-quality sample data.

[0485] Figure 2 A schematic diagram of a data construction method provided by an embodiment of this specification is shown. Figure 2 As shown:

[0486] For the code repository, based on the metadata information of the code repository, the relevant first code snippet is recalled from the code repository. The large language model generates an initial sample question and an initial sample answer for the first code snippet. The large language model extracts keyword information from the initial sample question and rewrites the initial sample question to obtain a sample question. Based on the first code snippet, the keyword information and the sample question, the relevant second code snippet is recalled from the code repository. A reference code snippet is obtained from the code repository. The large language model generates a sample answer corresponding to the sample question. Construct sample data for training a code question-answering model applicable to the code repository.

[0487] like Figure 2 As shown in the figure, the data construction method of warehouse-level code question and answer based on multi-step code base recall can construct high-quality code question and answer sample data specific to the code repository with the help of a general large language model and multi-stage warehouse code snippet retrieval and recall. It not only reduces the cost of manual labeling and saves the labeling cost of sample data, but also solves the problems of low relevance of sample questions constructed in the single-step recall strategy and omission of retrieval and recall code snippets, improves the quality of warehouse-level code question and answer data, and ensures the diversity of sample data and the quality of code question and answer.

[0488] Through the above data construction method embodiment, high-quality sample data for training the code question answering model is constructed. Correspondingly, Figure 3 A flowchart of a method for training a question-answering model provided by an embodiment of the present specification is shown, including the following specific steps:

[0489] Step 302: Obtain sample questions and sample answers, wherein the sample questions and sample answers are constructed according to the above data construction method.

[0490] The embodiments of this specification are applied to applications, platforms or systems with model training functions.

[0491] The embodiment of this specification and the above-mentioned data construction method belong to the same concept. For details not described in detail in step 302, please refer to the description of the technical solution of the above-mentioned data construction method, which will not be repeated here.

[0492] Obtaining sample questions and sample answers provides sample data support for training the code question answering model.

[0493] Step 304: Generate a predicted answer corresponding to the sample question using the initial question-answering model.

[0494] The initial question-answering model is a pre-trained general question-answering model that has certain text understanding and generation capabilities, but has not yet been trained for code question-answering in code repositories. The initial question-answering model can be a Transformer model, a BERT model, a diffusion model, a generative adversarial model, a large-scale language model, a GPT model, etc., without limitation here.

[0495] The predicted answer is the natural language answer generated by the initial question-answering model based on the input sample question. The predicted answer may be different from the actual sample answer. By comparing the predicted answer and the sample answer, the model performance of the initial question-answering model can be measured, and the initial question-answering model can be trained in a targeted manner to gradually improve the accuracy of the model.

[0496] For example, the sample question is: "How does this code base process and query Prometheus metric data?", and the sample question is input into the general question-answering model to generate the predicted answer corresponding to the sample question: "The mock_pull_metric function is used to process and query Prometheus metric data. This function obtains data related to a given PrometheusMetric object from the prometheus_data dictionary."

[0497] Step 306: Based on the predicted answers and the sample answers, the initial question-answering model is trained to obtain a code question-answering model suitable for the code repository in the code library.

[0498] Based on the predicted answers and sample answers, the initial question-answering model is trained to obtain a code question-answering model applicable to the code repository in the code base. An optional method is: based on the predicted answers and sample answers, the loss value is calculated, and based on the loss value, the model parameters of the initial question-answering model are adjusted, and when the training end condition is met, a code question-answering model applicable to the code repository in the code base is obtained. Among them, the loss value can be a cross entropy loss value, a mean square error loss value, a Kullback-Leibler (KL) divergence, etc., and the training end condition can be a preset number of iterations, a loss value threshold, and the performance on the validation set no longer improves.

[0499] Exemplarily, the sample reply is: "The process of this code base processing and querying Prometheus indicator data is mainly divided into the following parts:... In general, this code base effectively processes and queries Prometheus indicator data by defining the PrometheusMetric enumeration class, using asynchronous Hypertext Transfer Protocol (HTTP) requests to query Prometheus data, refreshing data periodically, and implementing a caching mechanism." Predicted reply: "The mock_pull_metric function is used to process and query Prometheus indicator data. This function gets the data related to a given PrometheusMetric object from the prometheus_data dictionary." Based on the predicted reply and the sample reply, the cross entropy loss value is calculated, and based on the cross entropy loss value, the model parameters of the general question and answer model are adjusted, and when the preset number of iterations is met, a code question and answer model suitable for the code repository "streams explorer" in the code base is obtained.

[0500] In the embodiments of this specification, high-quality sample data pre-constructed by a data construction method is used to train a general initial question-and-answer model to obtain a code question-and-answer model suitable for a code repository in a code base. This can complete warehouse-level, higher-quality code question-and-answer processing adapted to the code repository, thereby improving the user experience.

[0501] Through the embodiment of the training method of the above question-answering model, a code question-answering model suitable for the code repository in the code library is trained. Correspondingly, Figure 4 A flowchart of a code question and answer method provided by an embodiment of this specification is shown, including the following specific steps:

[0502] Step 402: Obtain user questions for the target code.

[0503] The embodiments of this specification are applied to applications, platforms or systems with code question and answer, such as an integrated development environment (IDE) with an API (Application Programming Interface) of a code question and answer model, a code hosting platform, or a code document generation and maintenance platform.

[0504] The target code is the specific code that the user wants to understand or query, which can be a code function, code class, code module or other code unit. The target code can be located in the code file in the code library. The user may want to understand the function, usage, implementation details, etc. of the target code by asking questions. User questions are natural language questions raised for the target code.

[0505] Exemplarily, the user question about the target code (mock_pull_metric function) sent by the receiving terminal: "How does the mock_pull_metric function process Prometheus metric data?"

[0506] Obtaining the user question about the target code provides a retrieval basis for subsequent recall of target code snippets and data support for subsequent generation of model responses.

[0507] Step 404: Based on the user question, recall relevant target code snippets from the code repositories in the code library.

[0508] The target code snippets are relevant code segments recalled from the code repositories in the code library. The target code snippets are closely related to the user's question. The target code snippets can be code files, code functions, code classes, or code variables, which are not limited here.

[0509] Based on the user question, to recall relevant target code snippets from the code repositories in the code library, an optional method is: using the retrieval engine of the database, with the user question as the retrieval keyword, recall relevant target code snippets from the code repositories in the code library.

[0510] Exemplarily, using the retrieval engine of the code library, with the user question as the retrieval keyword, recall 10 relevant target code snippets from the'streamsexplorer' code repository:

[0511] [backend / tests / test_metricprovider.py]:

[0512] <Fragment 1>< / Fragment 1>... <Fragment 10>< / Fragment 10>

[0513] Based on the user question, recalling relevant target code snippets from the code repositories in the code library provides context information for subsequent code Q&A.

[0514] Step 406: Using the pre-trained code Q&A model, based on the target code snippets, generate a model response corresponding to the user question, where the code Q&A model is trained according to the training method of the above Q&A model.

[0515] The model response is a natural language response generated for the user question, which is a specific, strongly relevant, and high-quality response regarding the details of the code repository. The model response refers to the target code snippets and provides detailed explanations and relevant information. The model response generates a specific and accurate response based on the content of the target code snippets. For explanatory Q&A, the model response explains the functions and implementation details of the code.

[0516] The embodiments of this specification and the training method of the above question-and-answer model belong to the same concept. For the detailed content not described in step 406, reference can be made to the description of the technical solution of the training method of the above question-and-answer model, which will not be elaborated here.

[0517] Exemplarily, the Prompt input to the large language model is:

[0518] "You are a question-and-answer assistant for code repositories.

[0519] I will provide you with the second code snippets retrieved according to the sample questions, which may be helpful for answering the questions. You need to answer the questions raised by the user based on the given information.

[0520] I will provide the retrieved second code snippets that may be helpful for the question-and-answer in the <retrieval>< / retrieval> area. Finally, the sample questions will be provided in the <query>< / query> area.

[0521] Let's start now.

[0522] ## Retrieval Results

[0523] [backend / tests / test_metricprovider.py]:

[0524] <Fragment 1>< / Fragment 1>……<Fragment 10>< / Fragment 10>

[0525] ## User Questions

[0526] <query> {query}{refine_query}< / query>

[0527] Please consider the retrieval results comprehensively and reply to the user's questions;

[0528] Please reply in Chinese;

[0529] The final returned result should be output in Markdown format.

[0530] The content format requirements are as follows. Please do not directly output the required text in the answer:

[0531] 1. Please reply to the questions raised by the user. Do not make excessive extensions in the content and do not output the table of contents tree;

[0532] 2. If the answer content involves specific code, do not show the fragment id. The user is not aware of the recalled code snippets. A good reply form is to give the appropriately abbreviated code and then give the explanation of the code;

[0533] 3. If the question answer does not need to involve specific code, such as asking about the architecture, etc., avoid outputting example code and other content in the answer;

[0534] Rendering format requirements:

[0535] 1. When the answer involves a specific file, make sure to use the [filename](file: / / filepath) hyperlink format, such as [user.java](file: / / extensions / firefox / user.java);

[0536] 2. Strictly ensure that the reference file is represented in the format of [filename](file: / / filepath), where filename is the file name without the file path, and filepath contains the full file path!

[0537] 3. If you use the entire path, make sure to use the [directory_name](file: / / filepath) format, such as [firefox](file: / / extensions / firefox / )"

[0538] Using 10 target code snippets as context information, generate user questions (“How does the mock_pull_metric function process Prometheus metric data?”

[0539] ) The corresponding model reply:

[0540] The mock_pull_metric function is an asynchronous function that processes and queries Prometheus metric data. The function receives a PrometheusMetric object as a parameter and returns data related to the metric.

[0541] Specifically, the mock_pull_metric function gets the metric data corresponding to the given PrometheusMetric object from the prometheus_data dictionary. Here is the definition of the function:

[0542] async def mock_pull_metric(_, metric: PrometheusMetric):

[0543] return prometheus_data[metric.metric]

[0544] parameter:

[0545] _: Placeholder parameter, usually not used.

[0546] metric: A PrometheusMetric object containing the name of the metric to be queried.

[0547] Return value:

[0548] Returns the value corresponding to the metric.metric key in the prometheus_data dictionary.

[0549] The PrometheusMetric class is defined as follows:

[0550] class PrometheusMetric:

[0551] def __init__(self, name: str):

[0552] self.name = name

[0553] self.metric = f"{name}_metric"

[0554] def __repr__(self):

[0555] return f"PrometheusMetric(name={self.name}, metric={self.metric})"

[0556] property:

[0557] name: The name of the indicator.

[0558] metric: The full name of the metric in the format of {name}_metric.

[0559] In this way, the mock_pull_metric function can simulate the process of obtaining indicator data from Prometheus and return the corresponding data. "

[0560] In the embodiments of this specification, a code question-and-answer model is trained using high-quality sample data pre-constructed by a data construction method, and model responses corresponding to user questions are generated based on target code snippets, thereby completing warehouse-level, higher-quality code question-and-answer processing adapted to the code repository and improving the user experience.

[0561] The following combination Figure 5 , taking the application of the data construction method provided in this specification in training an intelligent coding assistant as an example, the above data construction method is further explained. Figure 5A flowchart of a data construction method for training an intelligent coding assistant provided by an embodiment of the present specification is shown, including the following specific steps:

[0562] Step 502: Obtain development issues and function iteration requests of the open source code library.

[0563] Step 504: Using the search engine of the open source database built using the LangChain framework and the BM25 recall algorithm, the development issues and function iteration requests are used as search keywords to recall the relevant first code snippets from the open source code base.

[0564] Step 506: Utilize the large language model and the directory structure information of the open source code base as context information, and generate initial sample questions and reference answers for the first code snippet under the constraint that the length of the first code snippet meets the input limit of the large language model.

[0565] Step 508: Utilize the large language model to extract keyword information from the initial sample question, and based on the keyword information, use the directory structure information of the sample code repository in the open source code library as context information to rewrite the initial sample question to obtain a sample question.

[0566] Step 510: Using the LangChain framework and the BM25 recall algorithm to build a search engine for an open source database, the first code snippet, keyword information, sample questions, and reference answers are used as search keywords to recall the relevant second code snippet from the open source code library.

[0567] Step 512: Utilizing the large language model, and subject to the constraint that the length of the second code snippet satisfies the input limit of the large language model, using the directory structure information of the second code snippet and the sample code repository as context information, generate sample responses corresponding to the sample questions, wherein the sample questions and sample responses are sample data for training an intelligent coding assistant, and the intelligent coding assistant is applicable to code repositories in open source code repositories.

[0568] In the embodiments of this specification, compared with the problems of high data annotation cost, small data scale, and difficult to ensure data quality in manual data annotation solutions, based on the widely available open source code base, a multi-step code base recall strategy is used to gradually generate code base-related sample question and answer pairs, so that a large amount of warehouse-level code question and answer data can be obtained efficiently and with high quality. In view of the problem that the sample questions constructed in the single-step code base recall strategy cannot correspond to the code warehouse details, and the code retrieval recall accuracy is not high, through the multi-step code base retrieval recall strategy, a large language model can be used in the first round of code snippet retrieval to generate initial sample questions specific to the code base details. At the same time, in the second round of code snippet retrieval, the keyword information of the generated questions, the directory structure information of the code warehouse, and other information are used to overcome the inherent differences between code language and natural language, and the accuracy of the code snippet retrieval recall related to the question is further improved, thereby generating more accurate sample replies, effectively improving the data quality of the sample data constructed for training the intelligent coding assistant.

[0569] Corresponding to the above method embodiment, this specification also provides a task platform embodiment, Figure 6 FIG. 1 shows a schematic diagram of a task platform provided by an embodiment of the present specification. Figure 6 As shown, the task platform 600 includes a sample data construction interface 602 and a first response unit 604;

[0570] The sample data construction interface 602 is used to receive a sample data construction request sent by a terminal, wherein the sample data construction request carries metadata information of the code library;

[0571] The first response unit 604 is used to recall the relevant first code snippet from the code library based on the metadata information, generate an initial sample question for the first code snippet using a text processing model, rewrite the initial sample question based on the directory structure information of the sample code repository in the code library using the text processing model to obtain a sample question, recall the relevant second code snippet from the code library based on the first code snippet and the sample question, and generate a sample answer corresponding to the sample question based on the second code snippet using the text processing model, wherein the sample question and the sample answer are sample data for training a code question and answer model, and the code question and answer model is applicable to a code repository.

[0572] The task platform 600 is a comprehensive virtual platform for automatically building high-quality sample data and training code question-answering models. It receives requests sent by terminals, processes these requests, and generates corresponding sample data or trained models.

[0573] The sample data construction interface 602 is a module in the task platform 600, which is responsible for receiving sample data construction requests from the terminal and passing these requests to the first response unit 604 for processing. The first response unit 604 is the core processing module in the task platform 600, which is responsible for performing a series of operations based on metadata information, including code snippet recall, sample question generation, sample question rewriting, sample answer generation, and model training.

[0574] In the embodiment of the present specification, the first response unit utilizes a text processing model to rewrite the initial sample question based on the directory structure information of the sample code repository in the code library to obtain a sample question that is specific to the details of the sample code repository and is strongly correlated; based on the first code snippet and the sample question that is specific to the details of the sample code repository and is strongly correlated, a related second code snippet is recalled from the code library, thereby improving the accuracy of code snippet retrieval and recall; utilizing the text processing model, based on the high-quality second code snippet, accurate sample replies corresponding to the sample questions are generated, highly accurate sample questions and sample replies are obtained, and automatic construction of high-quality sample data is achieved, thereby improving data construction efficiency and reducing data construction costs.

[0575] Optionally, the task platform 600 also includes a model training interface;

[0576] Model training interface, used to receive model training requests sent by the terminal;

[0577] The first response unit 604 is also used to respond to the model training request, use the initial question and answer model to generate a predicted answer corresponding to the sample question, and train the initial question and answer model based on the predicted answer and the sample answer to obtain a code question and answer model suitable for the code repository.

[0578] The model training interface is a module in the task platform 600, which is responsible for receiving model training requests from the terminal and passing these requests to the first response unit 604 for processing.

[0579] In the embodiment of this specification, the first response unit trains a general initial question and answer model based on high-quality sample questions and sample answers to obtain a code question and answer model suitable for the code repository in the code base, which can complete warehouse-level, higher-quality code question and answer processing adapted to the code repository, thereby improving the user experience.

[0580] The above is a schematic scheme of a task platform of this embodiment. It should be noted that the technical scheme of the task platform and the technical scheme of the above-mentioned data construction method and question-answering model training method belong to the same concept. For details not described in detail in the technical scheme of the task platform, please refer to the description of the technical scheme of the above-mentioned data construction method or question-answering model training method.

[0581] Corresponding to the above method embodiment, this specification also provides a code question and answer system embodiment, Figure 7 FIG. 1 shows a schematic diagram of the structure of a code question answering system provided by an embodiment of this specification. Figure 7 As shown, the code question and answer system 700 includes a question and answer interface 702 and a second response unit 704;

[0582] The question-and-answer interface 702 is used to receive a code question-and-answer request sent by a terminal, wherein the code question-and-answer request carries a user question for a target code;

[0583] The second response unit 704 is used to recall relevant target code snippets from the code repository in the code library based on the user question, and use the pre-trained code question-answering model to generate a model response corresponding to the user question based on the target code snippet, wherein the code question-answering model is trained according to the training method of the question-answering model;

[0584] The question-answer interface 702 is also used to feed back the model answer to the terminal.

[0585] The code question answering system 700 is an intelligent question answering system for receiving code-related questions raised by users, and can generate detailed replies based on a pre-trained code question answering model. The system can recall relevant code snippets from the code library and use these code snippets as context information to generate accurate replies.

[0586] The question-and-answer interface 702 is a module in the code question-and-answer system 700 , which is responsible for receiving code question-and-answer requests from the terminal and feeding back the generated model responses to the terminal.

[0587] The second response unit 704 is the core processing module in the code question and answer system 700, responsible for recalling relevant target code snippets from the code library based on user questions, and generating model responses corresponding to user questions using a pre-trained code question and answer model.

[0588] In the embodiment of the present specification, the second response unit recalls relevant target code snippets from the code repository in the code base based on the user question, and uses a pre-trained code question and answer model to generate a model response corresponding to the user question with the target code snippet as context information, thereby completing a warehouse-level, higher-quality code question and answer processing adapted to the code repository and improving the user experience.

[0589] The above is a schematic scheme of a code question and answer system of this embodiment. It should be noted that the technical scheme of the code question and answer system and the technical scheme of the code question and answer method described above are of the same concept, and the details not described in detail in the technical scheme of the code question and answer system can be found in the description of the technical scheme of the code question and answer method described above.

[0590] Corresponding to the above method embodiment, this specification also provides a data construction device embodiment, the device comprising:

[0591] A first acquisition module is configured to acquire a first code snippet and an initial sample question for the first code snippet;

[0592] A first rewriting module is configured to rewrite the initial sample question by using a text processing model based on directory structure information of a sample code repository in a code library to obtain a sample question;

[0593] A first recall module is configured to recall a related second code snippet from a code base based on the first code snippet and the sample question;

[0594] The first generation module is configured to use the text processing model to generate a sample answer corresponding to the sample question based on the second code snippet, wherein the sample question and the sample answer are sample data for training a code question and answer model, and the code question and answer model is applicable to a code repository.

[0595] Optionally, the first acquisition module is further configured to: acquire metadata information of the code library; and recall the relevant first code snippet from the code library based on the metadata information.

[0596] Optionally, the first acquisition module is further configured to: utilize a retrieval engine of the code library to recall the relevant first code snippet from the code library based on metadata information.

[0597] Optionally, the first acquisition module is further configured to: generate initial sample questions for the first code snippet based on directory structure information of the code library using a text processing model.

[0598] Optionally, the first rewriting module is further configured to: extract keyword information from the initial sample question using a text processing model, and rewrite the initial sample question based on the keyword information and directory structure information of the sample code repository in the code library to obtain a sample question.

[0599] Optionally, the first recall module is further configured to: recall relevant second code snippets from the code library based on the first code snippet, keyword information and sample questions.

[0600] Optionally, the first recall module is further configured to: utilize a retrieval engine of the code library to recall relevant second code snippets from the code library based on the first code snippet, keyword information and sample questions.

[0601] Optionally, the first generation module is further configured to: utilize a text processing model to generate a sample answer corresponding to the sample question based on the second code snippet and directory structure information of the sample code repository.

[0602] The above is a schematic scheme of a data construction device of this embodiment. It should be noted that the technical scheme of the data construction device and the technical scheme of the above data construction method belong to the same concept, and the details not described in detail in the technical scheme of the data construction device can be referred to the description of the technical scheme of the above data construction method.

[0603] Corresponding to the above method embodiment, this specification also provides an embodiment of a training device for a question-answering model, the device comprising:

[0604] A second acquisition module is configured to acquire sample questions and sample answers, wherein the sample questions and sample answers are constructed according to the data construction device;

[0605] A second generation module is configured to generate a predicted answer corresponding to the sample question using the initial question-answering model;

[0606] The second training module is configured to train the initial question-answering model based on the predicted answers and the sample answers to obtain a code question-answering model suitable for the code repository in the code base.

[0607] The above is a schematic scheme of a training device for a question-answering model of this embodiment. It should be noted that the technical scheme of the training device for the question-answering model and the technical scheme of the training method for the question-answering model belong to the same concept, and the details not described in detail in the technical scheme of the training device for the question-answering model can be found in the description of the technical scheme of the training method for the question-answering model.

[0608] Corresponding to the above method embodiment, this specification also provides a code question and answer device embodiment, the device comprising:

[0609] A third acquisition module is configured to acquire user questions for the target code;

[0610] The third recall module is configured to recall relevant target code snippets from the code repository in the code base based on the user question;

[0611] The third generation module is configured to use a pre-trained code question and answer model to generate a model response corresponding to the user's question with the target code snippet as context information, wherein the code question and answer model is trained according to the training device of the above-mentioned question and answer model.

[0612] The above is a schematic scheme of a code question and answer device of this embodiment. It should be noted that the technical scheme of the code question and answer device and the technical scheme of the above-mentioned code question and answer method belong to the same concept, and the details not described in detail in the technical scheme of the code question and answer device can be referred to the description of the technical scheme of the above-mentioned code question and answer method.

[0613] Figure 8 The following is a block diagram of a computing device provided by an embodiment of the present specification. The computing device 800 includes:

[0614] Memory 810 and processor 820;

[0615] The memory 810 is used to store computer programs / instructions, and the processor 820 is used to execute the computer programs / instructions. When the computer program / instructions are executed by the processor 820, the steps of the above-mentioned data construction method, question and answer model training method or code question and answer method are implemented.

[0616] In one or more embodiments of the present specification, the computing device may be understood as an integrated intelligent terminal, including but not limited to a server, a desktop computer, a PC (Personal Computer), an all-in-one model machine, a mobile phone, a tablet computer or other portable intelligent terminal, etc., and the computing device may be pre-installed with the model described in the above embodiments of the present application.

[0617] Specifically, the computing device can pre-set multiple types of models, including but not limited to models in the fields of natural language processing, visual processing, speech processing, code processing, multimodal task processing, etc., so as to provide a variety of model choices. In different product forms, the computing device can support one or more model usage methods, including but not limited to model training, model calling, model fine-tuning, model deployment, model reasoning and application, etc. In some product forms, the computing device also supports model management, including but not limited to multi-type model management (supporting the management of multiple types of models such as discriminant and generative models), model version control (supporting the control of different model versions), model evaluation (based on model evaluation tools, evaluating the performance and effect of the model), etc. In other product forms, the computing device can also create applications based on models, provide API calling capabilities, and can call models to the created applications through API interfaces, while providing application management tools to achieve management and monitoring of applications.

[0618] Furthermore, the computing device can also include data management (supporting the creation and management of model tuning data sets), training center (providing rich training resources to help users learn and master AI technology), and basic management and control capabilities (providing enterprise-level basic management and control capabilities to ensure the security and efficient operation of the system). Through the above functions, a comprehensive, integrated AI development, training, deployment and application device is provided.

[0619] In addition, the components of the computing device 800 include but are not limited to a memory 810 and a processor 820. The processor 820 and the memory 810 may be connected via a bus.

[0620] The computing device 800 may also include an access device that enables the computing device 800 to communicate with a database storing data via one or more networks. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device may include one or more of any type of network interface (e.g., a Network Interface Controller (NIC)) of wired or wireless type, such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, and a Near Field Communication (NFC).

[0621] In one embodiment of the present specification, the above components of the computing device 800 and Figure 8 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 8 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0622] The computing device 800 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 800 may also be a mobile or stationary server.

[0623] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical schemes of the above-mentioned data construction method, question-answering model training method and code question-answering method belong to the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical schemes of the above-mentioned data construction method, question-answering model training method or code question-answering method.

[0624] An embodiment of the present specification also provides a computer-readable storage medium storing a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned data construction method, question-and-answer model training method, or code question-and-answer method.

[0625] An embodiment of the present specification also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the above-mentioned data construction method, question-and-answer model training method, or code question-and-answer method.

[0626] The above is a schematic scheme of a computer-readable storage medium and a computer program product of this embodiment. It should be noted that the technical scheme of the computer-readable storage medium and the computer program product belongs to the same concept as the technical scheme of the above-mentioned methods. For details not described in detail in the technical scheme of the computer-readable storage medium and the computer program product, please refer to the description of the technical scheme of the above-mentioned methods.

[0627] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0628] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the contents contained in the computer readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer readable media do not include electric carrier signals and telecommunication signals.

[0629] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0630] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0631] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A data construction method, comprising: Obtaining a first code snippet and an initial sample question for the first code snippet; Using a text processing model, based on directory structure information of a sample code repository in a code base, the initial sample question is rewritten to obtain a sample question; Recalling a related second code snippet from the code library based on the first code snippet and the sample problem; Utilizing the text processing model, based on the second code snippet, a sample answer corresponding to the sample question is generated, wherein the sample question and the sample answer are sample data for training a code question and answer model, and the code question and answer model is a deep learning model specifically used to answer questions regarding the code repository in the code library.

2. The method according to claim 1, wherein obtaining the first code snippet comprises: Get metadata information of the code base; Based on the metadata information, a related first code snippet is retrieved from the code library.

3. The method according to claim 2, wherein the step of recalling the first code snippet from the code base based on the metadata information comprises: Recalling the relevant first code snippet from the code library based on the metadata information by using the search engine of the code library.

4. The method according to claim 1, wherein obtaining an initial sample question for the first code snippet comprises: An initial sample question for the first code snippet is generated by using a text processing model based on directory structure information of the code library.

5. The method according to any one of claims 1 to 4, wherein the using of a text processing model to rewrite the initial sample question based on directory structure information of a sample code repository in a code library to obtain a sample question comprises: The text processing model is used to extract keyword information from the initial sample question, and based on the keyword information and directory structure information of the sample code repository in the code library, the initial sample question is rewritten to obtain a sample question.

6. The method according to claim 5, wherein the step of recalling a related second code snippet from the code base based on the first code snippet and the sample problem comprises: Based on the first code snippet, the keyword information and the sample question, a related second code snippet is recalled from the code library.

7. The method according to claim 6, wherein the step of recalling a related second code snippet from the code library based on the first code snippet, the keyword information and the sample question comprises: Recalling a related second code snippet from the code library based on the first code snippet, the keyword information and the sample question using a search engine of the code library.

8. The method according to any one of claims 1 to 4, wherein the step of using the text processing model to generate a sample answer corresponding to the sample question based on the second code snippet comprises: The text processing model is used to generate a sample answer corresponding to the sample question based on the second code snippet and the directory structure information of the sample code repository.

9. A method for training a question-answering model, comprising: Obtaining sample questions and sample answers, wherein the sample questions and the sample answers are constructed according to the data construction method according to any one of claims 1 to 8; Using the initial question-answering model, generating a predicted answer corresponding to the sample question; Based on the predicted answers and the sample answers, the initial question-answering model is trained to obtain a code question-answering model applicable to the code repository in the code base, wherein the code question-answering model is a deep learning model specifically used to answer questions regarding the code repository in the code base.

10. A code question and answer method, comprising: Get user questions for the target code; Based on the user question, recall relevant target code snippets from the code repository in the code library; A pre-trained code question-answering model is used to generate a model response corresponding to the user question based on the target code snippet, wherein the code question-answering model is trained according to the training method for the question-answering model according to claim 9.

11. A task platform, including a sample data construction interface and a first response unit; The data construction interface is used to receive a sample data construction request sent by a terminal, wherein: The sample data construction request carries metadata information of the code library; The first response unit is used to recall the relevant first code snippet from the code base based on the metadata information, generate an initial sample question for the first code snippet using a text processing model, rewrite the initial sample question based on directory structure information of a sample code repository in the code base using the text processing model to obtain a sample question, recall the relevant second code snippet from the code base based on the first code snippet and the sample question, and generate a sample answer corresponding to the sample question based on the second code snippet using the text processing model, wherein the sample question and the sample answer are sample data for training a code question and answer model, and the code question and answer model is a deep learning model specifically used to answer questions about a code repository in the code base.

12. The task platform according to claim 11, further comprising a model training interface; The model training interface is used to receive a model training request sent by a terminal; The first response unit is also used to respond to the model training request, use the initial question and answer model to generate a predicted answer corresponding to the sample question, and train the initial question and answer model based on the predicted answer and the sample answer to obtain a code question and answer model suitable for the code repository.

13. A code question and answer system, comprising a question and answer interface and a second response unit; The question-and-answer interface is used to receive a code question-and-answer request sent by a terminal, wherein: The code question and answer request carries a user question regarding the target code; The second response unit is used to recall relevant target code snippets from a code repository in a code library based on the user question, and generate a model response corresponding to the user question based on the target code snippet using a pre-trained code question-answering model, wherein the code question-answering model is trained according to the training method of the question-answering model according to claim 9; The question-and-answer interface is also used to feed back the model answer to the terminal.

14. A computing device comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the method described in any one of claims 1 to 10 are implemented.

15. A computer-readable storage medium storing a computer program / instruction, wherein the computer program / instruction, when executed by a processor, implements the steps of the method according to any one of claims 1 to 10.

16. A computer program product, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 10.

Images