Code processing model training method, code processing model testing method, and code task processing method

Abstract

Embodiments of the present disclosure provide a code processing model training method, a code processing model testing method, and a code task processing method. The code processing model training method comprises: acquiring a sample code document and sample question-answer code; extracting sample completion code and sample context code of the sample completion code from the sample code document; converting the sample context code into a question-answer format to obtain sample instruction code; and training an initial processing model using the sample question-answer code, the sample completion code, and the sample instruction code to obtain a code processing model. A model is trained using sample question-answer code, sample completion code, and sample instruction code, enabling a code processing model to have both code question-answering and completion capabilities. By converting sample context code into a question-answer format, the processing capability of a code processing model is improved.

Description

Methods for training, testing, and processing code processing models

[0001] This disclosure claims priority to Chinese Patent Application No. 202411683907.2, filed with the China Patent Office on November 21, 2024, entitled “Code Processing Model Training, Code Processing Model Testing and Code Task Processing Method”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the fields of computer technology and artificial intelligence technology, and in particular to methods for training code processing models, testing code processing models, and processing code tasks. Background Technology

[0003] With the development of computer technology, large-scale models have begun to shine, demonstrating remarkable capabilities in language understanding, generation, interaction, and reasoning, and are widely applied in natural language processing fields such as dialogue, translation, and code processing. Taking code processing as an example, large-scale models can provide developers with intelligent capabilities such as code completion and code question answering, thus gradually becoming a research focus.

[0004] Currently, different models are typically trained for different tasks. However, this approach consumes a lot of resources. Therefore, there is an urgent need for a solution that can train models to have different code processing capabilities. Summary of the Invention

[0005] In view of the above, embodiments of this disclosure provide a code processing model training method. One or more embodiments of this disclosure also relate to a code processing model testing method, a code task processing method, an information processing method based on a code processing model, a code processing model training device, a code processing model testing device, a code task processing device, an information processing device based on a code processing model, a task platform, a computing device, an electronic device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0006] According to a first aspect of the present disclosure, a method for training a code processing model is provided, comprising:

[0007] Obtain sample code documentation and sample Q&A code;

[0008] Extract the sample completion code and the sample context code of the sample completion code from the sample code document;

[0009] The sample context code is converted into a question-and-answer format to obtain the sample instruction code;

[0010] The initial processing model is trained using sample question-and-answer code, sample completion code, and sample instruction code to obtain a code processing model.

[0011] According to a second aspect of the present disclosure, a code processing model testing method is provided, comprising:

[0012] Get the test code documentation from the code repository;

[0013] Extract the test context code from the test code document, and convert the test context code into a question-and-answer format to obtain the test instruction code;

[0014] The test instruction code is input into the code processing model to obtain the test completion code. The code processing model is trained using the code processing model training method.

[0015] Complete the code based on the test and generate model test results for the code processing model.

[0016] According to a third aspect of the present disclosure, a code task processing method is provided, comprising:

[0017] Obtain the code data to be processed for the target code task;

[0018] The code data to be processed is input into the code processing model to obtain the code processing result. The code processing model is trained using the code processing model training method.

[0019] According to a fourth aspect of the present disclosure, an information processing method based on a code processing model is provided, applied to a task platform, comprising:

[0020] Receive a model request sent by a terminal device, wherein the model request includes at least one of the following: scene identifier of the target scene, scene input data of the target scene, and model specification parameters;

[0021] Based on the model request, the target code processing model is determined from multiple code processing models, wherein the multiple code processing models are trained based on the code processing model training method.

[0022] According to a fifth aspect of the present disclosure, a task platform is provided, comprising:

[0023] The request interface is used to receive model requests sent by terminal devices. The model request includes at least one of the following: the scene identifier of the target scene, the scene input data of the target scene, and the model specification parameters.

[0024] The response unit is used to determine the target code processing model from multiple code processing models based on the model request, wherein the multiple code processing models are trained based on the code processing model training method.

[0025] According to a sixth aspect of the present disclosure, a code processing model training apparatus is provided, comprising:

[0026] The first acquisition module is configured to acquire sample code documents and sample question-and-answer code.

[0027] The first extraction module is configured to extract the sample completion code and the sample context code of the sample completion code from the sample code document;

[0028] The first conversion module is configured to convert the sample context code into a question-and-answer format to obtain the sample instruction code;

[0029] The training module is configured to train the initial processing model using sample question-and-answer code, sample completion code, and sample instruction code to obtain a code processing model.

[0030] According to a seventh aspect of the present disclosure, a code processing model testing apparatus is provided, comprising:

[0031] The second acquisition module is configured to retrieve test code documentation from the code repository;

[0032] The second extraction module is configured to extract test context code from the test code document and convert the test context code into a question-and-answer format to obtain test instruction code;

[0033] The first input module is configured to input test instruction code into the code processing model to obtain test completion code, wherein the code processing model is trained using a code processing model training method;

[0034] The generation module is configured to complete the code based on the tests and generate model test results for the code processing model.

[0035] According to an eighth aspect of the present disclosure, a code task processing apparatus is provided, comprising:

[0036] The third acquisition module is configured to acquire the code data to be processed for the target code task;

[0037] The second input module is configured to input the code data to be processed into the code processing model to obtain the code processing result, wherein the code processing model is trained using the code processing model training method.

[0038] According to a ninth aspect of the present disclosure, an information processing apparatus based on a code processing model is provided, applied to a task platform, comprising:

[0039] The receiving module is configured to receive a model request sent by a terminal device, wherein the model request includes at least one of the following: a scene identifier of the target scene, scene input data of the target scene, and model specification parameters.

[0040] The determination module is configured to determine the target code processing model from multiple code processing models based on a model request, wherein the multiple code processing models are trained based on a code processing model training method.

[0041] According to a tenth aspect of the present disclosure, a computing device is provided, comprising:

[0042] Memory and processor;

[0043] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the methods provided in the first, second, third, or fourth aspects described above.

[0044] According to an eleventh aspect of the present disclosure, an electronic device is provided, comprising:

[0045] The memory and processor are connected via a bus;

[0046] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the methods provided in the first, second, third, or fourth aspects described above.

[0047] According to a twelfth aspect of the present disclosure, a computer-readable storage medium is provided that stores a computer program / instructions that, when executed by a processor, implement the steps of the methods provided in the first, second, third, or fourth aspects described above.

[0048] According to a thirteenth aspect of the present disclosure, a computer program product is provided, including a computer program / instructions that, when executed by a processor, implement the steps of the methods provided in the first, second, third, or fourth aspects described above.

[0049] This disclosure provides a code processing model training method according to one embodiment, comprising: acquiring sample code documents and sample question-and-answer codes; extracting sample completion codes and sample context codes of the sample completion codes from the sample code documents; performing question-and-answer format conversion on the sample context codes to obtain sample instruction codes; and training an initial processing model using the sample question-and-answer codes, sample completion codes, and sample instruction codes to obtain a code processing model. By training the initial processing model using sample question-and-answer codes, sample completion codes, and sample instruction codes, the code processing model simultaneously possesses code question-and-answer and code completion capabilities. Furthermore, by performing question-and-answer format conversion on the sample context codes, the format of code question-and-answer and code completion data is unified, thereby avoiding the problem of model capability degradation caused by inconsistent training data formats and improving the processing capability of the code processing model. Attached Figure Description

[0050] Figure 1 is an architecture diagram of a code task processing system provided in an embodiment of this disclosure;

[0051] Figure 2 is a flowchart of a code processing model training method provided in an embodiment of this disclosure;

[0052] Figure 3 is a flowchart of a code processing model testing method provided in an embodiment of this disclosure;

[0053] Figure 4 is a flowchart of a code task processing method provided in an embodiment of this disclosure;

[0054] Figure 5 is a flowchart of a code task processing method provided in an embodiment of this disclosure;

[0055] Figure 6 is a flowchart of an information processing method based on a code processing model provided in an embodiment of this disclosure;

[0056] Figure 7 is a schematic diagram of the structure of a task platform provided in an embodiment of this disclosure;

[0057] Figure 8 is a schematic diagram of the structure of a code processing model training device provided in an embodiment of this disclosure;

[0058] Figure 9 is a schematic diagram of the structure of a code processing model testing device provided in an embodiment of this disclosure;

[0059] Figure 10 is a schematic diagram of the structure of a code task processing device provided in an embodiment of the present disclosure;

[0060] Figure 11 is a schematic diagram of the structure of an information processing device based on a code processing model according to an embodiment of the present disclosure;

[0061] Figure 12 is a structural block diagram of a computing device provided in an embodiment of the present disclosure;

[0062] Figure 13 is a structural block diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0063] Numerous specific details are set forth in the following description to provide a full understanding of this disclosure. However, this disclosure can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this disclosure. Therefore, this disclosure is not limited to the specific implementations disclosed below.

[0064] The terminology used in one or more embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this disclosure. The singular forms “a,” “the,” and “the” as used in one or more embodiments of this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this disclosure refers to and includes any or all possible combinations of one or more associated listed items.

[0065] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this disclosure, and similarly, second may also be referred to as first. Depending on the context, the word “if” as used herein may be interpreted as “when”, “in response to a determination”, or “when…”.

[0066] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this disclosure are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0067] In one or more embodiments of this disclosure, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.

[0068] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as natural language processing tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios for large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0069] First, the terms and concepts involved in one or more embodiments of this disclosure will be explained.

[0070] Bidirectional Encoder Representations from Transformers (BERT) is a pre-trained NLP model. By learning from large amounts of unlabeled text data, this model can capture deep semantic information in text and achieves significant performance improvements on numerous NLP tasks.

[0071] The Text-to-Text Transfer Transformer (T5) is a deep self-attention model for text-to-text communication. A key feature of the T5 model is that it unifies all NLP tasks into a text-to-text format, meaning both input and output are text sequences. This design allows the model to more easily adapt to various tasks, such as translation, question answering, and summarization.

[0072] Large-scale language models refer to deep learning models with a large number of parameters, especially in the field of Natural Language Processing (NLP). These models learn from massive amounts of text data, enabling them to understand and generate natural language, demonstrating strong contextual understanding, generation, and reasoning abilities. Large-scale language models include, but are not limited to, BERT and T5. Due to their scale and complexity, large-scale language models are often better able to capture the subtle differences and complex structures of language.

[0073] Supervised fine-tuning (SFT) is a method of further training a model based on a pre-trained model. In this method, the model is trained on a dataset containing pairs of inputs and desired outputs so that it learns how to generate responses closer to human-level performance. Supervised fine-tuning is often used to adapt a model to task-specific or domain-specific data, thereby improving its performance on those tasks.

[0074] Code completion: refers to the task of completing the current code based on the code context.

[0075] Evaluation benchmark: refers to a set of standardized tests or tasks used to measure and compare the performance of different systems, algorithms or models.

[0076] Exact Match (EM): This is a simple and direct text matching method that requires the model's output to be exactly the same as the standard answer, meaning that every character must correspond one-to-one. The EM value is 1 only when the two texts are completely identical; otherwise, it is 0.

[0077] Edit Similarity (ES) is a method used to measure the differences between two strings. It is defined as the minimum number of editing operations required to transform one string into another. These editing operations include inserting a character, deleting a character, or replacing a character.

[0078] Limited context length: Code tasks typically require a deep understanding of a large codebase, including but not limited to function definitions, variable scope, and class structure. If the evaluation benchmark does not provide sufficient contextual information, it cannot accurately test the model's performance in a real-world development environment.

[0079] Relying on superficial evaluation metrics: Traditional evaluation methods often focus too much on the literal matching of the output or simple syntactic correctness, while neglecting deeper factors such as functional implementation quality, code readability, and maintainability. This evaluation standard based on superficial features may cause the model to optimize for the wrong goal, that is, to pursue high scores rather than truly improving code quality.

[0080] Potential overfitting to the training dataset: If the evaluation benchmark is too similar to the training dataset, then even if the model performs well on that specific dataset, it cannot be proven that it has good generalization ability. In real-world applications, models need to face a wide variety of new situations, so their ability to adapt to unknown scenarios is crucial.

[0081] tree-sitter is a language toolkit for parsing source code. It provides a high-performance parser generator and syntax tree parsing support for multiple programming languages.

[0082] tree-sitter-languages ​​is a collection of tree-sitter parsers for multiple programming languages, making it easy to integrate support for multiple languages ​​into a project.

[0083] This disclosure provides a code processing model training method, and also relates to a code task processing method, an information processing method based on a code processing model, a code processing model training device, a code task processing device, an information processing device based on a code processing model, a task platform, a computing device, an electronic device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0084] Considering the large number of model parameters in the code processing model and the limited computing resources of the client, the code task processing method proposed in this disclosure can be applied to the code task processing system shown in Figure 1, but is not limited thereto. Referring to Figure 1, Figure 1 shows an architecture diagram of a code task processing system provided in an embodiment of this disclosure. The code task processing system may include a client 100 and a server 200;

[0085] Client 100 is used to send the code data to be processed for the target code task to server 200;

[0086] Server 200 is used to input the code data to be processed into the code processing model and obtain the code processing result. The code processing model is trained on an initial processing model based on sample question-and-answer code, sample completion code, and sample instruction code. The sample instruction code is obtained by converting the sample context code into a question-and-answer format. The sample context code and sample completion code are selected from multiple sample code fragments based on code relationships. The multiple sample code fragments and the code relationships between them are obtained by parsing the sample code document. The code processing result is then sent to client 100.

[0087] Client 100 is also used to receive the code processing results sent by server 200.

[0088] As shown in Figure 1, the code processing model is deployed in server 200. Server 200 can connect to one or more clients 100 via a local area network (LAN), wide area network (WAN), internet connection, or other types of data network. Client 100 may include, but is not limited to, smartphones, tablets, laptops, PDAs, personal computers, smart home devices, and in-vehicle devices. Client 100 can also interact with users through a graphical user interface to invoke the code processing model, thereby implementing the code task processing method provided in this embodiment.

[0089] It is worth noting that the code task processing method provided in the embodiments of this disclosure is generally executed by the server. However, in other embodiments of this disclosure, if the client's runtime resources can meet the deployment and runtime conditions of the code processing model, the client can also have similar functions to the server, thereby executing the code task processing method provided in the embodiments of this disclosure. In other embodiments, the code task processing method provided in the embodiments of this disclosure can also be jointly executed by the client and the server.

[0090] Referring to Figure 2, Figure 2 shows a flowchart of a code processing model training method provided in an embodiment of this disclosure, which specifically includes the following steps:

[0091] Step 202: Obtain the sample code document and sample Q&A code.

[0092] In one or more embodiments of this disclosure, in order to train a code processing model that simultaneously supports code question answering and code completion tasks, sample code documents and sample question-and-answer code can be obtained, thereby training the model's code completion capability based on the sample code documents and training the model's code question answering capability based on the sample question-and-answer code.

[0093] It's important to note that sample code documentation refers to documentation including source code. There can be one or more sample code documents. Sample Q&A code refers to a collection of data surrounding code questions and their answers. There can also be one or more sample Q&A codes. Sample Q&A code includes sample code questions and sample code answers. Sample code questions are questions related to the code, typically including programming requirements, error messages encountered, attempted solutions, etc. Sample code answers are solutions provided for the sample code questions, which may include textual explanations, code correction suggestions, etc.

[0094] In practical applications, there are various ways to obtain sample code documents and sample Q&A codes, and the specific method chosen depends on the actual situation. This disclosure does not impose any limitations on these methods. Regarding sample code documents, one possible implementation is to extract them from a code repository. Another possible implementation is to receive sample code documents sent by a user through a client. Regarding sample Q&A codes, one possible implementation is to extract sample Q&A codes, including sample code questions and answers, from code history commit information. Another possible implementation is to read sample Q&A codes from other data acquisition devices or databases.

[0095] Step 204: Extract the sample completion code and the sample context code of the sample completion code from the sample code document.

[0096] It should be noted that sample completion code refers to the masked sample code fragment that enables the model to complete predictions. Sample context code refers to the known code fragment surrounding the sample completion code. Both the sample completion code and its sample context code originate from the same sample code document. Since complex dependencies may exist between different sample code documents, sample dependency code that depends on the sample completion code can also be extracted from sample dependency documents that depend on the sample code document, and this sample dependency code can be identified as the sample context code of the sample completion code.

[0097] Sample context code provides the model with the background information needed for the completion task. This information helps the model understand the location, function, and possible completion methods of the code. Sample context code includes sample prefix code and sample suffix code. Sample prefix code refers to the known code portion preceding the sample completion code. This part of the code provides contextual information preceding the sample completion code, helping the model understand the location and function of the current code. Sample suffix code refers to the known code portion following the sample completion code. This part of the code provides contextual information following the sample completion code, helping the model understand the structure and logic of the subsequent code.

[0098] In practical applications, there are various ways to extract sample completion code and sample context code from sample code documents. The specific method chosen depends on the actual situation, and this disclosure does not impose any limitations on these methods. In one possible implementation of this disclosure, a code parsing tool can be directly used to extract sample completion code and sample context code from the sample code document.

[0099] In another possible implementation of this disclosure, the sample completion code and the sample context code of the sample completion code can be extracted from the sample code document based on the code relationships in the sample code document. That is, the above extraction of the sample completion code and the sample context code of the sample completion code from the sample code document may include the following steps:

[0100] Parse the sample code documentation to determine multiple sample code snippets and the code relationships between them.

[0101] Based on code relationships, sample completion code and sample context code of sample completion code are selected from multiple sample code snippets.

[0102] It should be noted that sample code snippets are logical blocks with specific functions or characteristics extracted from sample code documents. Sample code snippets can be simple expressions, statements, functions, class definitions, or any other meaningful code blocks. Code relationships refer to the logical connections or dependencies between different sample code snippets. Code relationships include at least one of the following: call relationships, inheritance relationships, data transfer relationships, dependency relationships, and reference relationships.

[0103] In practical applications, there are various ways to parse sample code documents and determine the code relationships between multiple sample code fragments. The specific method chosen depends on the actual situation, and this disclosure does not impose any limitations on this approach. One possible implementation of this disclosure is to directly call code parsing tools (such as tree-sitter, tree-sitter-languages, etc.) or code parsing models to parse the sample code document and output multiple sample code fragments and the code relationships between them. Another possible implementation of this disclosure is to perform syntax parsing on the sample code document and determine the code relationships between multiple sample code fragments based on the abstract syntax tree obtained from the syntax parsing.

[0104] Furthermore, based on code relationships, there are various ways to select sample completion code and sample context code of sample completion code from multiple sample code fragments. The specific method chosen depends on the actual situation, and this disclosure does not impose any limitations on these methods. In one possible implementation, sample completion code can be randomly selected from multiple sample code fragments, and sample code fragments with code relationships with the sample completion code can be identified as the sample context code of the sample completion code. For example, the AST can be traversed and manipulated to randomly extract nodes from multiple levels of the AST, which can then be identified as sample completion code. Nodes with parent-child relationships with the extracted nodes can be identified as the sample context code of the sample completion code, thereby using the sample context to reveal the sample completion code. In another possible implementation, sample completion code and sample context code of sample completion code can be selected from multiple sample code fragments based on code completion dimensions and code relationships.

[0105] The solution of this disclosure improves the accuracy of sample completion code and sample context code by extracting sample completion code and sample context code from sample code documents based on code relationships.

[0106] In one optional embodiment of this disclosure, the above-described parsing of sample code documents to determine multiple sample code fragments and the code relationships between them may include the following steps:

[0107] The sample code document is parsed to obtain an abstract syntax tree, where the abstract syntax tree uses sample code fragments as nodes;

[0108] Based on the node hierarchy of the abstract syntax tree, determine multiple sample code snippets and the code relationships between them.

[0109] It's important to note that syntax parsing transforms the source code in the sample code document into a more easily processed intermediate representation, namely an Abstract Syntax Tree (AST). An AST is a tree-like data structure used to represent the syntactic structure of the source code. Each node represents a component in the source code, i.e., a sample code snippet. The hierarchical structure of the tree reflects the nesting of syntactic structures in the code and includes various elements such as expressions, statements, and functions. ASTs remove redundant information from the source code (such as parentheses and spaces), preserving the logical structure of the code, making it more suitable for further processing and analysis.

[0110] In practical applications, when parsing sample code documents, lexical analysis can be performed on the source code first, breaking it down into smaller tokens such as keywords, identifiers, operators, and constants. Then, according to the syntax rules of the programming language, these tokens can be combined into a syntactically correct structure to obtain an abstract syntax tree.

[0111] Furthermore, the entire abstract syntax tree can be traversed to visit each node. The traversal method can be either depth-first search or breadth-first search. During traversal, the type, name, and child node information of each node are recorded. The child node information includes the node type, name, parent node, and child nodes. Based on the child node information, the code relationships between multiple sample code snippets can be directly determined.

[0112] By applying the solution of this disclosure embodiment, based on the abstract syntax tree of the sample code document, multiple sample code fragments and the code relationships between them can be accurately determined, laying the foundation for the code completion capability of the subsequent training model.

[0113] In one optional embodiment of this disclosure, the process of filtering out sample completion code and sample context code of sample completion code from multiple sample code fragments based on code relationships may include the following steps:

[0114] Based on the code completion dimensions and code relationships of multiple sample code snippets, sample completion code and sample context code of the sample completion code are selected from multiple sample code snippets.

[0115] It's important to note that code completion dimension refers to the granularity of the sample code to be completed in a code completion task. Different code completion dimensions are suitable for different types and sizes of sample code snippets. If the code completion dimension is expression-level, the task is expression-level, and the sample code is a subexpression within a larger expression in the sample code document or a simple independent expression. For example, the model fills in operand or operator gaps in a binary operation, or provides appropriate function parameters. If the code completion dimension is statement-level, the task is statement-level, and the sample code is a single statement in the sample code document. The model completes variable assignments, control flow structures (if statements, for loops), etc. Filtering sample code based on the statement dimension maintains the coherence of the code logic flow and ensures syntactic correctness. If the code completion dimension is function-level, the task is function-level, and the sample code is a complete function body or function signature in the sample code document. For example, the model completes parameter lists, return types, and the internal logic of functions. If the code completion dimension is the line-of-code dimension, it means the code completion task is a random line completion task, and the sample completed code is a line of code randomly selected from the sample code document. If the code completion dimension is the range-of-code dimension, it means the code completion task is a random range completion task. Compared with the line-of-code dimension, under the range-of-code dimension, the sample completed code consists of multiple consecutive lines with closely related logical units.

[0116] In practical applications, there are multiple ways to filter out sample completion code and sample context code from multiple sample code snippets based on code completion dimensions and code relationships. The specific method chosen depends on the actual situation, and this disclosure does not impose any limitations on these methods. In one possible implementation, multiple sample code snippets can be updated according to code completion dimensions to obtain multiple updated sample code snippets, where the updated sample code snippets conform to the code completion dimensions. Then, based on code relationships, sample completion code and sample context code are filtered out from the multiple updated sample code snippets. In this case, both the sample completion code and the sample context code conform to the code completion dimensions. In another possible implementation, the sample context code does not need to conform to the code completion dimensions; only the sample completion code needs to conform. Therefore, sample completion code can be filtered out from multiple sample code snippets based on their code completion dimensions, where the sample completion code conforms to the code completion dimensions; and sample context code that has a code relationship with the sample completion code is then filtered out from the multiple sample code snippets.

[0117] When using the scheme of this disclosure embodiment to select sample completion code and sample context code of sample completion code from multiple sample code fragments, different code completion dimensions are considered, so that the sample completion code is a multi-granular code fragment, and further enables the code processing model to complete code fragments of various logical units (such as statements, expressions and functions).

[0118] In one optional embodiment of this disclosure, the process of selecting sample completion code and sample context code from multiple sample code snippets based on code completion dimensions and code relationships may include the following steps:

[0119] Based on the code completion dimensions of multiple sample code snippets, sample completed code is selected from multiple sample code snippets, where the sample completed code conforms to the code completion dimensions;

[0120] From multiple sample code snippets, filter out sample context code that has a code relationship with the sample completion code.

[0121] In practical applications, there are various ways to select sample completed code from multiple sample code snippets based on code completion dimensions. The specific method chosen depends on the actual situation, and this disclosure does not impose any limitations on this approach. One possible implementation of this disclosure is to directly select a target sample code snippet from multiple sample code snippets, and then concatenate or decompose the target sample code snippet based on the code completion dimensions to obtain sample completed code that conforms to the code completion dimensions. Another possible implementation of this disclosure is to update multiple sample code snippets based on the code completion dimensions to obtain multiple updated sample code snippets, wherein the updated sample code snippets conform to the code completion dimensions, and then select sample completed code from these updated sample code snippets.

[0122] The solution of this disclosure first filters out sample completion codes that meet the code completion dimensions, and then filters out sample context codes that have a code relationship with the sample completion codes from multiple sample code fragments. By taking into account different code completion dimensions, multi-granularity sample completion codes are obtained.

[0123] In an optional embodiment of this disclosure, before filtering out the sample completion code and the sample context code of the sample completion code from multiple sample code snippets based on the code completion dimensions and code relationships of multiple sample code snippets, the following steps may be included:

[0124] Obtain code completion dimensions from multiple sample code snippets. The code completion dimensions include at least one of the following: expression dimension, statement dimension, function dimension, line of code dimension, and code interval dimension. The code interval includes multiple consecutive lines of code.

[0125] In practical applications, there are various ways to obtain code completion dimensions for multiple sample code snippets, and the specific method chosen depends on the actual situation. In one possible implementation, this disclosure involves receiving code completion dimensions for multiple sample code snippets sent by the user through a client. In another possible implementation, at least one candidate completion dimension can be randomly selected from a pre-configured pool of candidate completion dimensions as the code completion dimensions for multiple sample code snippets. In yet another possible implementation, the code completion dimensions for multiple sample code snippets can be determined based on the code language type of the sample code document.

[0126] By applying the scheme of this disclosure embodiment, different code completion dimensions can be used to make the sample completed code into multi-granularity code fragments, and further enable the code processing model to complete code fragments of various logical units (such as statements, expressions and functions).

[0127] In one optional embodiment of this disclosure, obtaining the code completion dimensions of multiple sample code snippets may include the following steps:

[0128] Perform language type identification on the sample code document to obtain the code language type;

[0129] Based on the code language type, determine the code completion dimensions for multiple sample code snippets.

[0130] It should be noted that language type identification refers to the process of determining the programming language used in the code by analyzing the content of sample code documents. The code language type refers to the type of programming language used in the sample code document. Code language types include, but are not limited to, Python (an interpreted, object-oriented high-level programming language), Java (a statically typed, object-oriented programming language), and JavaScript (a scripting language primarily used in web browsers).

[0131] In practical applications, there are various ways to identify the language type of sample code documents and obtain the code language type. The specific method chosen depends on the actual situation, and this disclosure does not impose any limitations on this approach. One possible implementation of this disclosure involves determining the file extension of the sample code document (e.g., .py, .java, .js, etc.) and then determining the code language type based on the file extension. Another possible implementation involves using a code parsing tool to perform syntax analysis on the sample code document and determining the code language type based on the structure of the abstract syntax tree.

[0132] Furthermore, language type identification is performed on the sample code documents. After obtaining the code language type, the code completion dimensions for multiple sample code fragments can be determined based on the language characteristics and common usages of the sample code documents. For example, if the code language type is Python, the code completion dimensions can be expression dimension, statement dimension, function dimension, line of code dimension, and code range dimension.

[0133] By applying the solution of this disclosure, the programming language used in the code can be determined by performing language type recognition on the sample code document. Selecting appropriate code completion dimensions based on different programming languages ​​can improve the accuracy and efficiency of code completion.

[0134] Step 206: Convert the sample context code into a question-and-answer format to obtain the sample instruction code.

[0135] In code question answering tasks, the sample code questions input to the model are typically in the form of natural language instructions, such as "This is a sample code question [sample code question], please answer the sample code question and output the sample code answer." In code completion tasks, the sample context code input to the model is typically a concatenated prefix and suffix code, such as "prefix code [prefix code] suffix code [suffix code]." Because the model input formats differ between these two tasks, directly training the model using these two types of data can lead to a problem where the model fails to learn code completion due to a lack of understanding of the current task's processing objective, resulting in a decline in code question answering capabilities. Therefore, to address this issue, this embodiment of the disclosure can perform a question-answering format conversion on the sample context code, unifying the model input for code completion tasks with that for code question answering tasks, thereby avoiding the decline in model capabilities caused by inconsistent model input data formats.

[0136] It should be noted that question-and-answer format conversion refers to converting sample context codes into sample instruction codes in question-and-answer code format. Sample instruction codes are the sample context codes after question-and-answer format conversion. The format of the sample instruction codes is the same as that of the sample code questions; they are natural language-based instructions, such as "Please complete the middle sample completion code based on the sample context code [sample context code], and output the sample completion code," or "Please complete the middle sample completion code based on the sample prefix code [sample prefix code] and the sample suffix code [sample suffix code], and output the sample completion code."

[0137] In practical applications, there are various ways to convert sample context codes into question-and-answer formats to obtain sample instruction codes. The specific method chosen depends on the actual situation, and this disclosure does not impose any limitations on this approach. In one possible implementation, the sample context codes can be input into a format conversion model to obtain sample instruction codes. The format conversion model can be a large-scale code domain model or a deep learning model trained based on the training context codes and the training instruction code labels of the training context codes.

[0138] In another possible implementation of this disclosure, the sample context code can be converted into a question-and-answer format based on a code question-and-answer template. That is, the above-mentioned conversion of the sample context code into a question-and-answer format to obtain sample instruction code may include the following steps:

[0139] Get the code Q&A template;

[0140] Add the sample context code to the code Q&A template to obtain the sample instruction code.

[0141] It should be noted that code question-and-answer templates refer to natural language text that can convert sample context codes into the same format as sample question-and-answer codes.

[0142] In practical applications, there are various ways to obtain code Q&A templates, and the specific method should be selected according to the actual situation. This disclosure does not impose any limitations on this method. In one possible implementation, a code Q&A template sent by a user through a client can be received. In another possible implementation, code Q&A data can be obtained, and a general code Q&A template can be derived based on this data.

[0143] For example, the code Q&A template could be "Please complete the sample completion code in the middle based on the sample context code [], and output the sample completion code", or it could be "Please complete the sample completion code in the middle based on the sample prefix code [] and the sample suffix code [], and output the sample completion code". Here, "[]" is a conversion identifier; filling the sample context code into the position indicated by the conversion identifier will yield the sample instruction code.

[0144] By applying the solution of this disclosure embodiment, the sample context code is converted into a question-and-answer format based on the code question-and-answer template to obtain the sample instruction code, thereby achieving a unified format for code question-and-answer and code completion data, thus avoiding the problem of model performance degradation caused by inconsistent training data formats.

[0145] Step 208: Use sample question-and-answer code, sample completion code, and sample instruction code to train the initial processing model and obtain the code processing model.

[0146] It should be noted that the initial processing model can be a large model, a large model within a code domain with code processing capabilities, or an untrained neural network model. The initial processing model is trained using supervised fine-tuning.

[0147] The scheme of this disclosure uses a parsed abstract syntax tree to mask sample code fragments at different logical unit levels (code completion dimension), obtaining repository-level sample completion code. The initial processing model is trained using sample question-and-answer code, sample completion code, and sample instruction code, enabling the code processing model to simultaneously possess code question-and-answer and code completion capabilities. Furthermore, by converting the sample context code into a question-and-answer format, the format of code question-and-answer and code completion data is unified, thereby avoiding the problem of model performance degradation due to inconsistent training data formats and improving the accuracy of code completion and code question-and-answer in the code processing model.

[0148] In practical applications, there are various ways to train an initial processing model using sample question-and-answer code, sample completion code, and sample instruction code to obtain a code processing model. The specific method chosen depends on the actual situation, and this disclosure does not limit this approach. In one possible implementation, the sample question-and-answer code, sample completion code, and sample instruction code of the sample completion code can be integrated to obtain a sample set. This sample set can then be used to train the initial processing model to obtain the code processing model.

[0149] In another possible implementation of this disclosure, the code document model can be trained using sample completion code and sample instruction code to obtain a code processing model. The code question-answering model is trained based on sample question-answering code. That is, the above-mentioned training of the initial processing model using sample question-answering code, sample completion code, and sample instruction code to obtain the code processing model may include the following steps:

[0150] The initial processing model is trained using sample question-and-answer codes to obtain a code-based question-and-answer model;

[0151] By using sample code completion and sample instruction code, the code question answering model is trained to obtain a code processing model.

[0152] It should be noted that a code question-answering model refers to a model with code question-answering capabilities. When training the initial processing model using sample question-answering codes, the sample code questions from the sample question-answering codes can be input into the initial processing model to obtain the predicted code answers output by the initial processing model; the question-answering loss value is calculated based on the sample code answers and the predicted code answers; and the model parameters of the initial processing model are adjusted based on the question-answering loss value to obtain the code question-answering model.

[0153] The scheme of this disclosure embodiment uses a phased model training method. In the first phase, a code question answering model with code question answering capability is trained. Then, in the second phase, the code completion capability of the code question answering model is trained to obtain a code processing model, thereby enabling the code processing model to have both code question answering and completion capabilities.

[0154] In one optional embodiment of this disclosure, the above-mentioned training of the code question-answering model using sample completion code and sample instruction code to obtain a code processing model may include the following steps:

[0155] Input the sample instruction code into the code question answering model to obtain the predicted completion code;

[0156] The code question answering model is trained based on predicted code completion and sample code completion to obtain a code processing model.

[0157] It should be noted that the training method of the code processing model is supervised fine-tuning. That is, the sample instruction code carries the real processing label (sample completion code). The sample completion code is the generation target of the code question answering model and is used to guide the training process of the code question answering model.

[0158] In practical applications, when training a code-based question-answering model based on predicted and sample completed codes, a completion loss value can be calculated. The model parameters are then adjusted based on this loss value until a preset stopping condition is met, resulting in a trained code processing model. Many functions can be used to calculate the completion loss value, such as cross-entropy loss, L1 norm loss, maximum loss, mean squared error loss, and logarithmic loss. The specific function chosen depends on the actual situation, and this embodiment does not impose any limitations on this. The preset stopping condition includes, but is not limited to, the completion loss value being less than or equal to a preset threshold or the number of iterations reaching a preset number of iterations. The preset threshold and the preset number of iterations are specifically selected based on the actual situation, and this embodiment does not impose any limitations on these.

[0159] In one possible implementation of this disclosure, after calculating the completion loss value, it can be compared with a preset threshold. Specifically, if the completion loss value is greater than the preset threshold, it indicates that the difference between the predicted completed code and the sample completed code is large, and the code completion ability of the code question answering model is poor. At this time, the model parameters of the code question answering model can be adjusted, and the step of inputting the sample instruction code into the code question answering model to obtain the predicted completed code can be returned. The code question answering model is then trained until the completion loss value is less than or equal to the preset threshold, indicating that the difference between the predicted completed code and the sample completed code is small, and the preset stopping condition is met, thus obtaining a code processing model that has been trained.

[0160] In another possible implementation of this disclosure, in addition to comparing the completion loss value with the preset threshold, the number of iterations can also be considered to determine whether the current code question answering model has been trained successfully. Specifically, if the completion loss value is greater than the preset threshold, the model parameters of the code question answering model are adjusted, and the process returns to the step of inputting the sample instruction code into the code question answering model to obtain the predicted completed code. The code question answering model is then trained again until the preset number of iterations is reached, at which point the iteration stops, and the trained code processing model is obtained.

[0161] The scheme of this disclosure involves training a code question-answering model based on predicted and sample-completed code. Training continues until a preset stopping condition is met, completing the training and obtaining the code processing model. By continuously adjusting the model parameters, the model's code understanding and reasoning abilities can be enhanced, enabling it to perform code completion based on sample instruction codes.

[0162] In one optional embodiment of this disclosure, after the code processing model is trained, it can be directly applied to the processing of the target code task, or the code processing capability of the code processing model can be tested, and if the test is passed, the code processing model can be applied to the processing of the target code task.

[0163] Referring to Figure 3, Figure 3 shows a flowchart of a code processing model testing method provided in an embodiment of this disclosure, which specifically includes the following steps:

[0164] Step 302: Obtain the test code documentation from the code repository.

[0165] Step 304: Extract the test context code from the test code document, and convert the test context code into a question-and-answer format to obtain the test instruction code.

[0166] Step 306: Input the test instruction code into the code processing model to obtain the test completion code. The code processing model is trained using the code processing model training method.

[0167] Step 308: Complete the code based on the test and generate the model test results for the code processing model.

[0168] It's important to note that a code repository is a centralized place for storing and managing source code. It's typically used for version control, allowing multiple developers to collaborate on the same project and recording the history of each change. A code repository not only stores source code but can also store documentation, configuration files, test code, etc. Test code documentation refers to the documentation within the code repository that includes the source code. To ensure the accuracy of model test results, there are usually multiple test code documents. Test completion code refers to the code generated by the code processing model based on test instruction code. Model test results include test pass and test failure. If the model test result is a pass, it means that the difference between the test completion code and the actual test code (the code snippets between the test context code in the test code document) is small or even non-existent, indicating that the code processing model has good code completion capabilities and can be directly applied to the processing of the target code task.

[0169] In practical applications, the method of "extracting test context code from test code document and converting test context code into question-and-answer format to obtain test instruction code" is the same as the above-mentioned method of "extracting sample completion code and sample context code of sample completion code from sample code document; converting sample context code into question-and-answer format to obtain sample instruction code" and will not be described again in this disclosure embodiment.

[0170] Furthermore, there are various ways to generate model test results for the code processing model based on the test completion code, and the specific method chosen depends on the actual situation. This disclosure does not impose any limitations on this approach. One possible implementation of this disclosure is a static method that uses string matching to determine the model test results of the code processing model based on text matching metrics (such as EM and ES) between the real test code and the test completion code. However, this first static method cannot fully capture the dynamic nature of real-world coding environments and faces challenges such as limited context length, reliance on surface-level evaluation metrics, and potential overfitting of the training dataset. Therefore, a second possible implementation can be used to generate model test results for the code processing model based on the code functionality of the test completion code.

[0171] By applying the solution of this disclosure embodiment, based on test code completion, model test results of the code processing model are generated, thereby measuring the code completion capability of the code processing model and ensuring the accuracy of the code processing model.

[0172] In one optional embodiment of this disclosure, the model test results of generating a code processing model based on the code functionality of test-completed code are described. That is, the above-mentioned model test results of generating a code processing model based on test-completed code may include the following steps:

[0173] Based on the test code completion, update the test code documentation in the code repository to obtain the test code repository;

[0174] Perform unit tests on the test code repository and obtain the unit test results;

[0175] Based on the unit test results, generate model test results for the code processing model.

[0176] It's important to note that unit testing refers to automated tests written and run within a code repository to verify that the smallest testable unit (typically a function, method, or module) in the repository functions as expected. Unit test results include pass and fail. If a unit test passes, the code repository is determined to function as expected, and the model test result for the code processing model is "test passed." If a unit test fails, the code repository is determined to not function as expected, and the model test result for the code processing model is "test failed."

[0177] In practical applications, when updating the test code documentation in the code repository based on test completion code, the actual test code in the test code documentation can be replaced with the test completion code to obtain the updated test code documentation. By replacing the actual test code in all test code documentation with the test completion code generated by the code processing model, a test code repository consisting of the updated test code documentation can be obtained.

[0178] Furthermore, when performing unit tests on the test code repository, the repository can be installed locally, and an appropriate testing framework can be used. Common testing frameworks include Python, Java, and JavaScript. If the test code repository works as expected, it indicates that replacing the actual test code in the test code documentation with the test completion code does not affect the code's performance. In other words, the test completion code and the actual test code have the same functional purpose. Therefore, it can be determined that the model test result for the code processing model is a pass, and the code processing model possesses good code completion capabilities.

[0179] The solution in this disclosure uses repository-level unit testing to verify the code completion capability of the code processing model, improving the accuracy of model testing. For example, if the text similarity between the actual test code and the test completion code generated by the code processing model is low, and the model is tested using text matching metrics, the model test result will be "failed" regardless of whether the two codes have the same functionality. However, through functional testing, even if the text similarity between the actual test code and the test completion code is low, if their code functions are the same, the model test can still be determined to have passed, thus achieving accurate testing of the model processing capability.

[0180] In one optional embodiment of this disclosure, after generating the model test results of the code processing model based on the test completion code, the following steps may be further included:

[0181] If the model test result is a test failure, return to the steps of obtaining sample code documentation and sample question-and-answer code until the model test result is a test pass, and obtain the code processing model that has passed the test.

[0182] It should be noted that if the model test result is a test failure, it means that there is a large difference between the test completed code and the real test code, and the code completion ability of the code processing model is poor. It cannot be directly applied to the processing of the target code task. In this case, the code processing model can be trained again using the code processing model training method shown in Figure 2 until the model test result is a test pass, and a code processing model that has passed the test is obtained.

[0183] In practical applications, since the code processing model is already trained based on sample question-and-answer code, meaning it already possesses code question-and-answer capabilities, if the model fails the test, only the sample code document needs to be obtained. The sample code document is then parsed to determine multiple sample code snippets and the code relationships between them. Based on these relationships, sample completion code and its context code are selected from the multiple sample code snippets, and the context code is converted to a question-and-answer format to obtain sample instruction code. The code processing model is then trained using the completed sample code and the instruction code until the model passes the test, thus obtaining a successfully tested code processing model.

[0184] By applying the solution of this disclosure embodiment, a code processing model with high-precision code question answering and completion capabilities can be obtained by iteratively training the code processing model that failed the test.

[0185] Referring to Figure 4, Figure 4 shows a flowchart of a code task processing method provided in an embodiment of the present disclosure. The code task processing method includes four stages: construction of a multi-level syntax-based instruction sample set, model fine-tuning based on the instruction sample set, repository-level model testing, and code task processing based on the code processing model.

[0186] Construction of a multi-level grammar-based instruction sample set: Obtain sample code documentation; perform grammar parsing on the sample code documentation to obtain an abstract syntax tree (AST); determine multiple sample code fragments and their relationships based on the node hierarchy of the AST; identify the language type of the sample code documentation to obtain the code language type; determine the code completion dimensions for multiple sample code fragments based on the code language type, where the code completion dimensions include at least one of expression, statement, function, line of code, and code interval dimensions; based on the code relationships and code completion dimensions, select the completed sample code and its context code from the multiple sample code fragments; perform question-and-answer format conversion on the context code to obtain the sample instruction code.

[0187] Model fine-tuning based on instruction sample set: The code question answering model is trained using sample completion code and sample instruction code to obtain a code processing model with both code question answering and code completion capabilities. The code question answering model is trained based on sample question answering code.

[0188] Repository-level model testing: Merge branches of active code repositories to perform model testing on the main branch: Obtain test code documentation from the code repository; extract test context code from the test code documentation and perform question-and-answer format conversion on the test context code to obtain test instruction code; input the test instruction code into the code processing model to obtain test completion code; update the test code documentation in the code repository based on the test completion code to obtain the test code repository; perform unit tests on the test code repository in a virtual environment (such as a container) based on the repository documentation and repository installation documentation. If the test code repository passes the unit tests, the test completion code is determined to be correct, and the code processing model test passes.

[0189] Code task processing based on code processing model: Obtain the code data to be processed for the target code task; input the code data to be processed into the tested code processing model to obtain the code processing result.

[0190] Referring to Figure 5, Figure 5 shows a flowchart of a code task processing method provided in an embodiment of this disclosure, which specifically includes the following steps:

[0191] Step 502: Obtain the code data to be processed for the target code task.

[0192] Step 504: Input the code data to be processed into the code processing model to obtain the code processing result. The code processing model is trained using the code processing model training method.

[0193] It should be noted that the target code task can be of different types, such as code completion, code generation, code explanation, code question answering, etc. The code data to be processed can be code snippets to be completed, code questions to be answered, code snippets to be explained, etc. The code processing result corresponds to the target code task. For example, if the target code task is code completion and the code data to be processed is code snippets to be completed, then the code processing result is completed code. Or, if the target code task is a code question answering task and the code data to be processed is code questions to be answered, then the code processing result is the code answer.

[0194] In practical applications, there are various ways to obtain the code data to be processed for a target code task, and the specific method chosen depends on the actual situation. This disclosure does not impose any limitations on these methods. In one possible implementation, the code data to be processed for the target code task can be read from other data acquisition devices or a database. In another possible implementation, the code data to be processed for the target code task can be received from a user via a terminal device.

[0195] The solution applied in this embodiment of the disclosure, since the code processing model is trained on the initial processing model based on sample question-and-answer code, sample completion code, and sample instruction code, and the sample instruction code is obtained by converting the sample context code into a question-and-answer format, achieves a unified format for code question-and-answer and code completion data. This avoids the problem of model capability degradation caused by inconsistent training data formats. Therefore, the code processing model possesses both accurate code question-and-answer and code completion capabilities. Inputting the code data to be processed into the code processing model allows for precise code processing and yields highly accurate code processing results.

[0196] In one optional embodiment of this disclosure, when inputting code data to be processed into the code processing model, if the code data to be processed is a code problem to be solved, the code problem to be solved can be directly input into the code processing model to obtain the code processing result.

[0197] In another optional embodiment of this disclosure, taking the code data to be processed as including code fragments to be completed as an example; before inputting the code data to be processed into the code processing model to obtain the code processing result, the following steps may also be included:

[0198] The code snippet to be completed is converted to a question-and-answer format to obtain the instruction code to be completed;

[0199] Inputting the code data to be processed into the code processing model to obtain the code processing result may include the following steps:

[0200] Input the code to be completed into the code processing model to obtain the code completion result.

[0201] It should be noted that the code snippet to be completed can be understood as the context code of the code to be completed. The implementation method of "converting the code snippet to be completed into a question-and-answer format to obtain the instruction code to be completed" is the same as the implementation method of "converting the sample context code into a question-and-answer format to obtain the sample instruction code" described above, and will not be repeated in this embodiment.

[0202] By applying the solution of this disclosure embodiment, the code fragment to be completed is converted into a question-and-answer format so that the code fragment to be completed conforms to the input format of the code processing model, thereby avoiding the problem of poor model processing effect caused by incorrect input format and improving the quality of code completion results.

[0203] Referring to Figure 6, which shows a flowchart of an information processing method based on a code processing model according to an embodiment of this disclosure, the information processing method based on the code processing model is applied to a task platform and specifically includes the following steps:

[0204] Step 602: Receive a model request sent by the terminal device, wherein the model request includes at least one of the following: scene identifier of the target scene, scene input data of the target scene, and model specification parameters.

[0205] Step 604: Based on the model request, determine the target code processing model from multiple code processing models, wherein the multiple code processing models are trained based on the code processing model training method.

[0206] It should be noted that the target code processing model is a code processing model applicable to the target scenario. The model request includes at least one of the following: the scenario identifier of the target scenario, the scenario input data of the target scenario, and model specification parameters. Based on the model request, there are multiple ways to determine the target code processing model from multiple code processing models; the specific method is selected according to the actual situation, and this disclosure does not impose any limitations on this. In one possible implementation of this disclosure, the corresponding target code processing model can be searched from at least one code processing model included in the model library based on the model request. In another possible implementation, the target code processing model can be trained and obtained based on the model request. In yet another optional implementation, the target code processing model can be constructed based on the model request.

[0207] For example, based on the scene identifier of the target scene, at least one pre-trained code processing model can be searched from the model library. Then, based on the model specification parameters, an initial code processing model can be selected from the at least one code processing model. Finally, based on the scene input data of the target scene, the selected initial code processing model is trained to obtain a target code processing model suitable for user needs. The at least one code processing model can be trained according to the training method of the code processing model shown in Figure 2, which will not be elaborated further in this embodiment.

[0208] The solution applied in this disclosure is adapted to user needs to obtain target code processing models, realizes personalized model services, provides users with an efficient, flexible and easy-to-use model service method, and improves user experience.

[0209] In one optional embodiment of this disclosure, the model request includes a scene identifier of the target scene; the above-mentioned determination of the target code processing model from multiple code processing models based on the model request may include the following steps:

[0210] Based on the scene identifier of the target scene, the target code processing model suitable for the target scene is searched from the model library. The model library stores multiple code processing models suitable for different task scenes.

[0211] It should be noted that scene identifiers are unique or specific labels used to distinguish different code task scenarios. The model library is a database for storing and managing various pre-trained deep learning models. Multiple code processing models adapted to different code task scenarios cover different code application scenarios and needs. The model library allows users to select the appropriate model according to their needs, or directly call the model for code task processing through the application programming interface.

[0212] Multiple code processing models adapted to different code task scenarios are stored in the model library. Each model is optimized for a specific application environment, and any given code processing model is a model trained according to the training method shown in Figure 2. For example, based on the scenario identifier "code completion", a target code processing model suitable for the code completion scenario can be found from the model library.

[0213] The solution applied in this disclosure embodiment, based on scenario requirements, accurately finds the target code processing model suitable for the scenario through scenario identification, making code task processing more accurate and more scenario-appropriate, thereby improving user experience and code task processing quality.

[0214] In one optional embodiment of this disclosure, the model request includes scene input data of the target scene; the above-mentioned determination of the target code processing model from multiple code processing models based on the model request may include the following steps:

[0215] From multiple code processing models, determine the initial code processing model that is suitable for the target scenario;

[0216] Based on the scene input data of the target scene, the initial code processing model is trained to obtain the target code processing model.

[0217] It's important to note that different code processing models are suitable for different scenarios. For example, code processing model 1 is suitable for scenarios 1 and 2, while code processing model 2 is suitable for scenarios 2 and 3. The initial code processing model refers to the model among multiple code processing models that is applicable to the target scenario. If the target scenario is scenario 1, then the initial code processing model is code processing model 1, which is suitable for scenario 1. The initial code processing model may not only be applicable to the target scenario but also to other scenarios; it is a general code processing model applicable to different scenarios. While the initial code processing model can be used for code task processing, the results may not be ideal. In such cases, the initial code processing model can be optimized based on the scenario input data of the target scenario. For example, optimizing the initial code processing model based on the scenario input data of the code generation scenario can yield a target code processing model suitable for the code generation scenario. The scenario input data of the target scenario can be understood as sample data within the target scenario. Sample data can include sample completion code, sample instruction code, and sample question-and-answer code. The initial code processing model is trained based on the scene input data of the target scene. The method to obtain the target code processing model can be referred to the training method of the code processing model shown in Figure 2. The embodiments of this disclosure will not be described in detail.

[0218] The solution of this disclosure embodiment, based on scenario requirements, further trains a general initial code processing model through scenario input data to obtain a target code processing model adapted to the scenario, making the target code processing model more in line with the scenario, thereby improving user experience and code task processing quality.

[0219] In one optional embodiment of this disclosure, the model request includes model specification parameters; the determination of the target code processing model from multiple code processing models based on the model request may include the following steps:

[0220] Based on the model specification parameters, the corresponding target code processing model is searched from the model library, which stores multiple code processing models with different model specification parameters.

[0221] It's important to note that model specifications refer to the various parameters that define the model's structure and behavior. These parameters can be broadly categorized into two types: model parameters (learnable parameters) and hyperparameters. Model parameters are those automatically adjusted during model training via backpropagation, including but not limited to weight matrices and biases. For example, in a simple fully connected layer, the weight matrix is ​​a two-dimensional tensor connecting neurons in the input and output layers; the biases are one-dimensional vectors providing additional offset values ​​for each output neuron. Hyperparameters are parameters set before model training begins, controlling the model's learning process and architecture. Hyperparameters include, but are not limited to, the learning rate and the number of neurons per layer, chosen based on specific requirements.

[0222] By applying the solution of this disclosure embodiment, based on the model specification parameters, the corresponding target code processing model can be accurately found, ensuring the efficient and stable operation of the target code processing model and improving the user experience.

[0223] In one optional embodiment of this disclosure, after determining the target code processing model from multiple code processing models based on the model request, the following steps may be further included:

[0224] Deploy the target code processing model, and based on the target code processing model, build a code task processing interface so that the terminal device can schedule the target code processing model to execute target code tasks.

[0225] It should be noted that the code task processing interface is an interactive programming interface for terminal devices to schedule target code processing models to perform target code task processing, and it is usually provided in the form of an application programming interface (API). Through the code task processing interface, users can input task data for the target code task, such as the code data to be processed, in order to perform code processing or code development.

[0226] In practical applications, there are various ways to deploy the target code processing model, and the specific method chosen depends on the actual situation. This disclosure does not impose any limitations on this approach. One possible implementation of this disclosure is to deploy the target code processing model on cloud-side devices using infrastructure provided by a cloud service provider. Another possible implementation is to deploy the target code processing model on edge devices using a lightweight framework. For example, the target code processing model can be deployed on a distributed system, and a code task processing interface can be built based on the target code processing model and provided to terminal devices, enabling the terminal devices to schedule the target code processing model to execute target code tasks.

[0227] By applying the solution provided in the embodiments of this disclosure, deploying a target code processing model, and constructing a code task processing interface based on the target code processing model, terminal devices can efficiently call the target code processing model, thereby improving the processing quality and response speed of target code tasks.

[0228] Referring to Figure 7, Figure 7 shows a schematic diagram of the structure of a task platform provided in an embodiment of the present disclosure. The task platform 700 includes a request interface 702 and a response unit 704.

[0229] Request interface 702 is used to receive a model request sent by a terminal device, wherein the model request includes at least one of the following: scene identifier of the target scene, scene input data of the target scene, and model specification parameters.

[0230] The response unit 704 is used to determine the target code processing model from multiple code processing models based on the model request, wherein the multiple code processing models are trained based on the code processing model training method.

[0231] In one optional embodiment of this disclosure, the task platform further includes a code task processing interface, which is constructed based on the target code processing model;

[0232] The code task processing interface is used to allow terminal devices to schedule and execute target code tasks.

[0233] By applying the solution of this disclosure embodiment, the task platform adapts to user needs to obtain target code processing models, realizes personalized model services, provides users with an efficient, flexible and easy-to-use model service platform, and improves user experience.

[0234] The above is an illustrative scheme of a task platform according to this embodiment. It should be noted that the technical solution of this task platform and the technical solution of the information processing method based on the code processing model described above belong to the same concept. For details not described in detail in the technical solution of the task platform, please refer to the description of the technical solution of the information processing method based on the code processing model described above.

[0235] Corresponding to the above-described code processing model training method embodiments, this disclosure also provides a code processing model training device embodiment. Figure 8 shows a schematic diagram of the structure of a code processing model training device provided in one embodiment of this disclosure. As shown in Figure 8, the device includes:

[0236] The first acquisition module 802 is configured to acquire sample code documents and sample question-and-answer codes;

[0237] The first extraction module 804 is configured to extract sample completion code and sample context code of sample completion code from the sample code document;

[0238] The first conversion module 806 is configured to convert the sample context code into a question-and-answer format to obtain the sample instruction code;

[0239] Training module 808 is configured to train the initial processing model using sample question-and-answer code, sample completion code, and sample instruction code to obtain a code processing model.

[0240] Optionally, the first extraction module 804 is further configured to parse the sample code document, determine multiple sample code fragments and the code relationships between the multiple sample code fragments; and, based on the code relationships, filter out the sample completion code and the sample context code of the sample completion code from the multiple sample code fragments.

[0241] Optionally, the first extraction module 804 is further configured to filter out sample completion code and sample context code of sample completion code from multiple sample code snippets based on the code completion dimensions and code relationships of multiple sample code snippets.

[0242] Optionally, the first extraction module 804 is further configured to: filter sample completion code from multiple sample code snippets based on the code completion dimension of multiple sample code snippets, wherein the sample completion code conforms to the code completion dimension; and filter sample context code that has a code relationship with the sample completion code from multiple sample code snippets.

[0243] Optionally, the device further includes a fourth acquisition module configured to acquire code completion dimensions of multiple sample code snippets, wherein the code completion dimensions include at least one of expression dimension, statement dimension, function dimension, line of code dimension, and code interval dimension, and the code interval includes multiple consecutive lines of code.

[0244] Optionally, the fourth acquisition module is further configured to perform language type recognition on the sample code document to obtain the code language type; and determine the code completion dimensions of multiple sample code fragments based on the code language type.

[0245] Optionally, the first extraction module 804 is further configured to perform syntax parsing on the sample code document to obtain an abstract syntax tree of the sample code document, wherein the abstract syntax tree uses sample code fragments as nodes; and determine multiple sample code fragments and the code relationships between multiple sample code fragments based on the node hierarchy of the abstract syntax tree.

[0246] Optionally, the training module 808 is further configured to train the initial processing model using sample question-and-answer codes to obtain a code question-and-answer model; and to train the code question-and-answer model using sample completion codes and sample instruction codes to obtain a code processing model.

[0247] The scheme of this disclosure, by training the initial processing model using sample question-and-answer code, sample completion code, and sample instruction code, enables the code processing model to simultaneously possess code question-and-answer and code completion capabilities. Furthermore, by converting the sample context code into a question-and-answer format, the format of the code question-and-answer and code completion data is unified, thereby avoiding the problem of model performance degradation due to inconsistent training data formats and improving the processing capability of the code processing model.

[0248] The above is an illustrative scheme of a code processing model training device according to this embodiment. It should be noted that the technical solution of this code processing model training device and the technical solution of the code processing model training method described above belong to the same concept. For details not described in detail in the technical solution of the code processing model training device, please refer to the description of the technical solution of the code processing model training method described above.

[0249] Corresponding to the above-described code processing model testing method embodiments, this disclosure also provides a code processing model testing device embodiment. Figure 9 shows a schematic diagram of the structure of a code processing model testing device provided in one embodiment of this disclosure. As shown in Figure 9, the device includes:

[0250] The second acquisition module 902 is configured to acquire test code documents from the code repository;

[0251] The second extraction module 904 is configured to extract test context code from the test code document and perform question-and-answer format conversion on the test context code to obtain test instruction code;

[0252] The first input module 906 is configured to input test instruction code into the code processing model to obtain test completion code, wherein the code processing model is trained using a code processing model training method;

[0253] Module 908 is configured to complete the code based on the test and generate model test results for the code processing model.

[0254] Optionally, the generation module 908 is further configured to complete the code based on the tests, update the test code documentation in the code repository to obtain the test code repository; perform unit tests on the test code repository to obtain the unit test results; and generate model test results for the code processing model based on the unit test results.

[0255] By applying the solution of this disclosure embodiment, based on test code completion, model test results of the code processing model are generated, thereby measuring the code completion capability of the code processing model and ensuring the accuracy of the code processing model.

[0256] The above is a schematic scheme of a code processing model testing device according to this embodiment. It should be noted that the technical solution of this code processing model testing device and the technical solution of the code processing model testing method described above belong to the same concept. For details not described in detail in the technical solution of the code processing model testing device, please refer to the description of the technical solution of the code processing model testing method described above.

[0257] Corresponding to the above-described code task processing method embodiments, this disclosure also provides a code task processing apparatus embodiment. Figure 10 shows a schematic diagram of the structure of a code task processing apparatus provided in one embodiment of this disclosure. As shown in Figure 10, the apparatus includes:

[0258] The third acquisition module 1002 is configured to acquire the code data to be processed for the target code task;

[0259] The second input module 1004 is configured to input the code data to be processed into the code processing model to obtain the code processing result, wherein the code processing model is trained using the code processing model training method.

[0260] Optionally, the code data to be processed includes code fragments to be completed; the device further includes: a second conversion module configured to perform question-and-answer format conversion on the code fragments to be completed to obtain instruction codes to be completed; and a second input module 1004 further configured to input the instruction codes to be completed into the code processing model to obtain code completion results.

[0261] The solution applied in this embodiment of the disclosure, since the code processing model is trained on the initial processing model based on sample question-and-answer code, sample completion code, and sample instruction code, and the sample instruction code is obtained by converting the sample context code into a question-and-answer format, achieves a unified format for code question-and-answer and code completion data. This avoids the problem of model capability degradation caused by inconsistent training data formats. Therefore, the code processing model possesses both accurate code question-and-answer and code completion capabilities. Inputting the code data to be processed into the code processing model allows for precise code processing and yields highly accurate code processing results.

[0262] The above is an illustrative scheme of a code task processing device according to this embodiment. It should be noted that the technical solution of this code task processing device and the technical solution of the above-described code task processing method belong to the same concept. For details not described in detail in the technical solution of the code task processing device, please refer to the description of the technical solution of the above-described code task processing method.

[0263] Corresponding to the above-described information processing method embodiment based on the code processing model, this disclosure also provides an information processing device embodiment based on the code processing model. Figure 11 shows a schematic diagram of the structure of an information processing device based on the code processing model provided in one embodiment of this disclosure. As shown in Figure 11, the device is applied to a task platform and includes:

[0264] The receiving module 1102 is configured to receive a model request sent by a terminal device, wherein the model request includes at least one of the following: a scene identifier of the target scene, scene input data of the target scene, and model specification parameters.

[0265] The determination module 1104 is configured to determine the target code processing model from multiple code processing models based on a model request, wherein the multiple code processing models are trained based on a code processing model training method.

[0266] Optionally, the model request includes a scene identifier of the target scene; the determination module 1104 is further configured to search for a target code processing model suitable for the target scene from the model library based on the scene identifier of the target scene, wherein the model library stores multiple code processing models suitable for different task scenarios.

[0267] Optionally, the model request includes scene input data of the target scene; the determination module 1104 is further configured to determine an initial code processing model suitable for the target scene from multiple code processing models; and to train the initial code processing model based on the scene input data of the target scene to obtain the target code processing model.

[0268] Optionally, the model request includes model specification parameters; the determination module 1104 is further configured to search for the corresponding target code processing model from the model library based on the model specification parameters, wherein the model library stores multiple code processing models with different model specification parameters.

[0269] Optionally, the device further includes: a deployment module configured to deploy a target code processing model and, based on the target code processing model, construct a code task processing interface to enable a terminal device to schedule the target code processing model to execute target code tasks.

[0270] The solution applied in this disclosure is adapted to user needs to obtain target code processing models, realizes personalized model services, provides users with an efficient, flexible and easy-to-use model service method, and improves user experience.

[0271] The above is an illustrative scheme of an information processing device based on a code processing model according to this embodiment. It should be noted that the technical solution of this information processing device based on a code processing model and the technical solution of the information processing method based on a code processing model described above belong to the same concept. For details not described in detail in the technical solution of the information processing device based on a code processing model, please refer to the description of the technical solution of the information processing method based on a code processing model described above.

[0272] Figure 12 shows a structural block diagram of a computing device 1200 provided in one embodiment of the present disclosure.

[0273] The computing device 1200 includes:

[0274] Memory 1210 and processor 1220;

[0275] The memory 1210 is used to store computer programs / instructions, and the processor 1220 is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor 1220, they implement the steps of the above-mentioned code processing model training method, code processing model testing method, code task processing method, or information processing method based on code processing model.

[0276] In one or more embodiments of this disclosure, the computing device can be understood as an integrated smart terminal, including but not limited to a server, desktop computer, personal computer (PC), all-in-one model machine, mobile phone, tablet computer or other portable smart terminal, etc., and the computing device may have the model described in the above embodiments of this application pre-installed.

[0277] Specifically, this computing device can pre-install various types of models, including but not limited to models in natural language processing, visual processing, speech processing, code processing, and multimodal task processing, thus providing diverse model selection. In different product forms, this computing device can support one or more model usage methods, including but not limited to model training, model invocation, model fine-tuning, model deployment, model inference, and application. In some product forms, this computing device also supports model management, including but not limited to multi-type model management (supporting the management of discriminative, generative, and other types of models), model version control (supporting the control of different model versions), and model evaluation (evaluating model performance and effectiveness based on model evaluation tools). In other product forms, this computing device can also create applications based on models, providing application programming interface (API) invocation capabilities. Models can be invoked into created applications through the API interface, and application management tools are provided for application management and monitoring.

[0278] Furthermore, the computing device may also include data management (supporting the creation and management of model tuning datasets), a training center (providing abundant training resources to help users learn and master artificial intelligence technology), and basic control capabilities (providing enterprise-level basic control capabilities to ensure the security and efficient operation of the system). Through the above functions, it provides a comprehensive and integrated device for artificial intelligence development, training, deployment, and application.

[0279] Figure 13 shows a structural block diagram of an electronic device 1300 provided according to an embodiment of the present disclosure.

[0280] The memory 1310 and the processor 1320 are connected via a bus 1330;

[0281] The memory 1310 is used to store computer programs / instructions, and the processor 1320 is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor 1320, they implement the steps of the above-mentioned code processing model training method, code processing model testing method, code task processing method, or information processing method based on the code processing model.

[0282] Specifically, the components of the electronic device 1300 include, but are not limited to, a memory 1310 and a processor 1320. The processor 1320 and the memory 1310 can be connected via a bus 1330.

[0283] Electronic device 1300 may also include access device 1340, which enables electronic device 1300 to communicate with database 1350 storing data via one or more networks 1360. Examples of such networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. Access device 1340 may include one or more of any type of wired or wireless network interface (e.g., Network Interface Card (NIC)), such as IEEE 802.11 Wireless Local Area Networks (WLAN) wireless interface, Wi-MAX (World Interoperability for Microwave Access) interface, Ethernet interface, Universal Serial Bus (USB) interface, cellular network interface, Bluetooth interface, Near Field Communication (NFC) interface, and so on.

[0284] In one embodiment of this disclosure, the aforementioned components of the electronic device 1300, as well as other components not shown in FIG. 13, may also be connected to each other, for example, via a bus. It should be understood that the electronic device structural block diagram shown in FIG. 13 is merely for illustrative purposes and is not intended to limit the scope of this disclosure. Those skilled in the art can add or replace other components as needed.

[0285] Electronic device 1300 can be any type of stationary or mobile electronic device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable electronic devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary electronic devices such as desktop computers or PCs. Electronic device 1300 can also be a mobile or stationary electronic device.

[0286] The above is an illustrative scheme of an electronic device according to this embodiment. It should be noted that the technical solution of this electronic device belongs to the same concept as the technical solutions of the code processing model training method, code processing model testing method, code task processing method, and information processing method based on code processing model described above. For details not described in detail in the technical solution of the electronic device, please refer to the description of the technical solutions of the code processing model training method, code processing model testing method, code task processing method, or information processing method based on code processing model.

[0287] An embodiment of this disclosure also provides a computer-readable storage medium storing a computer program / instructions that, when executed by a processor, implement the steps of the above-described code processing model training method, code processing model testing method, code task processing method, or information processing method based on a code processing model.

[0288] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solutions of the code processing model training method, code processing model testing method, code task processing method, and information processing method based on the code processing model described above. Details not described in detail in the technical solution of the storage medium can be found in the descriptions of the technical solutions of the code processing model training method, code processing model testing method, code task processing method, or information processing method based on the code processing model.

[0289] An embodiment of this disclosure also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the above-described code processing model training method, code processing model testing method, code task processing method, or information processing method based on a code processing model.

[0290] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product belongs to the same concept as the technical solutions of the code processing model training method, code processing model testing method, code task processing method, and information processing method based on the code processing model described above. Details not described in detail in the technical solution of the computer program product can be found in the descriptions of the technical solutions of the code processing model training method, code processing model testing method, code task processing method, or information processing method based on the code processing model.

[0291] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0292] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0293] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this disclosure are not limited to the described order of actions, because according to the embodiments of this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this disclosure.

[0294] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0295] The preferred embodiments disclosed above are merely illustrative of this disclosure. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments of this disclosure. These embodiments are selected and specifically described in this disclosure to better explain the principles and practical applications of the embodiments of this disclosure, thereby enabling those skilled in the art to better understand and utilize this disclosure. This disclosure is limited only by the claims and their full scope and equivalents.

Claims

1. A method for training a code processing model, comprising: Obtain sample code documentation and sample Q&A code; Extract the sample completion code and the sample context code of the sample completion code from the sample code document; The sample context code is converted into a question-and-answer format to obtain the sample instruction code; The initial processing model is trained using the sample question-and-answer code, the sample completion code, and the sample instruction code to obtain a code processing model.

2. The method according to claim 1, wherein extracting the sample completion code and the sample context code of the sample completion code from the sample code document comprises: Parse the sample code document to determine multiple sample code fragments and the code relationships between the multiple sample code fragments; Based on the code relationships, sample completion code and sample context code of the sample completion code are selected from the plurality of sample code fragments.

3. The method according to claim 2, wherein filtering out sample completion code and sample context code of the sample completion code from the plurality of sample code fragments based on the code relationship comprises: Based on the code completion dimensions and code relationships of the multiple sample code snippets, sample completion code and sample context code of the sample completion code are selected from the multiple sample code snippets.

4. The method according to claim 3, wherein filtering out sample completion code and sample context code of the sample completion code from the plurality of sample code fragments based on the code completion dimension and the code relationship comprises: Based on the code completion dimensions of the plurality of sample code snippets, sample completed code is selected from the plurality of sample code snippets, wherein the sample completed code conforms to the code completion dimensions; From the multiple sample code snippets, sample context code that has a code relationship with the sample completion code is selected.

5. The method according to claim 3, before filtering out the sample completion code and the sample context code of the sample completion code from the plurality of sample code fragments based on the code completion dimension and the code relationship, further comprising: Obtain the code completion dimensions of the multiple sample code snippets, wherein the code completion dimensions include at least one of expression dimension, statement dimension, function dimension, line of code dimension, and code interval dimension, and the code interval includes multiple consecutive lines of code.

6. The method according to claim 5, wherein obtaining the code completion dimension of the plurality of sample code fragments includes: The sample code document is subjected to language type identification to obtain the code language type; Based on the code language type, determine the code completion dimensions of the multiple sample code snippets.

7. The method according to any one of claims 2 to 6, wherein parsing the sample code document and determining the plurality of sample code fragments and the code relationships between the plurality of sample code fragments comprises: The sample code document is parsed to obtain an abstract syntax tree of the sample code document, wherein the abstract syntax tree uses the sample code fragment as a node; Based on the node hierarchy of the abstract syntax tree, determine multiple sample code snippets and the code relationships between them.

8. The method according to any one of claims 1 to 7, wherein training the initial processing model using the sample question-answering code, the sample completion code, and the sample instruction code to obtain a code processing model comprises: The initial processing model is trained using the sample question-and-answer code to obtain a code question-and-answer model; The code question-answering model is trained using the sample completion code and the sample instruction code to obtain a code processing model.

9. A code processing model testing method, comprising: Get the test code documentation from the code repository; Extract the test context code from the test code document, and convert the test context code into a question-and-answer format to obtain the test instruction code; The test instruction code is input into the code processing model to obtain test completion code, wherein the code processing model is trained using the training method described in any one of claims 1 to 8; Based on the test code completion, generate the model test results for the code processing model.

10. The method according to claim 9, wherein generating the model test result of the code processing model based on the test-completed code includes: Based on the test completion code, update the test code documentation in the code repository to obtain the test code repository; Perform unit tests on the test code repository and obtain the unit test results; Based on the unit test results, generate model test results for the code processing model.

11. A code task processing method, comprising: Obtain the code data to be processed for the target code task; The code data to be processed is input into the code processing model to obtain the code processing result, wherein the code processing model is trained using the training method described in any one of claims 1 to 8.

12. The method according to claim 11, wherein the code data to be processed includes code fragments to be completed; Before inputting the code data to be processed into the code processing model to obtain the code processing result, the process further includes: The code fragment to be completed is converted into a question-and-answer format to obtain the instruction code to be completed; The step of inputting the code data to be processed into the code processing model to obtain the code processing result includes: The code to be completed is input into the code processing model to obtain the code completion result.

13. An information processing method based on a code processing model, applied to a task platform, comprising: The device receives a model request sent by a terminal device, wherein the model request includes at least one of the following: a scene identifier of the target scene, scene input data of the target scene, and model specification parameters. Based on the model request, a target code processing model is determined from a plurality of code processing models, wherein the plurality of code processing models are trained based on the training method described in any one of claims 1 to 8.

14. The method according to claim 13, wherein the model request includes a scene identifier of the target scene; The step of determining the target code processing model from multiple code processing models based on the model request includes: Based on the scene identifier of the target scene, a target code processing model suitable for the target scene is searched from the model library, wherein the model library stores multiple code processing models suitable for different task scenes.

15. The method of claim 13, wherein the model request includes scene input data of the target scene; The step of determining the target code processing model from multiple code processing models based on the model request includes: From multiple code processing models, determine an initial code processing model that is suitable for the target scenario; Based on the scene input data of the target scene, the initial code processing model is trained to obtain the target code processing model.

16. The method of claim 13, wherein the model request includes model specification parameters; The step of determining the target code processing model from multiple code processing models based on the model request includes: Based on the model specification parameters, the corresponding target code processing model is searched from the model library, wherein the model library stores multiple code processing models with different model specification parameters.

17. The method according to any one of claims 13 to 16, further comprising, after determining the target code processing model from a plurality of code processing models based on the model request: Deploy the target code processing model, and based on the target code processing model, construct a code task processing interface so that the terminal device can schedule the target code processing model to execute target code tasks.

18. A task platform, comprising a request interface and a response unit; The request interface is configured to receive a model request sent by the terminal device, wherein The model request includes at least one of the following: the scene identifier of the target scene, the scene input data of the target scene, and the model specification parameters. The response unit is configured to determine a target code processing model from a plurality of code processing models based on the model request, wherein the plurality of code processing models are trained based on the training method described in any one of claims 1 to 8.

19. The task platform according to claim 18, further comprising a code task processing interface, wherein the code task processing interface is constructed based on the target code processing model; The code task processing interface is used for the terminal device to schedule and execute target code tasks.

20. A computing device, comprising: Memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 17.

21. An electronic device, comprising: A memory and a processor, the memory and the processor being connected via a bus; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 17.

22. A computer readable storage medium storing computer programs / instructions, which, when executed by a processor, implement the steps of the method of any one of claims 1 to 17.

23. A computer program product comprising computer programs / instructions, which, when executed by a processor, implement the steps of the method of any one of claims 1 to 17.

Images