Task processing method, code completion method, code processing model training method, code processing model-based information processing method and model training platform
Hierarchical sampling in code processing models addresses long prediction times and low accuracy by training on code blocks, improving efficiency and accuracy in code processing tasks.
Patent Information
- Application Number
- KR1020267035150
- Authority / Receiving Office
- KR · KR
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-03-21
- Filing Date
- 2025-01-09
- Publication Date
- 2026-11-09
AI Technical Summary
Large-scale models in code processing predict very long content at once, leading to excessively long prediction times and low accuracy.
Implement hierarchical sampling to train code processing models based on sample code block data, enabling prediction in units of code blocks and adapting to various scenarios.
Shortens prediction time and improves accuracy by precisely predicting various types of code blocks, enhancing task processing efficiency and user experience.
Smart Images

Figure PCT00003_ABST
Abstract
Description
Technology Field
[0001] The embodiments of the present disclosure relate to the field of computer technology, and in particular to a task processing method, a code completion method, a code processing model learning method, a code processing model-based information processing method, and a model learning platform.
[0002] The present disclosure claims priority to a Chinese patent application filed with the National Intellectual Property Administration of China on March 21, 2024, with application number 202410331805.8 and the title of the application being "Task processing method, code completion method, code processing model learning method, code processing model-based information processing method and model learning platform," the entire contents of which are incorporated by reference into the present disclosure. Background Technology
[0003] With the advancement of computer technology, large-scale models are beginning to gain prominence and are being widely applied in natural language processing fields such as dialogue, translation, and code processing due to their outstanding capabilities in terms of language understanding, generation, interaction, and reasoning. Taking code processing as an example, large-scale models are increasingly becoming a focus of research because they can provide developers with intelligent features such as code completion and code comments.
[0004] Currently, code processing results are typically obtained by directly using large-scale models to process the code text of a target task. However, in the above method, since the large-scale model must predict very long content at once, the prediction time is excessively long and the accuracy is low. Therefore, an efficient and precise task processing method is urgently required. The problem to be solved
[0005] In light of this, embodiments of the present disclosure provide a task processing method. One or more embodiments of the present disclosure relate to a code completion method, a code processing model training method, a code processing model-based information processing method, a task processing device, a code completion device, a code processing model training device, a code processing model-based information processing device, a model training platform, a computing device, a computer-readable storage medium, and a computer program product in order to solve the technical defect existing in the prior art, which is that "since a large-scale model must predict very long content at once, the prediction time is excessively long and the accuracy is low." means of solving the problem
[0006] According to the first aspect of an embodiment of the present disclosure,
[0007] Step of obtaining the target code data for processing of the target task;
[0008] A task processing method is provided that includes the step of inputting code data to be processed into a code processing model to obtain a code processing result of a target task—the code processing model is obtained by learning based on a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results, and the sample processing results and sample code block data are obtained based on hierarchical sampling of the sample code data.
[0009] According to a second aspect of an embodiment of the present disclosure,
[0010] A step of obtaining the target code data for a code completion task;
[0011] A code completion method is provided that includes the step of inputting code data to be completed into a code completion model to obtain a code completion result of a code completion task—the code completion model is obtained by training based on a plurality of sample completion results and sample code block data corresponding to each of the plurality of sample completion results, and the sample completion results and sample code block data are obtained based on hierarchical sampling of the sample code data.
[0012] According to a third aspect of an embodiment of the present disclosure,
[0013] A step of obtaining a sample set containing multiple sample code data;
[0014] A step of performing hierarchical sampling on a plurality of sample code data to obtain a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results;
[0015] A step of inputting multiple sample code block data into a code processing model to obtain a prediction processing result corresponding to each of the multiple sample code block data;
[0016] A method for training a code processing model is provided, comprising the step of training a code processing model based on sample processing results and prediction processing results to obtain a trained code processing model.
[0017] According to the fourth aspect of an embodiment of the present disclosure,
[0018] A step of receiving a task creation request containing request information;
[0019] Step of obtaining a code processing model based on request information - the code processing model is obtained by training based on multiple sample processing results and sample code block data corresponding to each of the multiple sample processing results, and the sample processing results and sample code block data are obtained based on hierarchical sampling of the sample code data -;
[0020] A method for processing information based on a code processing model is provided, comprising the step of generating task information based on a code processing model—the task information is used to execute a target task.
[0021] According to the fifth aspect of an embodiment of the present disclosure,
[0022] A first acquisition module configured to acquire processing target code data of a target task;
[0023] A task processing device is provided comprising: a first input module configured to input code data to be processed into a code processing model to obtain a code processing result of a target task; wherein the code processing model is obtained by learning based on a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results, and the sample processing results and sample code block data are obtained based on hierarchical sampling of the sample code data.
[0024] According to the sixth aspect of an embodiment of the present disclosure,
[0025] A second acquisition module configured to acquire target code data for a code completion task;
[0026] A code completion device is provided that includes a second input module configured to input code data to be completed into a code completion model to obtain a code completion result of a code completion task; wherein the code completion model is obtained by learning based on a plurality of sample completion results and sample code block data corresponding to each of the plurality of sample completion results, and the sample completion results and sample code block data are obtained based on hierarchical sampling of the sample code data.
[0027] According to the seventh aspect of an embodiment of the present disclosure,
[0028] A third acquisition module configured to acquire a sample set containing multiple sample code data;
[0029] A sampling module configured to perform hierarchical sampling on multiple sample code data to obtain multiple sample processing results and sample code block data corresponding to each of the multiple sample processing results;
[0030] A third input module configured to input multiple sample code block data into a code processing model to obtain a prediction processing result corresponding to each of the multiple sample code block data;
[0031] A code processing model training device is provided that includes a training module configured to train a code processing model based on sample processing results and prediction processing results to obtain a trained code processing model.
[0032] According to the eighth aspect of an embodiment of the present disclosure,
[0033] A first receiving module configured to receive a task creation request including request information;
[0034] A fourth acquisition module configured to acquire a code processing model based on request information - the code processing model is acquired by learning based on multiple sample processing results and sample code block data corresponding to each of the multiple sample processing results, and the sample processing results and sample code block data are acquired based on hierarchical sampling of the sample code data -;
[0035] Based on a code processing model, an information processing device based on a code processing model is provided, comprising a generation module configured to generate task information used to execute a target task.
[0036] According to the ninth aspect of an embodiment of the present disclosure, it includes a request interface unit and a model unit, and
[0037] The request interface unit is used to receive a model training request containing request information;
[0038] The model unit is used to acquire a code processing model and to train the code processing model based on request information to acquire a completed code processing model. The request information includes multiple sample processing results and sample code block data corresponding to each of the multiple sample processing results. The sample processing results and sample code block data provide a model training platform obtained based on hierarchical sampling of the sample code data.
[0039] According to the tenth aspect of an embodiment of the present disclosure,
[0040] Includes memory and processor;
[0041] The above memory is used to store a computer program / instruction, and the above processor is used to execute the computer program / instruction, and a computing device is provided that implements the steps of a method according to the first aspect or the second aspect or the third aspect or the fourth aspect when the computer program / instruction is executed by the processor.
[0042] According to the first aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided that stores a computer program / instruction and implements a step of a method according to the first aspect, the second aspect, the third aspect, or the fourth aspect when the computer program / instruction is executed by a processor.
[0043] According to the 12th aspect of an embodiment of the present disclosure, a computer program product is provided that includes a computer program / instruction and implements a step of a method according to the first aspect or the second aspect or the third aspect or the fourth aspect when the computer program / instruction is executed by a processor. Effects of the invention
[0044] A task processing method provided in one embodiment of the present disclosure comprises: a step of obtaining code data to be processed for a target task; and a step of inputting the code data to be processed into a code processing model to obtain a code processing result for the target task, wherein the code processing model is obtained by learning based on a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results, and the sample processing results and sample code block data are obtained based on hierarchical sampling of the sample code data. Since the code processing model is obtained by learning based on sample code block data, prediction is implemented in units of code blocks, thereby shortening the prediction content. Furthermore, through hierarchical sampling, the code processing model can be enabled to adapt to various code processing scenarios, thereby precisely predicting various types of code blocks and improving task processing efficiency and accuracy. Brief explanation of the drawing
[0045] FIG. 1 is an architectural diagram of a task processing system provided in one embodiment of the present disclosure. FIG. 2 is an architectural diagram of another task processing system provided in one embodiment of the present disclosure. FIG. 3 is a flowchart of a task processing method provided in one embodiment of the present disclosure. FIG. 4 is a flowchart of a code completion method provided in one embodiment of the present disclosure. FIG. 5 is a flowchart of a code processing model learning method provided in one embodiment of the present disclosure. FIG. 6 is a flowchart of the processing process of a code processing model learning method provided in one embodiment of the present disclosure. FIG. 7 is a flowchart of a code processing model-based information processing method provided in one embodiment of the present disclosure. FIG. 8 is a schematic diagram of the structure of a model learning platform provided in one embodiment of the present disclosure. FIG. 9 is a schematic diagram of the structure of a task processing device provided in one embodiment of the present disclosure. FIG. 10 is a schematic diagram of the structure of a code completion device provided in one embodiment of the present disclosure. FIG. 11 is a schematic diagram of the structure of a code processing model learning device provided in one embodiment of the present disclosure. FIG. 12 is a schematic diagram of the structure of an information processing device based on a code processing model provided in one embodiment of the present disclosure. FIG. 13 is a structural block diagram of a computing device provided in one embodiment of the present disclosure. Specific details for implementing the invention
[0046] In the following description, many specific details have been described to ensure a full understanding of the present disclosure. However, the present disclosure may be practiced in many ways different from those described herein, and those skilled in the art may similarly extend it without departing from the spirit of the present disclosure; therefore, the present disclosure is not limited by the specific practices disclosed below.
[0047] The terms used in one or more embodiments of the present disclosure are used merely for the purpose of describing specific embodiments and are not intended to limit one or more embodiments of the present disclosure. The singular forms of "one," "above," and "corresponding" used in one or more embodiments of the present disclosure and in the appended claims are intended to include the plural forms unless the context clearly indicates otherwise. Furthermore, it should be understood that the term "and / or" used in one or more embodiments of the present disclosure means and includes any or all possible combinations of one or more associated listed items.
[0048] In one or more embodiments of the present disclosure, various information may be described using the terms first, second, etc., but it should be understood that such information is not limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of one or more embodiments of the present disclosure, the first may be named the second, and similarly, the second may be named the first. Depending on the context, the word "if" as used herein may be interpreted as "when...", "in the case of...", or "in response to a decision."
[0049] In addition, it is noted that user information (including, but not limited to, user device information, user personal information, etc.) and data (including, but not limited to, data for analysis, storage data, display data, etc.) related to one or more embodiments of the present disclosure are all information and data that have been approved by the user or have received sufficient approval from each party, and that the collection, use, and processing of the related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and that a corresponding operating interface is provided for the user to select approval or rejection.
[0050] In one or more embodiments of the present disclosure, a large-scale model refers to a deep learning model having large-scale model parameters, typically including hundreds of millions, tens of billions, hundreds of billions, tens of millions, or even tens of trillions or more of model parameters. A large-scale model is also called a foundation model, and pre-training of a large-scale model is performed using a large-scale unlabeled corpus to generate a pre-trained model having hundreds of millions or more of parameters, and such a model can adapt to a wide range of downstream tasks and the model has relatively good generalization ability, such as a large language model (LLM) and a multi-modal pre-training model.
[0051] Large-scale models can be applied to different tasks by fine-tuning the pre-trained model with only a small amount of samples during actual application. These models can be widely applied in fields such as Natural Language Processing (NLP) and computer vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and image generation, as well as natural language processing tasks such as text-based sentiment classification, text summarization, and machine translation. Major application scenarios for large-scale models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.
[0052] First, noun terms related to one or more embodiments of the present disclosure are explained.
[0053] Intelligent Coding Assistant: Utilizing large-scale model technology, it enhances programming tasks during the software development process and improves developers' coding efficiency. Key features of the Intelligent Coding Assistant include code auto-completion, code Q&A, automatic unit test generation, and automatic code optimization.
[0054] Code Auto-completion: Also known as code completion, this refers to improving a developer's coding speed and accuracy by predicting and suggesting possible code completion items in real time while the developer is writing code.
[0055] Abstract Syntax Tree (AST): As one of the intermediate representation forms of source code, it is used to abstract the structure of source code into a tree structure and for parsing.
[0056] Fill-In-Middle: As a data pattern for training large-scale code models, it is a data organization format structured to enable the model to predict the content of the target code once it understands the code context.
[0057] Code Metadata: Structured data of the code itself that includes supplementary information describing code components (e.g., classes, methods, properties, interfaces, filenames, etc.). While this information does not directly constitute the execution logic of the code, it is crucial for understanding, managing, and using the code.
[0058] Child Node: In an Abstract Syntax Tree (AST), all sub-nodes directly contained under a single code block node are referred to as child nodes of that code block. For example, in Java code, every statement or expression within a single code block is a child node of that code block node.
[0059] Sibling Node: Any two nodes in an AST that have the same parent node are referred to as sibling nodes.
[0060] In the field of software development, writing and maintaining code is a time-consuming and highly focused task. Intelligent coding assistants can significantly reduce manual input time and accelerate development speed through features such as code auto-completion, syntax checking, and code generation. On the other hand, intelligent coding assistants learn programming norms and best practices based on large amounts of training data, helping developers avoid common errors and providing suggestions for high-quality, standard-compliant code.
[0061] Taking code autocomplete as an example, it can currently be performed in the following two ways. In the first method, the autocomplete capability of a large-scale model can be trained using the Fill-In-Middle method; however, during the model training process, a simple random sampling method is used to randomly determine the end position of the code prefix and the start position of the code suffix in the given source code, and then all code between these two positions (middle) is used as the prediction result.<prefix, suffix, middle> The model is trained by structuring the data. However, in the above approach, the model performs no processing on the predicted middle code, causing it to predict a large amount of content at once—for example, five functions. This not only prolongs the prediction time but also disrupts the user's coding rhythm, which can severely impact the user experience of the intelligent coding assistant. In the second method, the model mostly predicts in the form of single-line completion. However, in some cases, for example, when a developer intends to complete the development of an entire function after writing a function comment, code auto-completion must be triggered multiple times, which can significantly reduce user efficiency and experience.
[0062] To solve the aforementioned problem, an embodiment of the present disclosure proposes a method for training a code processing model of an intelligent coding assistant, wherein the model can perform code task processing based on target code data written by a user by performing parsing and hierarchical sampling on original sample code data and configuring training data capable of model training through a Fill-In-Middle mode. Specifically, the target code data of a target task can be obtained, and the target code data can be input into a code processing model to obtain the code processing result of the target task. Here, the code processing model is obtained by training based on a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results, and the sample processing results and sample code block data are obtained based on hierarchical sampling of the sample code data.
[0063] It should be noted that by applying the method proposed in the embodiments of the present disclosure, an intelligent coding assistant can significantly improve a developer's coding efficiency by implementing functions such as code auto-completion, code snippet recommendation, and error detection and correction, through a deep understanding of the developer's programming intent and context information based on advanced artificial intelligence technology. Since the code processing model is obtained by learning based on sample code block data, prediction is implemented in units of code blocks, thereby shortening the prediction process. Furthermore, through hierarchical sampling, the code processing model can be enabled to adapt to various code processing scenarios (e.g., code completion scenarios, code recommendation scenarios, code comment generation scenarios, etc.), thereby precisely predicting various types of code blocks (e.g., classes, functions, logical blocks, comments, syntax, etc.) and improving task processing efficiency and accuracy.
[0064] The present disclosure provides a task processing method, and the present disclosure simultaneously relates to a code completion method, a code processing model learning method, a code processing model-based information processing method, a task processing device, a code completion device, a code processing model learning device, a code processing model-based information processing device, a model learning platform, a computing device, a computer-readable storage medium, and a computer program product, and each of these is described in detail in the following embodiments.
[0065] Referring to FIG. 1, FIG. 1 illustrates an architectural diagram of a task processing system provided in one embodiment of the present disclosure, and the task processing system may include a client (100) and a server side (200).
[0066] The client (100) is used to transmit the target code data of the target task to the server side (200);
[0067] The server side (200) inputs the code data to be processed into a code processing model to obtain the code processing result of the target task - the code processing model is obtained by learning based on a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results, and the sample processing results and sample code block data are obtained based on hierarchical sampling of the sample code data - ; is used to transmit the code processing result to the client (100);
[0068] The client (100) is also used to receive code processing results transmitted from the server side (200).
[0069] By applying the method according to the embodiment of the present disclosure, the code processing model is obtained by learning based on sample code block data, thereby implementing prediction in units of code blocks and shortening the prediction content. In addition, through hierarchical sampling, the code processing model can be adapted to various code processing scenarios, thereby accurately predicting various types of code blocks and improving task processing efficiency and accuracy.
[0070] Referring to FIG. 2, FIG. 2 illustrates an architecture diagram of another task processing system provided in one embodiment of the present disclosure, wherein the task processing system may include a plurality of clients (100) and a server side (200), wherein the clients (100) may include terminal-side devices and the server side (200) may include cloud-side devices. Communication connections may be established between the plurality of clients (100) through the server side (200), and in a task processing scenario, the server side (200) is used to provide code processing services between the plurality of clients (100), and the plurality of clients (100) may each act as a transmitting end or a receiving end, and may implement communication through the server side (200).
[0071] The user can interact with the server side (200) through the client (100) to receive data transmitted from another client (100) or transmit data to another client (100), etc. In a task processing scenario, the user transmits a data stream to the server side (200) through the client (100), and the server side (200) may generate a code processing result based on the data stream and push the code processing result to another client with which a communication connection has been established.
[0072] Here, a connection is established between the client (100) and the server side (200) via a network. The network provides a medium for a communication link between the client (100) and the server side (200). The network may include various types of connections, such as wired, wireless communication links, or fiber optic cables. Data transmitted by the client (100) can be delivered to the server side (200) only after undergoing processing such as encoding, transcoding, and compression.
[0073] The client (100) may be a web application such as a browser, an APP (Application), or an H5 (HyperText Markup Language 5) application, or a lightweight application (also called a mini-program, which is one of the lightweight applications), or a cloud application, and the client (100) may be a Software Development Kit (SDK) of a corresponding service provided by the server side (200), such as one developed and obtained based on a Real Time Communication (RTC) SDK. The client (100) may be deployed on an electronic device and may be executed by relying on the execution of the device or some APP within the device. The electronic device may be equipped with a display screen, for example, to support information viewing, and may be a personal mobile terminal such as a mobile phone, tablet computer, or personal computer. The electronic device may typically be further configured with various other types of applications, such as human-machine conversation applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messenger tools, email clients, social platform software, etc.
[0074] The server side (200) may include a server that provides various services, such as a server that provides communication services to multiple clients, a background learning server to support models used by clients, and a server that processes data transmitted by clients. It should be noted that the server side (200) may be implemented as a distributed server cluster composed of multiple servers or as a single server. The server may be a server of a distributed system or a server combined with a blockchain. The server may be a cloud server of a base cloud computing service such as a cloud service, cloud database, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, Content Delivery Network (CDN), and big data and artificial intelligence platform, or it may be an intelligent cloud computing server or an intelligent cloud host equipped with artificial intelligence technology.
[0075] It should be noted that while the task processing method provided in the embodiments of the present disclosure is generally executed by the server side, in other embodiments of the present disclosure, the client may also have a function similar to that of the server side and execute the task processing method provided in the embodiments of the present disclosure. In other embodiments, the task processing method provided in the embodiments of the present disclosure may be executed jointly by the client and the server side.
[0076] Referring to FIG. 3, FIG. 3 illustrates a flowchart of a task processing method provided in one embodiment of the present disclosure, specifically including the following steps.
[0077] Step (302): Obtain the target code data for processing of the target task.
[0078] In one or more embodiments of the present disclosure, by obtaining code data to be processed for a target task, the code data to be processed based on the target task can be processed by a task processing method based on parsing and hierarchical sampling to obtain the code processing result of the target task.
[0079] Specifically, the target task refers to a code processing task, and the target task includes, but is not limited to, code completion tasks, code recommendation tasks, and code comment generation tasks. The code data to be processed may be code data of various programming languages, such as Python, Java, etc. The code data to be processed may be code text data or code block data, and specifically, the choice is made according to the actual situation, and the embodiments of the present disclosure make no limitations thereon. Code block data may also be referred to as code block data. A code block refers to a relatively complete code semantic logic block, and a code block may be a single line or multiple lines. Some code blocks may have block headers, such as conditional statements in if blocks or method definitions in method definition blocks. Code blocks include coordinate ranges, making it convenient to extract block code from the original code, and some code blocks have unique names defined, making it convenient to associate the reference side with the definition through the unique name when restoring references or associating test cases.
[0080] In actual application, there are various methods for acquiring the target code data for a target task, and specifically, the method is selected according to the actual situation, and the embodiments of the present disclosure make no limitations thereon. In one possible embodiment of the present disclosure, the target code data for a target task transmitted from a user may be received. In another possible embodiment of the present disclosure, the target code data for a target task may be read from another data acquisition device or database.
[0081] Step (304): Code data to be processed is input into a code processing model to obtain the code processing result of the target task, wherein the code processing model is obtained by learning based on a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results, and the sample processing results and sample code block data are obtained based on hierarchical sampling of the sample code data.
[0082] In one or more embodiments of the present disclosure, after obtaining code data to be processed for a target task, the code data to be processed can be further input into a code processing model to obtain the code processing result of the target task.
[0083] Specifically, the code processing model may be a pre-trained large-scale code model, or a machine learning model obtained by training a large-scale model based on multiple sample processing results and sample code block data corresponding to each of the multiple sample processing results. The code processing model is, for example, a deep self-attention model of bidirectional encoder representations (BERT, Bidirectional Encoder Representations from Transformers) or a deep self-attention model of text-to-text (T5, Text-to-Text Transfer Transformer). The task processing result corresponds to the target task; if the target task is a code completion task, the task processing result is the code completion result, and if the target task is a code comment generation task, the task processing result is the code comment result.
[0084] It should be noted that the code processing model includes an encoding unit and a decoding unit, and when code data to be processed is input into the code processing model, the encoding unit first captures complex dependency relationships between each element within the code data to be processed through a self-attention mechanism and converts the code data to be processed into high-level semantic embedding representations hierarchically to obtain an encoding vector. Subsequently, the decoding unit obtains a code processing result by decoding the encoding vector provided by the encoding unit stepwise through a self-attention mechanism and a mask mechanism.
[0085] In one optional embodiment of the present disclosure, prior to the step of inputting code data to be processed into a code processing model to obtain a code processing result of a target task, parsing may also be performed on the code data to be processed to obtain a plurality of code block data to be processed, and furthermore, a plurality of code block data to be processed may be input into a code processing model to obtain a code processing result of a target task.
[0086] By applying the method according to the embodiment of the present disclosure, the code processing model is obtained by learning based on sample code block data, thereby implementing prediction in units of code blocks and shortening the prediction process. Furthermore, through hierarchical sampling, the code processing model can be enabled to adapt to various code processing scenarios, thereby precisely predicting various types of code blocks and improving task processing efficiency and accuracy.
[0087] In actual application, when inputting code data to be processed into a code processing model, the code processing results of the target task can be obtained by directly inputting the code data to be processed into the code processing model. Furthermore, to ensure that the code processing model can process the code data to be processed precisely, processing prompt information of the code processing model can also be obtained, and the code data to be processed can be standardized using the processing prompt information.
[0088] In one optional embodiment of the present disclosure, the step of inputting the code data to be processed into a code processing model to obtain a code processing result of a target task is:
[0089] Step of obtaining processing prompt information of the code processing model;
[0090] It may include a step of inputting processing prompt information and processing target code data into a code processing model to obtain the code processing result of a target task.
[0091] Specifically, processing prompt information is used to instruct and guide a code processing model to process target code data. The processing prompt information may include code metadata, such as a code filename, and the processing prompt information is specifically configured according to the actual situation, and the embodiments of the present disclosure make no limitations thereon. Through the code filename, the code processing model is enabled to identify the language type of the target code data, and accordingly, the target code data can be processed precisely based on syntax information corresponding to the language type.
[0092] In actual application, there are various methods for obtaining processing prompt information of a code processing model, and specifically, the method is selected according to the actual situation, and the embodiments of the present disclosure make no limitations thereon. In one possible embodiment of the present disclosure, processing prompt information of a code processing model transmitted from a user may be received. In another possible embodiment of the present disclosure, processing prompt information of a code processing model may be read from another data acquisition device or database.
[0093] By applying the method according to an embodiment of the present disclosure, processing prompt information of a code processing model is obtained, and the processing prompt information and code data to be processed are input into the code processing model to obtain the code processing result of a target task. By inputting the processing prompt information into the code processing model, the code processing model is enabled to better understand the code data to be processed, thereby ensuring the accuracy of the code processing result.
[0094] In one optional embodiment of the present disclosure, after the step of inputting the code data to be processed into a code processing model to obtain the code processing result of a target task,
[0095] Step of sending code processing results to the frontend user;
[0096] Step of receiving result feedback information sent from a frontend user - Result feedback information is information that provides feedback on the code processing result based on the task information of the target task -;
[0097] Step of constructing model optimization data based on result feedback information;
[0098] Using model optimization data, a step of performing parameter adjustments on the code processing model may be further included.
[0099] Specifically, result feedback information may be information providing feedback on the content, quality, and completeness of the code processing results, and reflects the actual feelings and expectations of frontend users regarding the code processing results; result feedback information includes, but is not limited to, result quality evaluation information, corrected accurate code processing results, and areas of model optimization. Model optimization data refers to accurate optimization sample data for optimizing the code processing model.
[0100] In actual application, there are various methods for constructing model optimization data based on result feedback information, and specifically, the method is selected according to the actual situation, and the embodiments of the present disclosure make no limitations thereon. In one possible embodiment of the present disclosure, model optimization data can be constructed directly based on result feedback information. In another possible embodiment of the present disclosure, optimization prompt information can be generated based on result feedback information, and model optimization data transmitted by a frontend user based on the optimization prompt information can be received.
[0101] Taking the example of directly constructing model optimization data based on result feedback information, it should be noted that if the result feedback information is an accurate code processing result that has been accurately modified, model optimization data can be constructed based on the target code data of the target task and the modified accurate code processing result. If the result feedback information is the optimization field of the model, e.g., the XXX field, sample code data for the XXX field can be obtained, and the sample code data for the XXX field can be determined as the model optimization data. Here, the process of performing parameter adjustments on the code processing model using the model optimization data is identical to the training process of the code processing model, so it is not described redundantly in the embodiments of this disclosure.
[0102] By applying the method according to an embodiment of the present disclosure, a code processing result is transmitted to a frontend user, and result feedback information transmitted from the frontend user is received—the result feedback information is information that provides feedback on the code processing result based on task information of a target task—model optimization data is constructed based on the result feedback information, and parameter adjustments are performed on the code processing model using the model optimization data. By collecting and utilizing the result feedback information, the performance of the code processing model is continuously optimized to more accurately satisfy the actual requirements of the frontend user, and the quality and accuracy of the final code processing result are improved.
[0103] In one optional embodiment of the present disclosure, the step of constructing model optimization data based on the result feedback information is,
[0104] Step of generating optimization prompt information based on result feedback information - Optimization prompt information is used to guide the frontend user to send model optimization data to optimize the code processing model -;
[0105] It may include the step of sending optimization prompt information to a frontend user and receiving model optimization data sent by the frontend user based on the optimization prompt information.
[0106] It should be noted that there are various methods for generating optimization prompt information based on result feedback information, and specifically, the method is selected according to the actual situation, and the embodiments of the present disclosure do not make any limitations thereon.
[0107] In one possible embodiment of the present disclosure, pre-set prompt information may be obtained directly, and result feedback information may be added to the pre-set prompt information to obtain optimization prompt information. For example, the pre-set prompt information is "I am very sorry for providing inaccurate information. If you could specifically point out which parts are not sufficiently accurate or provide an accurate answer to the relevant problem, I will correct and optimize my answer as soon as possible to provide better service." If the result feedback information is "The result is inaccurate," the optimization prompt information is "Regarding the inaccurate result you provided feedback on, I am very sorry for providing inaccurate information. If you could specifically point out which parts are not sufficiently accurate or provide an accurate answer to the relevant problem, I will correct and optimize my answer as soon as possible to provide better service."
[0108] In another possible embodiment of the present disclosure, type identification is performed on result feedback information to determine the information type of the result feedback information, and further, the information type is matched with the prompt type of each prompt information in the prompt information library, and the prompt information whose prompt type is the same as the information type can be determined as the optimized prompt information.
[0109] By applying the method according to an embodiment of the present disclosure, optimization prompt information is generated based on result feedback information—the optimization prompt information is used to guide a frontend user to transmit model optimization data for optimizing a code processing model—the optimization prompt information is transmitted to a frontend user, and model optimization data transmitted by the frontend user based on the optimization prompt information is received. By obtaining model optimization data through an interactive guidance method, interactivity with the user is improved, and user satisfaction is enhanced.
[0110] In one optional embodiment of the present disclosure, after the step of inputting the code data to be processed into a code processing model to obtain the code processing result of a target task,
[0111] A step of obtaining an updated code processing result by annotating key information among the code processing results;
[0112] It may further include a step of sending the updated code processing results to the frontend user.
[0113] Specifically, core information refers to information among the results of code processing that helps users understand, debug, and maintain the code, such as functions, classes, and objects.
[0114] In actual application, prior to the step of obtaining an updated code processing result by annotating the key information within the code processing result, key information identification can be performed on the code processing result to determine the key information within the code processing result. Here, there are various methods for performing key information identification on the code processing result, and specifically, the method is selected according to the actual situation, and the embodiments of the present disclosure do not limit such methods. In one possible embodiment of the present disclosure, the code processing result can be matched with pre-set key information, and information identical to the pre-set key information appearing in the code processing result can be determined as key information. In another possible embodiment of the present disclosure, the code processing result can be input into a key information identification model to obtain key information, wherein the key information identification model is obtained by training based on a plurality of training code data and the key information labels of each training code data.
[0115] When annotating key information among the code processing results, processing such as bolding, highlighting, italicizing, and adding borders can be performed on the key text information, and the method of annotating key information is specifically selected according to the actual situation, and the embodiments of the present disclosure do not make any limitations thereon.
[0116] By applying the method according to the embodiment of the present disclosure, key information among the code processing results is annotated to obtain an updated code processing result, and the updated code processing result is transmitted to a frontend user, thereby making it convenient for the frontend user to check the result and improving the user experience.
[0117] In one optional embodiment of the present disclosure, after the step of inputting the code data to be processed into a code processing model to obtain the code processing result of a target task,
[0118] Step of sending code processing results to the frontend user;
[0119] Step of receiving modification information sent from the frontend user - modification information is used to modify the code processing result -;
[0120] It may further include a step of obtaining a modified code processing result by modifying the code processing result based on the modification information.
[0121] Specifically, modification information is used to describe modification requests from frontend users regarding the results of code processing. Modification requests include, but are not limited to, requests to add comments and requests to generate test information. Furthermore, modification information may also be modification request information regarding key information within the results of code processing.
[0122] In actual application, there are various methods for modifying code processing results based on modification information, and specifically, the method is selected according to the actual situation, and the embodiments of the present disclosure make no limitations thereon. In one possible embodiment of the present disclosure, a code modification tool (e.g., a text editor) may be called to modify the code processing results based on the modification information. In another possible embodiment of the present disclosure, the task processing results and modification information may be input into a modification content generation model to obtain modification content corresponding to the modification information, and the code processing results may be updated using the modification content to obtain the modified code processing results, wherein the modification content generation model is obtained by learning based on a plurality of sample codes, sample modification information of each sample code, and actual modification results.
[0123] By applying the method according to an embodiment of the present disclosure, a code processing result is transmitted to a frontend user, and modification information transmitted from the frontend user is received—wherein the modification information is used to modify the code processing result—; and the code processing result is modified based on the modification information to obtain a modified code processing result. By modifying the code processing result based on the modification information transmitted from the frontend user, human-machine interactivity is increased, and the adaptability and flexibility of task processing are also improved.
[0124] In one optional embodiment of the present disclosure, a learning method of a code processing model is described, that is, prior to the step of inputting the code data to be processed into the code processing model to obtain the code processing result of a target task,
[0125] A step of obtaining a sample set containing multiple sample code data;
[0126] A step of performing hierarchical sampling on a plurality of sample code data to obtain a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results;
[0127] A step of inputting multiple sample code block data into a code processing model to obtain a prediction processing result corresponding to each of the multiple sample code block data;
[0128] It may further include a step of training a code processing model based on sample processing results and prediction processing results to obtain a trained code processing model.
[0129] Specifically, the learning method of the code processing model is supervised learning; that is, sample code block data contains actual processing labels (sample processing results), and these sample processing results serve as the generation goals of the code processing model and are used to guide the model's learning process. The sample code block data may include the preceding and succeeding contexts of the code block processing results, and through this data, the model can predict the sample processing results. The code processing model may be a pre-trained large-scale model or an untrained natural language processing neural network model. The sample code data can be referred to as source code data, and since the sample code data relates to various programming languages and fields, this allows the model to learn from rich source code data. This ensures that the data encountered during the learning process closely approximates the developer's actual development scenarios, while simultaneously improving the model's knowledge density and generalization ability to provide valuable code processing suggestions for any developer's arbitrary development scenario.
[0130] In actual code processing scenarios, it should be noted that there is a mutual nesting relationship between code blocks, for example, that the SortAlgorithm class contains two functions, and the first function contains content such as two statements and one logic block. Data features between mutually nested code blocks belong to different layers, and data features of different layers can be viewed as a type of variable. Since various combinations may be required for each value of these variables, by considering these variables and the combination situations of these variables as much as possible during sampling, richer context scenarios can be simulated, thereby improving the generalization of the sampled model training data.
[0131] Accordingly, in order to enable the model to provide valuable code processing suggestions in various development scenarios, embodiments of the present disclosure propose a hierarchical sampling method, namely, performing hierarchical sampling based on data features of different layers of the code and constructing model training data (sample code block data and sample processing results) for each language and various context scenarios. During hierarchical sampling, random sampling may be performed on data features of different layers, or goal-oriented sampling may be performed according to the actual needs of the user. The model training data obtained after sampling is used for the continuous training of the model.<prefix+suffix, middle> It can be output in the format. For example, data features of different layers include processing topics and context scenarios, and when performing hierarchical sampling on multiple sample code data, a code processing topic can be randomly sampled first, and then a code block data can be found as a model creation goal (sample processing result) under any code processing topic, that is, under the sampled code processing topic, a context scenario can be randomly sampled to determine the model creation goal.
[0132] Furthermore, hierarchical sampling is performed on multiple sample code data to obtain multiple sample processing results and sample code block data corresponding to each of the multiple sample processing results, and then the multiple sample code block data is directly input into a code processing model to obtain prediction processing results corresponding to each of the multiple sample code block data. In addition, the sample processing results and the sample code block data corresponding to the sample processing results are configured as model training data according to a preset format, and the code processing model is continuously trained using these model training data, and subsequently, the trained code processing model can be deployed to an intelligent coding assistant. Here, the preset format is, for example, " <filename>Code filename<fim_prefix> Code block preceding context<fim_suffix> Code block trailing context<fim_middle> It is the "target code block," the code block preceding context and the code block succeeding context are sample code block data, and the target code block is the sample processing result.
[0133] It should be noted that there are various methods for acquiring sample sets, specifically selected according to actual circumstances, and the embodiments of the present disclosure make no limitations thereon. In one possible embodiment of the present disclosure, a large amount of sample code data may be acquired from other data acquisition devices or databases (e.g., The Stack dataset containing source code data for projects of various languages). In another possible embodiment of the present disclosure, a large amount of sample code data may be crawled from a network.
[0134] In actual application, when training a code processing model based on sample processing results and prediction processing results, a loss value is calculated based on the sample processing results and prediction processing results, the model parameters of the code processing model are adjusted based on the loss value, and the training process is performed until a preset stopping condition is satisfied to obtain a trained code processing model. Here, there are various functions for calculating the loss value, such as cross-entropy loss function, L1 norm loss function, maximum loss function, mean squared error loss function, log loss function, etc., and are specifically selected according to actual conditions, and the embodiments of the present disclosure make no limitations thereto. The preset stopping condition includes, but is not limited to, that the loss value is below a preset threshold and that the number of iterations reaches a preset number of iterations, and the preset threshold and the preset number of iterations are specifically selected according to actual conditions, and the embodiments of the present disclosure make no limitations thereto.
[0135] In one possible embodiment of the present disclosure, after calculating a loss value, the loss value is compared with a preset threshold. Specifically, if the loss value is greater than the preset threshold, it means that the difference between the sample processing result and the prediction processing result is relatively large and the prediction ability of the code processing model for the sample code block data is relatively low. In this case, the model parameters of the code processing model are adjusted, and the step of inputting multiple sample code block data into the code processing model to obtain a prediction processing result corresponding to each of the multiple sample code block data is executed again, thereby allowing the code processing model to be continuously trained until the loss value becomes less than or equal to the preset threshold. This means that the difference between the sample processing result and the prediction processing result is relatively small, reaching a preset stopping condition, and a trained code processing model is obtained.
[0136] In other possible embodiments of the present disclosure, in addition to comparing the magnitude relationship between the loss value and a preset threshold, the number of iterations may be combined to determine whether the current code processing model has completed training. Specifically, if the loss value is greater than the preset threshold, the model parameters of the code processing model are adjusted, and the step of inputting a plurality of sample code block data into the code processing model to obtain a prediction processing result corresponding to each of the plurality of sample code block data is executed again to continue training the code processing model, and when a preset number of iterations is reached, the iteration is stopped to obtain a code processing model that has completed training.
[0137] By applying the method according to the embodiment of the present disclosure, a code processing model is trained based on sample processing results and prediction processing results, and the code processing model is continued to be trained when a preset stopping condition is not satisfied, and the training is completed by performing the process until the preset stopping condition is reached, thereby obtaining a code processing model. By continuously adjusting the model parameters of the code processing model, the finally obtained code processing model can be made more precise.
[0138] In practical applications, there are various methods for performing hierarchical sampling on multiple sample code data, and specifically, the method is selected according to the actual situation, and the embodiments of the present disclosure make no limitations thereon. In one possible embodiment of the present disclosure, hierarchical sampling may be performed directly on the sample code data to obtain multiple sample processing results and sample code block data corresponding to each of the multiple sample processing results. In another possible embodiment of the present disclosure, in order to ensure that hierarchical sampling can be performed on code blocks as units, hierarchical sampling may be performed on multiple candidate code block data to obtain multiple sample processing results and sample code block data corresponding to each of the multiple sample processing results, wherein the multiple candidate code block data is obtained based on performing syntactic analysis on the sample code data.
[0139] In one optional embodiment of the present disclosure, the step of performing hierarchical sampling on a plurality of sample code data to obtain a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results is
[0140] A step of obtaining multiple candidate code block data by performing parsing on the first sample code data for the first sample code data - the first sample code data is any one of the multiple sample code data -;
[0141] It may include a step of performing hierarchical sampling on multiple candidate code block data to obtain multiple sample processing results and sample code block data corresponding to each of the multiple sample processing results.
[0142] Specifically, parsing is used to determine multiple code blocks within code data. Through code blocks, the model can understand which code in the data is a class, which is a function, which is a logical block, and so on. Furthermore, by enabling the model to recognize the syntax tree of the generated result, it is possible to clarify what level of granularity the generated content should possess.
[0143] In actual application, there are various methods for obtaining multiple candidate code block data by performing parsing on the first sample code data, and specifically, the method is selected according to the actual situation, and the embodiments of the present disclosure make no limitations thereon. In one possible embodiment of the present disclosure, the first sample code data may be matched with pre-set code block data, and data identical to the pre-set code block data appearing in the first sample code data may be determined as candidate code block data. In another possible embodiment of the present disclosure, parsing may be performed on the first sample code data using an AST parsing tool, and multiple candidate code block data may be obtained, wherein the abstract syntax tree is a type of structured representation of programming language source code.
[0144] It is noted that when performing hierarchical sampling on multiple candidate code block data, a context-aware hierarchical sampling method can be used to obtain sample processing results and sample code block data corresponding to each sample processing result from multiple candidate code block data related to different topics, context scenarios, completeness, and completeness granularity.
[0145] By applying the method according to the embodiment of the present disclosure, for a first sample code data, syntactic analysis is performed on the first sample code data to obtain a plurality of candidate code block data - the first sample code data is any one of the plurality of sample code data -; hierarchical sampling is performed on the plurality of candidate code block data to obtain a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results, and through a syntactic analysis and rich scenario-based hierarchical sampling method, model training data for each language and various context scenarios is constructed so that the model training data more closely matches the actual development scenario.
[0146] In one optional embodiment of the present disclosure, a specific implementation process of hierarchical sampling is described by citing a hierarchical sampling method classified into code processing topics and code sampling strategies, namely, the step of performing hierarchical sampling on the plurality of candidate code block data to obtain a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results,
[0147] A step of obtaining a target code processing topic by sampling from code processing topics - the code processing topic includes at least one of a code block processing topic, a code block content processing topic, a code block partial content processing topic, and a code block empty content processing topic -;
[0148] Under the target code processing topic, according to the code sampling strategy, the method may include the step of sampling from multiple candidate code block data to obtain multiple sample processing results and sample code block data corresponding to each of the multiple sample processing results.
[0149] Specifically, a code sampling strategy refers to a strategy of sampling based on code context scenarios. A code context scenario refers to a scenario composed of code context content. A code processing topic refers to the processing topic of the code data sampled each time. Code processing topics can be used to describe the model's target processing scope for code data. Taking code completion topics as an example, a code completion topic includes at least one of a code block completion topic, a code block content completion topic, a code block partial content completion topic, and a code block empty content completion topic.
[0150] The Code Block Processing topic indicates that the model's target processing scope under that topic is a single code block. Taking the Code Block Completion topic as an example, the model's goal under the Code Block Completion topic is to complete a single complete code block. For example, this involves completing a single complete function or a single complete logic block.
[0151] The Code Block Content Processing topic indicates that the model's target processing scope under that topic is the complete content within a single code block. Taking the Code Block Content Completion topic as an example, the model's goal under this topic is to completely complete the content within a single code block—that is, to complete the entire content. For instance, completion is performed on statements with parts removed in the middle, ultimately completing the complete content of the corresponding code block.
[0152] The Code Block Partial Content Processing topic indicates that the model's target processing scope under that topic is a portion of the content within a single code block. Taking the Code Block Partial Content Completion topic as an example, the model's goal under the Code Block Partial Content Completion topic is to complete a portion of the content within a single code block. For example, when there is currently one incomplete statement "arr[]", only the content inside the brackets [] is completed, and the content outside the brackets [] does not need to be completed.
[0153] The Code Block Empty Content Handling topic indicates that the model's target processing scope under that topic is empty content, that is, not processing any content. Taking the Code Block Empty Content Completion topic as an example, the model's goal under the Code Block Empty Content Completion topic is not to complete any content within a single complete context. For instance, if the current statement is "result = sort.bubble_sort(arr)" and the cursor is positioned after "arr" and before ""), the model predicts that empty content is completed.
[0154] By applying the method according to the embodiment of the present disclosure, a target code processing topic is obtained by sampling from a code processing topic, and a plurality of candidate code block data are sampled according to a code sampling strategy under the target code processing topic to obtain a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results. Through a rich scenario-based hierarchical sampling method, model training data is constructed in different languages (e.g., Python, Java, etc.) and different context scenarios, thereby ensuring that the model training data better matches actual development scenarios.
[0155] In actual application, under the subject of target code processing, when obtaining multiple sample processing results and sample code block data corresponding to each of the multiple sample processing results by sampling from multiple candidate code block data according to a code sampling strategy, at least one strategy among the multiple code sampling strategies is randomly selected, and the multiple sample processing results and sample code block data corresponding to each of the multiple sample processing results can be obtained by sampling from multiple candidate code block data using the selected strategy. Furthermore, to ensure the accuracy of the sample processing results and sample code block data obtained by sampling, the code sampling strategy can be classified into a result sampling strategy for obtaining sample processing results by sampling and a code block sampling strategy for obtaining sample code block data by sampling. Accordingly, at least one strategy is randomly selected from the result sampling strategy and the code block sampling strategy, respectively, and the sample processing results and sample code block data are obtained by sampling using the selected strategy.
[0156] In one optional embodiment of the present disclosure, the code sampling strategy includes a result sampling strategy and a code block sampling strategy, the result sampling strategy includes at least one of a category sampling strategy, a preceding context category sampling strategy, a succeeding context category sampling strategy, a child node sampling strategy, and a completeness sampling strategy, and the code block sampling strategy includes at least one of a preceding context content sampling strategy and a succeeding context content sampling strategy, and the step of obtaining a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results by sampling from a plurality of candidate code block data according to the code sampling strategy under the target code processing topic is
[0157] Under the subject of target code processing, the method may include the step of obtaining multiple sample processing results by sampling from multiple candidate code block data according to a result sampling strategy, and obtaining sample code block data corresponding to each of the multiple sample processing results by sampling from multiple candidate code block data according to a code block sampling strategy.
[0158] A category sampling strategy refers to a strategy of sampling processing results based on category variables. Categories include classes, functions, logical blocks, comments, syntax, etc. Through a category sampling strategy, code blocks belonging to a specific category can be selected as the sample processing result from data of multiple candidate code blocks.
[0159] The preceding context category sampling strategy refers to a strategy that samples processing results based on the categorical variables of the previous code block. Through the preceding context category sampling strategy, among multiple candidate code block data, a code block for which the previous code block belongs to a specific category can be selected as the processing result. For example, if the previous code block is a comment and the processing result is a function, this indicates that the user's coding scenario requires the user to write a complete function corresponding to a single function comment.
[0160] The trailing context category sampling strategy refers to a strategy that samples processing results based on the category variable of the next code block. Through the trailing context category sampling strategy, among multiple candidate code block data, a code block belonging to a specific category can be selected as the processing result. For example, if the processing result is a comment and the next code block is a class, this indicates that the user's coding scenario is to write a class and then add documentation comments to it.
[0161] The child node sampling strategy refers to a strategy for sampling processing results based on whether a data point contains child nodes or the number of child nodes included. Through the child node sampling strategy, among multiple candidate code block data, code blocks containing child nodes, code blocks not containing child nodes, or code blocks with a number of child nodes greater than a preset value can be selected as the sample processing results. For example, if a code block contains child nodes and another code block does not, these two situations can be distinguished during sampling to ensure that model training data exists for each situation. Furthermore, the number of child nodes included in a code block may vary; the more child nodes included, the higher the difficulty of the code block, and the fewer child nodes included, the lower the difficulty of the code block. For example, some functions may contain only a single return statement, while others may contain dozens of child nodes such as various logical blocks, statements, and comments; since their difficulties are not uniform, simple code blocks can be sampled, as can complex code blocks.
[0162] The completeness sampling strategy refers to a strategy that samples processing results based on the completeness of code blocks. Through the completeness sampling strategy, code blocks with different completeness levels can be selected as the processing result from among multiple candidate code block data. For example, when the completed processing result is a single function, there are three possible cases: first, when the user does not enter any content for the function to be completed; second, when the user enters a single indentation (tab) for the function to be completed; and third, when the user enters part of the function to be completed. Since developers may encounter arbitrary cases while coding, code blocks with different completeness levels can be sampled as the processing result.
[0163] The preceding context content sampling strategy refers to a strategy of sampling sample code block data based on the preceding context content of the sample processing result. The preceding context content of the sample processing result may include the following: first, the original preceding context content of the sample code data; second, the preceding context content of the last K child nodes of the sample processing result; third, the preceding context content of the last K sibling nodes of the sample processing result; and fourth, the preceding context content being empty. Different sample code block data of the same sample processing result can be obtained by sampling through different preceding context contents. Here, K is a positive integer.
[0164] The trailing context content sampling strategy refers to a strategy for sampling sample code block data based on the trailing context content of the sample processing result. The trailing context content of the sample processing result may include the following: first, the original trailing context content of the sample code data; second, the trailing context content of the last K child nodes of the sample processing result; third, the trailing context content of the last K sibling nodes of the sample processing result; and fourth, the trailing context content being empty. Through different trailing context contents, different sample code block data of the same sample processing result can be sampled and obtained.
[0165] By applying the method according to the embodiment of the present disclosure, under the target code processing topic, a plurality of sample processing results are obtained by sampling from a plurality of candidate code block data according to a result sampling strategy, and sample code block data corresponding to each of the plurality of sample processing results is obtained by sampling from a plurality of candidate code block data according to a code block sampling strategy. The category sampling strategy refers to a strategy of sampling sample processing results based on category variables. Categories include classes, functions, logical blocks, comments, syntax, etc. Through the category sampling strategy, a code block belonging to one of the categories from a plurality of candidate code block data can be selected as a sample processing result. Since different processing ranges under a code processing scenario can be simulated through the target code processing topic, sample processing results that are richer and more consistent with actual development scenarios can be obtained by sampling through the result sampling strategy, and sample code block data that is richer and more consistent with actual development scenarios can be obtained by sampling through the code block sampling strategy, the model training data is made to be more consistent with actual development scenarios.
[0166] Hereinafter, with reference to FIG. 4, the task processing method provided in the present disclosure is further explained by citing an example where the method is applied to a code auto-completion scenario. FIG. 4 illustrates a flowchart of a code completion method provided in one embodiment of the present disclosure and specifically includes the following steps.
[0167] Step (402): Obtain the target code data for the code completion task.
[0168] Step (404): Code data to be completed is input into a code completion model to obtain a code completion result of a code completion task, wherein the code completion model is obtained by training based on a plurality of sample completion results and sample code block data corresponding to each of the plurality of sample completion results, and the sample completion results and sample code block data are obtained based on hierarchical sampling of the sample code data.
[0169] It should be noted that the implementation forms of steps (402) to (404) are identical to the implementation forms of steps (302) to (304), and since the relevant descriptions of the code data to be completed and the code completion model can all be referenced from the relevant descriptions of the code data to be processed and the code processing model described above, they are not further explained redundantly in the embodiments of the present disclosure.
[0170] By applying the method according to the embodiment of the present disclosure, regarding the capability that a code completion model must satisfy, model training data is constructed through a code syntax analysis and rich scenario-based hierarchical sampling method, and a code completion model is obtained by training using the model training data. This enables the code completion model to accurately predict various types of code blocks (which may be a class, function, logic block, comment, syntax, etc.) that the user intends to input next, based on the code preceding context, code succeeding context, and some code metadata written by the user in various scenarios of actual development, thereby improving the user's coding efficiency and experience.
[0171] In one optional embodiment of the present disclosure, prior to the step of obtaining the code data to be completed of the code completion task,
[0172] Step of sending completion prompt information to the frontend user - Completion prompt information is used to guide the frontend user to send the code data to be completed for code completion task processing -;
[0173] It may further include a step of receiving the target code data for a code completion task sent by a frontend user based on completion prompt information.
[0174] It should be noted that when a code completion task starts, completion prompt information can first be sent to the frontend user. Based on this prompt information, the frontend user can understand the processing procedure or requirements of the code completion task and, accordingly, input the code data to be completed that is more suitable for the actual requirements. For example, the code completion information could be: "Hello, I am your intelligent coding assistant. I am ready for various code completions, so you can directly input the code data you wish to complete."
[0175] Furthermore, after sending completion prompt information to the frontend user, the code data to be completed for the code completion task sent by the frontend user based on the completion prompt information is received, and the code data to be completed is input into the code completion model to obtain the code completion result of the code completion task.
[0176] By applying the method according to an embodiment of the present disclosure, completion prompt information is transmitted to a frontend user—the completion prompt information is used to guide the frontend user to transmit code data to be completed for processing a code completion task—; and the code data to be completed for a code completion task transmitted by the frontend user based on the completion prompt information is received. By guiding the frontend user to input the code data to be completed through the completion prompt information, human-machine interaction is increased, thereby enabling the code data to be completed to accurately reflect the user's actual requirements.
[0177] In one optional embodiment of the present disclosure, after the step of inputting the code data to be completed into a code completion model to obtain the code completion result of a code completion task,
[0178] Step of sending code completion results to the frontend user;
[0179] Step of receiving result feedback information sent from a frontend user - Result feedback information is information that provides feedback on the code completion result based on the task information of the code completion task -;
[0180] It may further include the step of transmitting result feedback information to a model training platform—the model training platform is used to perform parameter adjustments for a code completion model using the result feedback information—;
[0181] Specifically, the model training platform may be deployed on a terminal device or on a cloud-side device, and specifically configured according to actual conditions, and the embodiments of the present disclosure make no limitations thereto. The model training platform provides large-scale data processing and high-performance computing resources and is used to train, optimize, and deploy various machine learning models, particularly deep learning models. In the model training platform, users can upload data, select or customize algorithm models, and efficiently perform model training and verification through distributed computing capabilities. The model training platform receives a task creation request from a terminal device, obtains a corresponding code processing model based on the request information, and can generate task information based on the code processing model. The model training platform can also respond quickly to the demands of different tasks, call an appropriate code processing model to perform task processing, and finally generate high-quality code processing results.
[0182] In actual application, after receiving result feedback information transmitted from a frontend user, the result feedback information can be transmitted to a model training platform, and the model training platform can perform parameter adjustments for a code completion model based on the result feedback information. Since the method by which the model training platform performs parameter adjustments for a code completion model based on the result feedback information can refer to an implementation form of "constructing model optimization data based on the result feedback information and performing parameter adjustments for a code processing model using the model optimization data," it is not described redundantly in the embodiments of the present disclosure.
[0183] By applying the method according to the embodiment of the present disclosure, the model learning platform performs parameter adjustments on the code completion model based on result feedback information, thereby ensuring the precision and learning efficiency of the code completion model while simultaneously lowering the model learning cost and providing a convenient and efficient model learning service to the user.
[0184] Referring to FIG. 5, FIG. 5 illustrates a flowchart of a code processing model learning method provided in one embodiment of the present disclosure, specifically including the following steps.
[0185] Step (502): Obtain a sample set, wherein the sample set includes multiple sample code data.
[0186] Step (504): Hierarchical sampling is performed on multiple sample code data to obtain multiple sample processing results and sample code block data corresponding to each of the multiple sample processing results.
[0187] Step (506): Input multiple sample code block data into a code processing model to obtain prediction processing results corresponding to each of the multiple sample code block data.
[0188] Step (508): A code processing model is trained based on the sample processing results and the prediction processing results, and the trained code processing model is obtained.
[0189] It should be noted that the implementation forms of steps (502) to (508) can refer to the learning method of the code processing model, and therefore are not further described redundantly in the embodiments of the present disclosure.
[0190] In actual application, after obtaining a trained code processing model, the model parameters of the trained code processing model can be transmitted to a terminal device, and the frontend user can build a code processing model locally based on the model parameters and perform tasks such as code completion using the code processing model.
[0191] By applying the method according to the embodiment of the present disclosure, a code processing model is trained based on sample processing results and prediction processing results, and the code processing model is continued to be trained when a preset stopping condition is not satisfied, and the training is completed by performing the process until the preset stopping condition is reached, thereby obtaining a code processing model. By continuously adjusting the model parameters of the code processing model, the finally obtained code processing model can be made more precise.
[0192] Referring to FIG. 6, FIG. 6 illustrates a flowchart of the processing process of a code processing model learning method provided in one embodiment of the present disclosure, and specifically includes the following.
[0193] Source Code Data Collection: By collecting source code data related to various programming languages and fields as sample code data, the model is enabled to learn from rich source code data. This ensures that the data encountered during the model training process closely approximates the user's actual development scenarios, while simultaneously improving the model's knowledge density and generalization ability to provide valuable complete proposals for any developer's arbitrary development scenarios.
[0194] AST parsing of source code data: For the first sample code data, parsing is performed on the first sample code data to obtain multiple candidate code block data, wherein the first sample code data is any one of the multiple sample code data. By performing parsing on the source code data, the model can understand which code in the code data is a class, which code is a function, and which code is a logical block. Additionally, by enabling syntax tree recognition for the generated result to clarify what level of granularity the generated content should have, only the content of the next code block (one function, one logical block, or one syntax) is predicted, thereby significantly shortening the prediction content, precisely satisfying user requirements, and providing more valuable code processing suggestions to the user without disrupting the user's coding rhythm.
[0195] Construction of training data through multi-scenario-based hierarchical sampling: obtaining a target code processing topic by sampling from a code processing topic, wherein the code processing topic includes at least one of a code block processing topic, a code block content processing topic, a code block partial content processing topic, and a code block empty content processing topic; under the target code processing topic, obtaining multiple sample processing results by sampling from multiple candidate code block data according to a result sampling strategy, and obtaining sample code block data corresponding to each of the multiple sample processing results by sampling from multiple candidate code block data according to a code block sampling strategy, wherein the result sampling strategy includes at least one of a category sampling strategy, a preceding context category sampling strategy, a succeeding context category sampling strategy, a child node sampling strategy, and a completeness sampling strategy, and the code block sampling strategy includes at least one of a preceding context content sampling strategy and a succeeding context content sampling strategy; and through a hierarchical sampling method, sampling multiple candidate code block data obtained by performing syntactic analysis, and constructing model training data for each language and various context scenarios.
[0196] Model Training: Multiple sample code block data are input into a code processing model to obtain prediction processing results corresponding to each of the multiple sample code block data, and the code processing model is trained based on the sample processing results and prediction processing results to obtain a trained code processing model.
[0197] By applying the method according to the embodiment of the present disclosure, through AST parsing and hierarchical sampling methods, it is possible to accurately predict the type of logical block that a user intends to process in different topics and different context scenarios, thereby enabling the model to predict the code content of each level of granularity, such as a complete class, function, logical block, comment, or syntax.
[0198] Referring to FIG. 7, FIG. 7 illustrates a flowchart of an information processing method based on a code processing model provided in one embodiment of the present disclosure, specifically including the following steps.
[0199] Step (702): Receive a task creation request containing request information.
[0200] Specifically, the information processing method based on a code processing model may be applied to terminal devices or to model training platforms. A task creation request is used to request the creation of task information for a target task. A task creation request typically includes a task type, an expected output format, and request information. For example, when a user selects the "Code Completion" function on the frontend interface of a model training platform and uploads code data, a task creation request may be constructed; this request includes information such as code data, a task type (i.e., code completion), and the language type of the code data. Request information refers to parameters or descriptive information related to the target task included in the task creation request. Request information includes, but is not limited to, the target task scenario identifier, task model identifier, or the target task model training data.
[0201] Step (704): A code processing model is obtained based on request information, wherein the code processing model is obtained by training based on multiple sample processing results and sample code block data corresponding to each of the multiple sample processing results, and the sample processing results and sample code block data are obtained based on hierarchical sampling of the sample code data.
[0202] In one optional embodiment of the present disclosure, the request information includes a task scenario identifier or a task model identifier of a target task, and the step of obtaining a code processing model based on the request information is
[0203] A step of determining a target scenario template from multiple pre-configured scenario templates based on a task scenario identifier, and searching for a code processing model in a model library based on the target scenario template - multiple processing models are stored in the model library -;
[0204] or,
[0205] It may include a step of searching for a code processing model in a model library based on a task model identifier.
[0206] Specifically, a task scenario identifier refers to a unique or specific tag for distinguishing different task application scenarios. In an embodiment of the present disclosure, the task scenario identifier is part of the request information. Through the task scenario identifier, task information can be generated by selecting a target scenario template that matches the request information from a series of pre-configured scenario templates. For example, if the task scenario identifier is "code completion," it means that the frontend user wishes to perform code completion on the uploaded code data, and a code completion scenario template can be selected from a plurality of pre-configured scenario templates based on the task scenario identifier.
[0207] Pre-configured scenario templates are standard configuration scenario templates predefined for different task application scenarios, and each template includes information such as model information and task processing procedures that match the said task application scenario. Through a series of pre-configured scenario templates, it is possible to respond quickly to task creation requests for different scenarios. Different pre-configured scenario templates correspond to different task types, model information, and processing procedures. For example, among the pre-configured scenario templates, there may be a scenario template specialized for code completion, which includes model information and processing procedures of an already trained code completion model.
[0208] A target scenario template refers to a scenario template that matches a task scenario identifier. When parsing a task creation request, a corresponding target scenario template is identified based on the task scenario identifier, and a corresponding code processing model and other related configuration information can be selected from the model library according to the model information included in the target scenario template. For example, when the task scenario identifier is "Code Completion," the target scenario template is a template that includes the model information of the code completion model and related configuration parameters.
[0209] A model library is a resource library that stores deep learning models intensively, and these models are trained and optimized to solve different processing tasks. Processing models stored in the model library include, but are not limited to, code completion models, code comment generation models, and code recommendation models. Additionally, processing models within the model library may be classified into different versions depending on differences in application scenarios; for example, the model library may store multiple versions of a code completion model, such as a code completion model for the Python language and a code completion model for the Java language.
[0210] A task model identifier refers to a unique or specific tag used to distinguish models applied to different tasks. For example, a task model identifier could be "Code Completion," and based on this identifier, you can search the model library for the code completion model applied to handling the code completion task.
[0211] By applying the method according to an embodiment of the present disclosure, a target scenario template is determined from a plurality of preset scenario templates based on a task scenario identifier, and a code processing model is searched from a model library based on the target scenario template—where a plurality of processing models are stored in the model library—; or, a code processing model is searched from a model library based on a task model identifier. Through predefined task scenario templates, task model identifiers, and model library resources, the process of acquiring a code processing model is made more flexible, efficient, and standardized.
[0212] In another optional embodiment of the present disclosure, in addition to selecting a pre-trained code processing model from a model library, a code processing model may also be trained and obtained for a specific purpose based on model training data in request information, that is, the request information includes a plurality of sample code data of a target task, and the step of obtaining a code processing model based on the request information is
[0213] Based on multiple sample code data, the method may include a step of training a code processing model corresponding to a target task to obtain a trained code processing model.
[0214] It should be noted that the implementation form of "the step of training a code processing model corresponding to a target task based on multiple sample code data to obtain a trained code processing model" is identical to the training method of the code processing model, and therefore is not described redundantly in the embodiments of the present disclosure.
[0215] By applying the method according to the embodiment of the present disclosure, a code processing model corresponding to a target task is trained based on a plurality of sample code data to obtain a trained code processing model, thereby allowing the code processing model to better meet the user's requirements and ensuring the precision of the code processing model.
[0216] Step (706): Creates task information based on a code processing model, where the task information is used to execute the target task.
[0217] Specifically, task information includes the model configuration and processing procedures necessary to execute a target task. A terminal device or other server-side component can process the target task by accurately using a code processing model based on the task information.
[0218] It should be noted that when generating task information based on a code processing model, task information can be obtained by directly packaging the code processing model. Additionally, model information from the code processing model can be acquired, and task information can be constructed based on this model information; here, the model information includes, for example, the configuration of model parameters, the method of processing input data, specifications of expected output results, and related intermediate steps and other auxiliary information.
[0219] For example, in a code completion task, task information may include parameters such as the address information of a selected code completion model, the storage location of input code data, the target path of the output completion result, and other environment configurations required for the execution of the code completion model, and based on this information, the code completion model can be accurately loaded on a local or remote server and the code completion task can be executed.
[0220] By applying the method according to the embodiment of the present disclosure, by generating task information for a target task, the quality and efficiency of target task processing can be guaranteed, while simultaneously lowering system deployment and operation maintenance costs and providing a convenient and efficient task processing service to the user.
[0221] Referring to FIG. 8, FIG. 8 illustrates a schematic diagram of the structure of a model learning platform provided in one embodiment of the present disclosure, wherein the model learning platform includes a request interface unit (802) and a model unit (804);
[0222] The request interface unit (802) is used to receive a model training request containing request information;
[0223] The model unit (804) is used to obtain a code processing model and to train the code processing model based on request information to obtain a completed code processing model, wherein the request information includes a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results, and the sample processing results and sample code block data are obtained based on hierarchical sampling of the sample code data.
[0224] It should be noted that the processing method of the request interface unit and the model unit is identical to the learning method of the code processing model described above, and therefore is not further described redundantly in the embodiments of the present disclosure.
[0225] In one optional embodiment of the present disclosure, the model learning platform further comprises a model library;
[0226] Model units are further used to save trained code processing models to the model library.
[0227] It should be noted that the model unit may train a code processing model based on request information to obtain a completed code processing model, and then store the completed code processing model in a model library. Consequently, since the completed code processing model can be subsequently obtained directly from the model library, redundant training is not required. Here, the method of obtaining a completed code processing model from the model library may refer to an implementation form in which "a target scenario template is determined from a plurality of pre-configured scenario templates based on the task scenario identifier, and a code processing model is searched from the model library based on the target scenario template—where a plurality of processing models are stored in the model library—; or, a code processing model is searched from the model library based on the task model identifier," and thus is not described redundantly in the embodiments of this disclosure.
[0228] Corresponding to the embodiment of the task processing method above, the present disclosure further provides an embodiment of a task processing device, and FIG. 9 illustrates a schematic diagram of the structure of a task processing device provided in one embodiment of the present disclosure. As shown in FIG. 9, the device comprises,
[0229] A first acquisition module (902) configured to acquire processing target code data of a target task;
[0230] A first input module (904) configured to input code data to be processed into a code processing model to obtain a code processing result of a target task - the code processing model is obtained by learning based on a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results, and the sample processing results and sample code block data are obtained based on hierarchical sampling of the sample code data -; is included.
[0231] Optionally, the first input module (904) is further configured to obtain processing prompt information of a code processing model; and to input the processing prompt information and the code data to be processed into the code processing model to obtain the code processing result of the target task.
[0232] Optionally, the device further includes a first transmission module configured to transmit a code processing result to a frontend user and receive result feedback information transmitted from the frontend user, wherein the result feedback information is information that provides feedback on the code processing result based on task information of a target task; construct model optimization data based on the result feedback information; and perform parameter adjustment for a code processing model using the model optimization data.
[0233] Optionally, the first transmission module generates optimization prompt information based on result feedback information—the optimization prompt information is used to guide the frontend user to transmit model optimization data for optimizing the code processing model—; and is further configured to transmit the optimization prompt information to the frontend user and receive the model optimization data transmitted by the frontend user based on the optimization prompt information.
[0234] Optionally, the device further includes an annotation module configured to annotate key information among the code processing results to obtain an updated code processing result and to transmit the updated code processing result to a frontend user.
[0235] Optionally, the device further comprises a second transmission module configured to transmit a code processing result to a frontend user and receive modification information transmitted from the frontend user—the modification information being used to modify the code processing result—; and to modify the code processing result based on the modification information to obtain a modified code processing result.
[0236] Optionally, the device further comprises a fifth acquisition module configured to acquire a sample set—the sample set includes a plurality of sample code data—; perform hierarchical sampling on the plurality of sample code data to acquire a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results, input the plurality of sample code block data into a code processing model to acquire a prediction processing result corresponding to each of the plurality of sample code block data, and train the code processing model based on the sample processing results and the prediction processing result to acquire a trained code processing model.
[0237] Optionally, the fifth acquisition module is further configured to perform parsing on the first sample code data to acquire a plurality of candidate code block data, wherein the first sample code data is any one of the plurality of sample code data; and to perform hierarchical sampling on the plurality of candidate code block data to acquire a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results.
[0238] Optionally, the fifth acquisition module is further configured to acquire a target code processing topic by sampling from a code processing topic, wherein the code processing topic includes at least one of a code block processing topic, a code block content processing topic, a code block partial content processing topic, and a code block empty content processing topic; and, under the target code processing topic, to acquire a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results by sampling from a plurality of candidate code block data according to a code sampling strategy.
[0239] Optionally, the code sampling strategy includes a result sampling strategy and a code block sampling strategy, the result sampling strategy includes at least one of a category sampling strategy, a preceding context category sampling strategy, a succeeding context category sampling strategy, a child node sampling strategy, and a completeness sampling strategy, and the code block sampling strategy includes at least one of a preceding context content sampling strategy and a succeeding context content sampling strategy, and the fifth acquisition module is further configured to obtain a plurality of sample processing results by sampling from a plurality of candidate code block data according to the result sampling strategy under the target code processing topic, and to obtain sample code block data corresponding to each of the plurality of sample processing results by sampling from a plurality of candidate code block data according to the code block sampling strategy.
[0240] By applying the method according to the embodiment of the present disclosure, the code processing model is obtained by learning based on sample code block data, thereby implementing prediction in units of code blocks and shortening the prediction process. Furthermore, through hierarchical sampling, the code processing model can be enabled to adapt to various code processing scenarios, thereby precisely predicting various types of code blocks and improving task processing efficiency and accuracy.
[0241] The foregoing is an exemplary method of a task processing device according to the present embodiment. It should be noted that the technical method of the task processing device described above belongs to the same inventive concept as the technical method of the task processing method described above, and that all details not described in detail in the technical method of the task processing device may be referred to in the description of the technical method of the task processing method described above.
[0242] Corresponding to the embodiment of the code completion method described above, the present disclosure further provides an embodiment of a code completion device, and FIG. 10 illustrates a schematic diagram of the structure of a code completion device provided in one embodiment of the present disclosure. As shown in FIG. 10, the device comprises:
[0243] A second acquisition module (1002) configured to acquire target code data for a code completion task;
[0244] It includes a second input module (1004) configured to input code data to be completed into a code completion model to obtain a code completion result of a code completion task, wherein the code completion model is obtained by learning based on a plurality of sample completion results and sample code block data corresponding to each of the plurality of sample completion results, and the sample completion results and sample code block data are obtained based on hierarchical sampling of the sample code data.
[0245] Optionally, the device further comprises a second receiving module configured to transmit completion prompt information to a frontend user—the completion prompt information is used to guide the frontend user to transmit code data to be completed for processing a code completion task—and to receive code data to be completed for a code completion task transmitted by the frontend user based on the completion prompt information.
[0246] Optionally, the device further includes a third receiving module configured to transmit a code completion result to a frontend user and receive result feedback information transmitted from the frontend user—the result feedback information is information that provides feedback on the code completion result based on task information of the code completion task—; transmit the result feedback information to a model learning platform—the model learning platform is used to perform parameter adjustments on a code completion model using the result feedback information.
[0247] By applying the method according to the embodiment of the present disclosure, regarding the capability that a code completion model must satisfy, model training data is constructed through a code syntax analysis and rich scenario-based hierarchical sampling method, and a code completion model is obtained by training using the model training data. This enables the code completion model to accurately predict various types of code blocks (which may be classes, functions, logical blocks, comments, syntax, etc.) that the user intends to input next, based on the code preceding context, code succeeding context, and some code metadata written by the user in various scenarios of actual development, thereby improving the user's coding efficiency and experience.
[0248] The foregoing is an exemplary method of a code completion device according to the present embodiment. It should be noted that the technical method of the code completion device described above belongs to the same inventive concept as the technical method of the code completion method described above, and that all details not described in detail in the technical method of the code completion device may be referred to in the description of the technical method of the code completion method described above.
[0249] Corresponding to the embodiment of the code processing model learning method described above, the present disclosure further provides an embodiment of a code processing model learning apparatus, and FIG. 11 illustrates a schematic diagram of the structure of a code processing model learning apparatus provided in one embodiment of the present disclosure. As shown in FIG. 11, the apparatus comprises,
[0250] A third acquisition module (1102) configured to acquire a sample set containing multiple sample code data;
[0251] A sampling module (1104) configured to perform hierarchical sampling on multiple sample code data to obtain multiple sample processing results and sample code block data corresponding to each of the multiple sample processing results;
[0252] A third input module (1106) configured to input multiple sample code block data into a code processing model to obtain a prediction processing result corresponding to each of the multiple sample code block data;
[0253] It includes a learning module (1108) configured to train a code processing model based on sample processing results and prediction processing results to obtain a trained code processing model.
[0254] By applying the method according to the embodiment of the present disclosure, a code processing model is trained based on sample processing results and prediction processing results, and the code processing model is continued to be trained when a preset stopping condition is not satisfied, and the training is completed by performing the process until the preset stopping condition is reached, thereby obtaining a code processing model. By continuously adjusting the model parameters of the code processing model, the finally obtained code processing model can be made more precise.
[0255] The foregoing is an exemplary method for a code processing model learning device according to the present embodiment. The technical method of the code processing model learning device described above belongs to the same inventive concept as the technical method of the code processing model learning method described above, and it should be noted that for all details not described in detail in the technical method of the code processing model learning device described above, one may refer to the description regarding the technical method of the code processing model learning method described above.
[0256] Corresponding to an embodiment of the method described above, the present disclosure further provides an embodiment of an information processing device based on a code processing model, and FIG. 12 illustrates a schematic diagram of the structure of an information processing device based on a code processing model provided in one embodiment of the present disclosure. As shown in FIG. 12, the device comprises:
[0257] A first receiving module (1202) configured to receive a task creation request including request information;
[0258] A fourth acquisition module (1204) configured to acquire a code processing model based on request information - the code processing model is acquired by learning based on a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results, and the sample processing results and sample code block data are acquired based on hierarchical sampling of the sample code data -;
[0259] Based on a code processing model, it includes a generation module (1206) configured to generate task information used to execute a target task.
[0260] Optionally, the request information includes a task scenario identifier or a task model identifier of the target task, and the fourth acquisition module (1204) determines a target scenario template among a plurality of preset scenario templates based on the task scenario identifier, and searches for a code processing model in a model library based on the target scenario template—where a plurality of processing models are stored in the model library—; or is further configured to search for a code processing model in a model library based on the task model identifier.
[0261] Optionally, the request information includes multiple sample code data of the target task, and the fourth acquisition module (1204) is further configured to acquire a code processing model that has been trained by learning a code processing model corresponding to the target task based on the multiple sample code data.
[0262] By applying the method according to the embodiment of the present disclosure, by generating task information for a target task, the quality and efficiency of target task processing can be guaranteed, while simultaneously lowering system deployment and operation maintenance costs and providing a convenient and efficient task processing service to the user.
[0263] The foregoing is an exemplary method for an information processing device based on a code processing model according to the present embodiment. The technical method of the information processing device based on the code processing model belongs to the same inventive concept as the technical method of the information processing method based on the code processing model described above. It should be noted that for all details not described in detail in the technical method of the information processing device based on the code processing model, one may refer to the description of the technical method of the information processing method based on the code processing model described above.
[0264] FIG. 13 illustrates a structural block diagram of a computing device provided in one embodiment of the present disclosure. The components of the computing device (1300) include, but are not limited to, a memory (1310) and a processor (1320). The processor (1320) is connected to the memory (1310) via a bus (1330), and a database (1350) is used to store data.
[0265] The computing device (1300) further includes an access device (1340), and the access device (1340) enables the computing device (1300) to communicate through one or more networks (1360). Examples of these networks include a combination of communication networks such as a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or the Internet. The access device (1340) may include one or more of any type of wired or wireless network interface (e.g., Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Networks (WLAN) wireless interface, a World Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, etc.
[0266] In one embodiment of the present disclosure, the components of the computing device (1300) and other components not shown in FIG. 13 may also be connected to each other, for example, via a bus. It should be understood that the structural block diagram of the computing device shown in FIG. 13 is for illustrative purposes only and is not intended to limit the scope of the present disclosure. Those skilled in the art may add or replace other components as needed.
[0267] The computing device (1300) may be any type of fixed or mobile computing device and includes mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or fixed computing devices such as desktop computers or personal computers (PCs). The computing device (1300) may also be a mobile or fixed server.
[0268] Here, the processor (1320) is used to execute a computer program / instruction, and when the computer program / instruction is executed by the processor, the steps of the task processing method, or code completion method, or code processing model learning method, or code processing model-based information processing method are implemented.
[0269] The foregoing is an exemplary method for a computing device according to the present embodiment. The technical method of the computing device belongs to the same inventive concept as the technical methods of the task processing method, code completion method, code processing model learning method, and code processing model-based information processing method described above. It should be noted that for any details not described in detail in the technical method of the computing device, one may refer to the descriptions regarding the technical methods of the task processing method, code completion method, code processing model learning method, or code processing model-based information processing method described above.
[0270] One embodiment of the present disclosure further provides a computer-readable storage medium in which a computer program / instruction is stored, and implements the steps of the task processing method, code completion method, code processing model learning method, or code processing model-based information processing method when the computer program / instruction is executed by a processor.
[0271] The foregoing is an exemplary method for a computer-readable storage medium according to the present embodiment. The technical method of the storage medium belongs to the same inventive concept as the technical methods of the task processing method, code completion method, code processing model learning method, and code processing model-based information processing method described above. It should be noted that for any details not described in detail in the technical method of the storage medium, one may refer to the descriptions regarding the technical methods of the task processing method, code completion method, code processing model learning method, or code processing model-based information processing method described above.
[0272] One embodiment of the present disclosure further provides a computer program product comprising a computer program / instruction, and implements the steps of a task processing method, a code completion method, a code processing model learning method, or a code processing model-based information processing method when the computer program / instruction is executed by a processor.
[0273] The foregoing is an exemplary method for a computer program product according to the present embodiment. The technical method of the computer program product belongs to the same inventive concept as the technical methods of the task processing method, code completion method, code processing model learning method, and code processing model-based information processing method described above. It should be noted that for any details not described in detail in the technical method of the computer program product, one may refer to the descriptions regarding the technical methods of the task processing method, code completion method, code processing model learning method, or code processing model-based information processing method described above.
[0274] Specific embodiments of the present disclosure have been described above. Other embodiments are within the scope of the appended claims. In some cases, the operations or steps described in the claims may be performed in a different order than in the embodiments and may still achieve the expected results. Furthermore, the processes illustrated in the drawings do not necessarily require the specific order or sequential order illustrated to achieve the expected results. In some embodiments, multitasking and parallel processing may also be possible or advantageous.
[0275] The above computer instruction includes computer program code, and the computer program code may be in source code format, object code format, executable file, or some intermediate format. The above computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB memory, a removable hard disk, a magnetic disk, an optical disk, computer memory, Read-Only Memory (ROM), Random Access Memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. The contents included in the above computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice, and it should be noted that, for example, in some regions according to patent practice, the computer-readable medium does not include electrical carrier signals and telecommunications signals.
[0276] For each of the aforementioned method embodiments, they have all been expressed as a combination of a series of operations for convenience of explanation; however, it should be noted that those skilled in the art will understand that the embodiments of the present disclosure are not limited by the described sequence of operations, as some steps may be performed in a different order or simultaneously according to the embodiments of the present disclosure. Furthermore, those skilled in the art should understand that all embodiments described in the specification belong to preferred embodiments, and that the related operations and modules are not necessarily all essential to the embodiments of the present disclosure.
[0277] In the above embodiments, the descriptions for each embodiment focus on different aspects, and any parts not described in detail in any embodiment may be referred to in the relevant descriptions of other embodiments.
[0278] The preferred embodiments of the present disclosure disclosed above are intended only to aid in explaining the present disclosure. The optional embodiments do not describe all details in detail, nor is the invention limited to the specific embodiments described above. Clearly, many modifications and changes may be made to the contents of the embodiments of the present disclosure. The selection and specific description of these embodiments in the present disclosure is intended to better interpret the principles and practical applications of the embodiments of the present disclosure, thereby enabling those skilled in the art to understand and use the present disclosure. The present disclosure is limited only by the claims and their entire scope and equivalents.< / filename>
Claims
Claim 1 A task processing method comprising: a step of obtaining code data to be processed for a target task; a step of inputting the code data to be processed into a code processing model to obtain a code processing result of the target task, wherein the code processing model is obtained by learning based on a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results, and wherein the sample processing results and the sample code block data are obtained based on hierarchical sampling of the sample code data. Claim 2 A method according to claim 1, wherein the step of inputting the code data to be processed into a code processing model to obtain the code processing result of the target task comprises: the step of obtaining processing prompt information of the code processing model; and the step of inputting the processing prompt information and the code data to be processed into the code processing model to obtain the code processing result of the target task. Claim 3 A method according to claim 1 or 2, further comprising: a step of inputting the code data to be processed into a code processing model to obtain the code processing result of the target task, and then, a step of transmitting the code processing result to a frontend user; a step of receiving result feedback information transmitted from the frontend user, wherein the result feedback information is information that provides feedback on the code processing result based on the task information of the target task; a step of constructing model optimization data based on the result feedback information; and a step of performing parameter adjustment on the code processing model using the model optimization data. Claim 4 In paragraph 3, the step of constructing model optimization data based on the result feedback information comprises: a step of generating optimization prompt information based on the result feedback information—the optimization prompt information is used to guide the frontend user to transmit model optimization data for optimizing the code processing model—; and a step of transmitting the optimization prompt information to the frontend user and receiving the model optimization data transmitted by the frontend user based on the optimization prompt information. Claim 5 A method according to any one of claims 1 to 4, further comprising: a step of obtaining a code processing result of the target task by inputting the processing target code data into a code processing model, a step of obtaining an updated code processing result by annotating key information among the code processing results; and a step of transmitting the updated code processing result to a front-end user. Claim 6 A method according to any one of claims 1 to 5, further comprising: a step of transmitting the code processing result to a frontend user after the step of inputting the processing target code data into a code processing model to obtain the code processing result of the target task; a step of receiving modification information transmitted from the frontend user - the modification information is used to modify the code processing result -; and a step of modifying the code processing result according to the modification information to obtain a modified code processing result. Claim 7 A method according to any one of claims 1 to 6, further comprising, prior to the step of inputting the code data to be processed into a code processing model to obtain the code processing result of the target task, a step of obtaining a sample set including a plurality of sample code data; a step of performing hierarchical sampling on the plurality of sample code data to obtain a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results; a step of inputting the plurality of sample code block data into a code processing model to obtain a prediction processing result corresponding to each of the plurality of sample code block data; and a step of training the code processing model based on the sample processing result and the prediction processing result to obtain a trained code processing model. Claim 8 In claim 7, the step of obtaining a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results by performing hierarchical sampling on the plurality of sample code data comprises: a step of obtaining a plurality of candidate code block data by performing syntactic analysis on the first sample code data for the first sample code data, wherein the first sample code data is any one of the plurality of sample code data; and a step of obtaining a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results by performing hierarchical sampling on the plurality of candidate code block data. Claim 9 In claim 8, the step of performing hierarchical sampling on the plurality of candidate code block data to obtain a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results comprises: a step of obtaining a target code processing topic by sampling from a code processing topic - wherein the code processing topic includes at least one of a code block processing topic, a code block content processing topic, a code block partial content processing topic, and a code block empty content processing topic -; and a step of obtaining a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results by sampling from the plurality of candidate code block data according to a code sampling strategy under the target code processing topic. Claim 10 In claim 9, the code sampling strategy comprises a result sampling strategy and a code block sampling strategy, wherein the result sampling strategy comprises at least one of a category sampling strategy, a preceding context category sampling strategy, a succeeding context category sampling strategy, a child node sampling strategy and a completeness sampling strategy, and the code block sampling strategy comprises at least one of a preceding context content sampling strategy and a succeeding context content sampling strategy; and, under the target code processing topic, the step of obtaining a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results by sampling from the plurality of candidate code block data according to the code sampling strategy comprises, under the target code processing topic, obtaining a plurality of sample processing results by sampling from the plurality of candidate code block data according to the result sampling strategy, and obtaining sample code block data corresponding to each of the plurality of sample processing results by sampling from the plurality of candidate code block data according to the code block sampling strategy. Claim 11 A code completion method comprising: a step of obtaining code data to be completed for a code completion task; a step of inputting the code data to be completed for completion into a code completion model to obtain a code completion result of the code completion task, wherein the code completion model is obtained by learning based on a plurality of sample completion results and sample code block data corresponding to each of the plurality of sample completion results, and wherein the sample completion results and the sample code block data are obtained based on hierarchical sampling of the sample code data. Claim 12 A method according to claim 11, further comprising, prior to the step of obtaining the code data to be completed of the code completion task, the step of transmitting completion prompt information to a frontend user—the completion prompt information is used to guide the frontend user to transmit the code data to be completed for processing the code completion task—; and the step of receiving the code data to be completed of the code completion task transmitted by the frontend user based on the completion prompt information. Claim 13 A method according to claim 11 or 12, further comprising: a step of transmitting the code completion result to a front-end user after the step of inputting the code data to be completed into a code completion model to obtain the code completion result of the code completion task; a step of receiving result feedback information transmitted from the front-end user, wherein the result feedback information is information that provides feedback on the code completion result based on the task information of the code completion task; a step of transmitting the result feedback information to a model learning platform, wherein the model learning platform is used to perform parameter adjustment for the code completion model using the result feedback information. Claim 14 A method for training a code processing model, comprising: a step of obtaining a sample set including a plurality of sample code data; a step of performing hierarchical sampling on the plurality of sample code data to obtain a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results; a step of inputting the plurality of sample code block data into a code processing model to obtain a prediction processing result corresponding to each of the plurality of sample code block data; and a step of training the code processing model based on the sample processing results and the prediction processing results to obtain a trained code processing model. Claim 15 A method for processing information based on a code processing model, comprising: receiving a task creation request including request information; obtaining a code processing model based on the request information, wherein the code processing model is obtained by learning based on a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results, and wherein the sample processing results and the sample code block data are obtained based on hierarchical sampling of the sample code data; and generating task information based on the code processing model, wherein the task information is used to execute a target task. Claim 16 In claim 15, the request information includes a task scenario identifier or a task model identifier of a target task, and the step of obtaining a code processing model based on the request information comprises: determining a target scenario template from a plurality of preset scenario templates based on the task scenario identifier, and searching for a code processing model in a model library based on the target scenario template - wherein a plurality of processing models are stored in the model library -; or, a step of searching for a code processing model in the model library based on the task model identifier. Claim 17 In claim 15, the above request information includes a plurality of sample code data of a target task; and the step of obtaining a code processing model based on the above request information includes the step of training a code processing model corresponding to the target task based on the plurality of sample code data and obtaining a trained code processing model. Claim 18 A model learning platform comprising a request interface unit and a model unit, wherein the request interface unit is used to receive a model learning request including request information, the model unit is used to obtain a code processing model and to train the code processing model based on the request information to obtain a completed code processing model, the request information includes a plurality of sample processing results and sample code block data corresponding to each of the plurality of sample processing results, and the sample processing results and the sample code block data are obtained based on hierarchical sampling of the sample code data. Claim 19 In paragraph 18, a model learning platform further comprising a model library, wherein the model unit is further used to store the trained code processing model in the model library. Claim 20 A computing device comprising a memory and a processor, wherein the memory is used to store a computer program / instruction and the processor is used to execute the computer program / instruction, and when the computer program / instruction is executed by the processor, the step of a method according to any one of claims 1 to 10, or any one of claims 11 to 13, or claim 14, or any one of claims 15 to 17 is implemented. Claim 21 A computer-readable storage medium having a computer program / instruction stored therein, and implementing a step of a method according to any one of claims 1 to 10, or any one of claims 11 to 13, or claim 14, or any one of claims 15 to 17 when said computer program / instruction is executed by a processor. Claim 22 A computer program product comprising a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the step of a method according to any one of claims 1 to 10, or any one of claims 11 to 13, or claim 14, or any one of claims 15 to 17 is implemented.
