Code processing model training method and device, electronic equipment and storage medium

Abstract

The invention provides a code processing model training method and device, electronic equipment and a storage medium. The method comprises the steps of obtaining a sample instruction set; calling a pre-trained first code processing model based on the sample instruction set to perform code generation processing, and obtaining a prediction code text corresponding to each sample instruction; forming a plurality of sample pairs by the plurality of sample instructions and the prediction code texts respectively corresponding to the plurality of sample instructions; calling a first code processing model to determine a quality index corresponding to each sample pair, and labeling each sample pair according to the quality index corresponding to each sample pair; and training a first code processing model based on the plurality of labeled sample pairs to obtain a second code processing model. Through the method, the code generation accuracy of the code processing model can be improved.

Description

Technical Field

[0001] This application relates to artificial intelligence technology, and more particularly to a method, apparatus, electronic device, and storage medium for training a code processing model. Background Technology

[0002] Natural Language Processing (NLP) is an important field within computer science and artificial intelligence. It studies the theories and methods for enabling effective communication between humans and computers using natural language. NLP deals with natural language—the language people use in daily life—and is closely related to linguistics research; it also involves crucial model training techniques in computer science, mathematics, and artificial intelligence. Pre-trained models are developed from Large Language Models (LLMs) in NLP. After fine-tuning, large language models can be widely applied to downstream tasks. NLP techniques typically include text processing, semantic understanding, machine translation, question answering, and knowledge graphs.

[0003] In related technologies, code generation processing based on large language models relies on a large amount of manually labeled data, which results in a large amount of computational resources being consumed in training the code processing model. However, the acquisition of manually labeled data is limited and it is difficult to cover all programming languages ​​and scenarios, resulting in low generalization of the code processing model, that is, low accuracy in code generation processing for different application scenarios.

[0004] Currently, there is no good solution to improve the accuracy of code generated by the code processing model. Summary of the Invention

[0005] This application provides a training method, apparatus, electronic device, and storage medium for a code processing model, which can improve the accuracy of code generation by the code processing model.

[0006] The technical solution of this application embodiment is implemented as follows:

[0007] This application provides a method for training a code processing model, the method comprising:

[0008] Obtain a set of sample instructions, wherein the set of sample instructions includes multiple sample instructions;

[0009] Based on the set of sample instructions, a pre-trained first code processing model is invoked to perform code generation processing, thereby obtaining the predicted code text corresponding to each of the sample instructions.

[0010] The plurality of sample instructions and the prediction code text corresponding to the plurality of sample instructions are combined into a plurality of sample pairs, wherein each sample pair includes a sample instruction and the prediction code text corresponding to the sample instruction;

[0011] The first code processing model is invoked to determine the quality index corresponding to each sample pair, and each sample pair is labeled according to the quality index corresponding to each sample pair.

[0012] The first code processing model is trained based on the labeled sample pairs to obtain the second code processing model.

[0013] This application provides a training apparatus for a code processing model, comprising:

[0014] A data acquisition module is used to acquire a sample instruction set, wherein the sample instruction set includes multiple sample instructions;

[0015] The code generation module is used to call a pre-trained first code processing model to perform code generation processing based on the sample instruction set, so as to obtain the predicted code text corresponding to each sample instruction.

[0016] The data acquisition module is further configured to form multiple sample pairs by combining the plurality of sample instructions and the prediction code text corresponding to the plurality of sample instructions respectively, wherein each sample pair includes a sample instruction and the prediction code text corresponding to the sample instruction;

[0017] The code evaluation module is used to call the first code processing model to determine the quality index corresponding to each sample pair, and to label each sample pair according to the quality index corresponding to each sample pair.

[0018] The model training module is used to train the first code processing model based on the labeled sample pairs to obtain the second code processing model.

[0019] This application provides an electronic device, the electronic device comprising:

[0020] Memory is used to store executable instructions or computer programs.

[0021] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the training method for the code processing model provided in the embodiments of this application.

[0022] This application provides a computer-readable storage medium storing computer-executable instructions or computer programs, which, when executed by a processor, implement the training method of the code processing model provided in this application.

[0023] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the training method of the code processing model provided in this application.

[0024] The embodiments of this application have the following beneficial effects:

[0025] The first code processing model itself is invoked to evaluate the quality of the generated predicted code text, and the quality evaluation results are used as labels to annotate each sample pair. The first code processing model is used as its own reward model. Compared with the method of manually annotating training data in related technologies, this saves the computing resources and time consumed in the training process of the code processing model. The first code processing model is trained based on the sample pairs labeled with quality indicators, so that the first code processing model has the function of determining the better code among multiple predicted codes, realizing the self-optimization of the model and improving the accuracy of the code generated by the trained second code processing model. Attached Figure Description

[0026] Figure 1 This is a schematic diagram illustrating the application mode of the training method for the code processing model provided in the embodiments of this application;

[0027] Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0028] Figure 3A This is a first flowchart illustrating the training method of the code processing model provided in this application embodiment;

[0029] Figure 3B This is a second flowchart illustrating the training method for the code processing model provided in this application embodiment;

[0030] Figure 3C This is a schematic diagram of the third process of the training method for the code processing model provided in the embodiments of this application;

[0031] Figure 3D This is a schematic diagram of the fourth process of the training method for the code processing model provided in the embodiments of this application;

[0032] Figure 3E This is a schematic diagram of the fifth step of the training method for the code processing model provided in the embodiments of this application;

[0033] Figure 4 This is a schematic diagram of the sixth process of the training method for the code processing model provided in the embodiments of this application;

[0034] Figure 5This is a schematic diagram illustrating the principle of the training method for the code processing model provided in the embodiments of this application;

[0035] Figure 6 This is a schematic diagram of the code processing model provided in the embodiments of this application;

[0036] Figure 7 This is a schematic diagram of the output content of the code processing model provided in the embodiments of this application.

[0037] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0039] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0040] If the application documents contain similar descriptions such as "first / second", the following explanation shall be added: In the following description, the terms "first / second / third" are used only to distinguish similar objects and do not represent a specific order of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0041] It should be noted that the data collection and processing in this application (e.g., user input instructions, sample instructions used as training data) should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0042] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0043] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0044] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0045] 1) Large Language Model (LLM): This is a natural language processing technique based on artificial neural networks. It can automatically learn and understand the syntax, semantics, and context of natural language to generate relevant text. Large language models are typically based on deep learning models, such as recurrent neural networks (RNNs) or transformer models, and their performance is improved through training on large-scale corpora. These models usually consist of millions or billions of parameters and are capable of generating highly coherent and natural text.

[0046] 2) Code Large Language Model (Code LLM): A large-scale language model specifically designed to understand and generate programming language code. By training on a large dataset of code, Code Large Language Model learns the syntax, structure, and common patterns of programming languages, thereby assisting developers in writing, understanding, and optimizing code. Code Large Language Models can be applied to functions such as code completion, error detection, code review, and automatic code generation, aiming to improve the efficiency and quality of software development.

[0047] 3) Direct Preference Optimization (DPO): A method that learns directly from human preferences for training large language models. DPO optimizes the model by comparing preferences across different outputs without requiring an explicit reward function.

[0048] 4) Artificial Intelligence Feedback (AIF): This uses data generated by an artificial intelligence system to provide processing of training signals. The data generated by the artificial intelligence system can be examples generated by the model, evaluations, or other forms of feedback, used to guide the model's learning process.

[0049] 5) Reward Signal: A scalar value used to represent the quality or success of an action performed by the artificial intelligence (AI) during its interaction with the environment. This reward signal is provided by the external environment or system as a feedback mechanism for AI learning. In this embodiment, the code processing model can serve as its own reward model. The reward signal is typically a numerical value, which can be positive, negative, or zero, used to indicate the potential impact of the AI ​​on the environment after performing an action. A positive reward indicates that the action is beneficial or aligns with the AI's goals, a negative reward indicates that the action is detrimental or results in a loss of benefits, and a zero reward indicates that the action is neutral.

[0050] 6) Supervised Fine-Tuning (SFT): This is a machine learning technique used to improve the performance of a pre-trained model by using labeled data for a specific task for additional training.

[0051] 7) Artificial Intelligence Feedback Training (AIFT): Training on self-generated data examples to improve the model's generation and evaluation capabilities. AIFT allows the model to optimize its performance through self-evaluation without external rewards.

[0052] 8) Instructions: Text that commands the artificial intelligence to perform a certain task, used as input for the model.

[0053] 9) Quality Indicators: These are aggregate indicators reflecting production efficiency or work quality. Their numerical values ​​represent the qualitative attributes of a matter. In this embodiment, the quality indicators for code processing are used to characterize parameters such as the degree of matching and standardization between the code generated by the code generation process and the instructions used to generate the code.

[0054] In related technologies, code processing models rely on a large amount of manually labeled data, resulting in high computational resource consumption during training and difficulty in covering all programming languages ​​and scenarios. Furthermore, code processing models often lack code quality checks during code generation, leading to potential issues such as inefficiency, poor readability, and bugs.

[0055] This application provides a training method for a code processing model, a training device for a code processing model, an electronic device, a computer-readable storage medium, and a computer program product, which can improve the accuracy of code generated by the code processing model.

[0056] The following describes exemplary applications of the electronic devices provided in the embodiments of this application. These electronic devices can be implemented as terminal devices, such as laptops, tablets, desktop computers, set-top boxes, smart TVs, in-vehicle terminals, virtual reality (VR) devices, augmented reality (AR) devices, and other various types of terminals. They can also be implemented as servers. The following will describe exemplary applications when the electronic device is implemented as a terminal device or a server.

[0057] refer to Figure 1 , Figure 1 This is a schematic diagram illustrating the application mode of the training method for the code processing model provided in the embodiments of this application; for example, Figure 1 The system involves server 200, network 300, terminal device 400, and database 500. Terminal device 400 is connected to server 200 through network 300, which can be a wide area network (WAN), a local area network (LAN), or a combination of both.

[0058] For example, server 200 can be a server of a code processing platform, terminal device 400 can be a user's computer or mobile phone, and database 500 stores a large amount of training data, including: corpus data, code data, and text of code generation instructions. The user can be a developer.

[0059] In some embodiments, a user inputs a training request through a terminal device 400. The terminal device 400 sends the training request to a server 200 via a network 300. The server 200 retrieves training data from a database 500. The training data includes a set of sample instructions. Based on the set of sample instructions and a pre-trained code processing model, the server 200 calls the training method of the code processing model provided in this embodiment to iteratively train the code processing model, obtaining a trained code processing model. The user sends the instruction text to be processed to the server 200 through the terminal device 400. The server 200 calls the trained code processing model based on the instruction text to generate the text of the target code required by the user and sends the text of the target code back to the terminal device 400. The user can modify or use the received text of the target code.

[0060] In some embodiments, the training method of the code processing model of this application can also be applied in the following application scenarios: 1. Code quality evaluation, for example: during the development of a new application, the code processing model trained by the training method of the code processing model of this application can perform quality detection on multiple code texts to determine the optimal code text, thereby improving the efficiency of developers in developing new applications; 2. Code completion and error detection, for example: when a user writes a new code text, the code processing model trained by the training method of the code processing model of this application can perform error detection on the written code text, determine the defects in the code text, and complete the defective parts based on the entire code text and the user's input instructions to obtain optimized code text, thereby improving the efficiency of users writing code and the user experience.

[0061] This application embodiment can be implemented using database technology. A database, simply put, can be viewed as an electronic filing cabinet storing electronic files, where users can perform operations such as adding, querying, updating, and deleting data. A "database" is a collection of data stored together in a certain way, capable of being shared by multiple users, having minimal redundancy, and being independent of application programs.

[0062] A Database Management System (DBMS) is a computer software system designed to manage databases, generally possessing basic functions such as storage, retrieval, security, and backup. DBMSs can be classified according to the database model they support, such as relational or XML (Extensible Markup Language); or according to the type of computer they support, such as server clusters or mobile devices; or according to the query language used, such as Structured Query Language (SQL) or XQuery; or according to performance priorities, such as maximum scale or maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMSs can cross categories, for example, simultaneously supporting multiple query languages.

[0063] The embodiments of this application can be implemented using artificial intelligence (AI) technology. AI is the theory, methods, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can react in a way similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0064] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, pre-trained model technology, operating / interactive systems, and mechatronics. Among these, pre-trained models, also known as large-scale models or foundational models, can be widely applied to downstream tasks across various AI fields after fine-tuning. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0065] This application embodiment can also be implemented using cloud technology. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology based on cloud computing business models. It can form a resource pool, available on demand, offering flexibility and convenience. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, and driven by demands for search services, social networks, mobile commerce, and open collaboration, every item may eventually possess its own hash-coded identification mark, requiring transmission to a backend system for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0066] In some embodiments, the server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Electronic devices can be smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, etc., but are not limited to these. Terminal devices and servers can be directly or indirectly connected via wired or wireless communication, which is not limited in this embodiment.

[0067] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. The electronic device may be a server 200. Figure 2 The server 200 shown includes at least one processor 410, memory 450, and at least one network interface 420. The various components of server 200 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 440.

[0068] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0069] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0070] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0071] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0072] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0073] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0074] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 A training device 455 for a code processing model stored in memory 450 is shown. It can be software in the form of programs and plug-ins, including the following software modules: data acquisition module 4551, code generation module 4552, code evaluation module 4553, and model training module 4554. These modules are logically related and can be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0075] In some embodiments, the terminal device or server can implement the training method of the code processing model provided in this application by running a computer program. For example, the computer executable instructions can be microprogram-level commands, machine instructions, or software instructions. The computer program can be a native program or software module in the operating system; it can be a native application (APP), that is, a program that needs to be installed in the operating system to run, such as a programming APP or an instant messaging APP; or it can be a small program that can be embedded in any APP, that is, a program that only needs to be downloaded to the browser environment to run. In summary, the above-mentioned computer executable instructions can be any form of instructions, and the above-mentioned computer program can be any form of application, module, or plugin.

[0076] The training method of the code processing model provided in this application will be described in conjunction with exemplary applications and implementations of the terminal devices provided in the embodiments of this application.

[0077] The following describes the training method of the code processing model provided in the embodiments of this application. As mentioned above, the electronic device implementing the training method of the code processing model in the embodiments of this application can be a terminal device or a server, or a combination of both. Therefore, the executing entity of each step will not be described again below.

[0078] It should be noted that the code processing examples below are illustrated using the example of generating code text based on instruction text. Those skilled in the art, based on their understanding of the following text, can apply the training method of the code processing model provided in the embodiments of this application to the processing of generating code based on other types of input content, such as generating code based on images or generating code based on videos.

[0079] See Figure 3A , Figure 3A This is a flowchart illustrating the training method for the code processing model provided in this application embodiment, which will be combined with... Figure 3A The steps shown are explained. Figure 3A The entity responsible for executing the steps is Figure 1 Server 200.

[0080] In step 301, the sample instruction set is obtained.

[0081] Here, the sample instruction set includes multiple sample instructions.

[0082] For example, sample instructions are used to invoke a code processing model to perform code generation processing. In this embodiment, the sample instructions are in text form. The content of a sample instruction is, for example, "Instruction: Please provide me with several pieces of Python code for timing." The sample instructions in the sample instruction set can be obtained from an existing database or generated by a large language model (e.g., a converter model).

[0083] In some embodiments, the sample instruction set is used to train a first code processing model, which includes a transformer model (feature extraction layer, encoder, and decoder) and a code evaluation model; wherein the transformer model is used to acquire the sample instruction set and perform code generation processing; and the code evaluation model is used to determine quality metrics.

[0084] In this embodiment of the application, the first code processing model can be used to achieve the following functions: code generation, instruction generation, and determining code quality indicators (code evaluation). (See reference...) Figure 6 , Figure 6This is a schematic diagram of the code processing model provided in this application embodiment. The code processing model 600 includes a first converter model 610, a second converter model 620, and a code evaluation model 630. The first converter model 610 is used to generate code based on instruction text, and includes a first feature extraction layer 611, a first encoder 612, and a first decoder 613. The second converter model 620 is used to generate new instruction text based on input content, and includes a second feature extraction layer 621, a second encoder 622, and a second decoder 623. The code evaluation model 630 is used to determine the quality indicators of the code. The code evaluation model 630 can be a recurrent neural network (RNN) or a dynamic code analysis tool (Valgrind). Taking an RNN as an example, the code evaluation model 630 includes an input layer 631, a hidden layer 632, and an output layer 633.

[0085] In some embodiments, step 301 can be implemented by: obtaining at least one first seed instruction, wherein the type of the first seed instruction is a code generation instruction; based on the at least one first seed instruction, calling a pre-trained first code processing model to perform instruction generation processing to obtain multiple sample instructions; and combining the multiple sample instructions into a sample instruction set.

[0086] For example, a seed instruction is a special input value that serves as the starting point for a random number generator algorithm; in this application, it is used as a reference for generating sample instructions. Referring to the above, the first code processing model has the function of generating new instruction text based on the input content. The first seed instruction can be text used to command the model to generate instruction text. For example: "Instruction: You are required to propose a set of 20 different code generation task instructions. The following are the requirements for generating task instructions…". The specific content of the first seed instruction can be manually set according to the application scenario.

[0087] In some embodiments, based on at least one first seed instruction, a pre-trained first code processing model is invoked to perform instruction generation processing to obtain multiple sample instructions. This can be achieved by: based on each first seed instruction, invoking the pre-trained first code processing model to perform the following processing: extracting text features from the seed instruction to obtain first text features; performing text prediction processing based on the first text features to obtain multiple different first character sequences; and decoding the multiple first character sequences to obtain multiple sample instructions corresponding to the seed instruction.

[0088] For example, the converter model in the code processing model performs text prediction based on the first text feature, performs semantic understanding on the seed instruction, generates a character sequence that conforms to the corresponding rules, and decodes the character sequence into text to obtain multiple sample instructions corresponding to the seed instruction. For further examples, please refer to [link / reference]. Figure 6 During the training phase, the first seed instruction is input into the second converter model 620 of the code processing model 600. The second feature extraction layer 621 of the second converter model 620 extracts features from the first seed instruction to obtain the first text features. The second encoder 622 encodes the first text features, and the second decoder 623 predicts and decodes the encoding results to obtain multiple sample instructions corresponding to the seed instruction.

[0089] In some embodiments, prior to step 301, the following processes are performed to fine-tune the initial first code processing model to obtain a pre-trained first code processing model: obtaining a seed instruction set, wherein the seed instruction set includes a first seed instruction, a second seed instruction, and a third seed instruction; calling the initial first code processing model based on the first seed instruction to perform fine-tuning processing corresponding to the instruction generation processing to obtain a first fine-tuned code processing model; calling the first fine-tuned code processing model based on the second seed instruction to perform fine-tuning processing corresponding to the code generation processing to obtain a second fine-tuned code processing model and the predicted code text corresponding to each second seed instruction; and calling the second fine-tuned code processing model based on the third seed instruction and the predicted code text corresponding to the second instructions to perform fine-tuning processing corresponding to the determined quality index to obtain a pre-trained first code processing model.

[0090] For example, the first seed instruction can be called an instruction generation instruction, used to call the model to generate different sample instructions. Based on the generated sample instructions and the first seed instruction, the parameters related to the instruction generation function of the first code processing model can be fine-tuned. The second seed instruction can be called a code generation instruction, used to call the model to generate different predicted codes. Based on the generated predicted codes and the second seed instruction, the parameters related to the code generation function of the first code processing model can be fine-tuned. The second seed instruction can also be called an evaluation instruction, used to call the model to determine the quality index of different predicted codes. The parameters related to the quality index determination function of the first code processing model can be fine-tuned.

[0091] In step 302, the pre-trained first code processing model is invoked based on the sample instruction set to perform code generation processing, thereby obtaining the predicted code text corresponding to each sample instruction.

[0092] For example, code generation processing is the process by which a code processing model understands the text content of sample instructions and generates code text that matches the text content associated with the sample instructions. This can be achieved through the converter model in the first code processing model.

[0093] In some embodiments, reference Figure 3B , Figure 3B This is a second flowchart illustrating the training method for the code processing model provided in this application embodiment; Figure 3A Step 302 can be achieved through Figure 3B Steps 3021 to 3024 are implemented, and the details are explained below.

[0094] In step 3021, based on each sample instruction, the pre-trained first code processing model is invoked to perform the following processing: text feature extraction processing is performed on the sample instruction to obtain the second text feature of the sample instruction.

[0095] For example, the sample instruction is segmented into words to obtain a sequence of multiple sub-words. The sequence of sub-words is then encoded, and each sub-word is replaced with its corresponding index in the vocabulary. The sequence is then reduced in dimensionality to obtain the second text feature represented as a feature vector.

[0096] In step 3022, code prediction processing is performed based on the second text features to obtain the second character sequence.

[0097] Here, the second character sequence includes multiple code characters.

[0098] For example, code prediction processing is a process that predicts the next character in the code based on the characteristics of the input instruction and the already predicted code content. A code character refers to any character that can appear in the code; it can be Chinese, English, or a symbol. See further... Figure 6 During the training phase, the sample instruction set is input into the first converter model 610 of the code processing model 600. The first feature extraction layer 611 of the first converter model 610 extracts features from the first seed instruction to obtain the second text features. The first encoder 612 encodes the second text features, and the first decoder 613 predicts and decodes the encoding results to obtain the second character sequence corresponding to each sample instruction.

[0099] In some embodiments, step 3022 can be implemented as follows: performing character prediction processing based on the second text features to obtain the first code character, and using the first code character as the predicted code character; iteratively performing code prediction processing based on the second text features and the predicted code character to obtain multiple code characters, wherein the input of each code prediction processing includes: the predicted code character obtained from the previous code prediction processing and the second text features, and the code prediction processing stops when the number of iterations reaches a threshold.

[0100] For example, the number of iterations threshold is set according to the user's needs in the actual application scenario. Iterative code prediction processing is performed, for example: the current iteration number is i, based on the second text feature and the i-1 predicted code characters as input, predicting the i-th code character. i is a positive integer greater than 1. When i equals the number of iterations threshold, the iteration stops, and all predicted code characters are output.

[0101] In step 3023, each predicted code character is combined according to the generation order to obtain the second character sequence.

[0102] For example, continuing from the above example, we can combine each code character of the mahjong tiles in the order from 1 to N to obtain the second character sequence, where N is a positive integer and is the total number of code characters.

[0103] In step 3024, each code character in the second character sequence is arranged according to the code format to obtain the predicted code text corresponding to the sample instruction.

[0104] For example, the second character sequence is presented as a complete string of characters. The code has a corresponding arrangement format in practical applications. Based on the arrangement method corresponding to the language type of the code, the code characters in the second character sequence are converted from sequence form to code format arrangement. For example, if the second character sequence is represented as [```python def binary_search(arr,x):……], it is converted to... Figure 7 The code format in [the document / document].

[0105] In this embodiment, the next code character is predicted based on the predicted content, which makes the code characters logically related and improves the accuracy of generating code characters.

[0106] Continue to refer to Figure 3A In step 303, multiple sample instructions and their corresponding prediction code texts are combined into multiple sample pairs.

[0107] Here, each sample pair includes a sample instruction and the corresponding prediction code text.

[0108] For example, a single sample instruction can generate multiple different predicted code texts. Therefore, the sample instructions in multiple sample pairs can be the same, but the predicted code texts can be different. (e.g., {x}) i} represents the set of sample instructions, x i Refers to any sample instruction, based on sample instruction x i The generated prediction code text can be represented as a set If i and k are both positive integers, then the sample instruction x i Any corresponding sample pair can be characterized as n is a positive integer less than or equal to k.

[0109] In step 304, the first code processing model is called to determine the quality index corresponding to each sample pair, and each sample pair is labeled according to the quality index corresponding to each sample pair.

[0110] For example, a quality metric is a parameter used to characterize the degree of matching between a sample instruction and its corresponding predicted code text in a sample pair. A quality metric can be determined from at least one aspect, such as naming conventions or the degree to which the code satisfies the requirements of the sample instruction. This embodiment uses a combination of multiple aspects as an example; however, in practical applications, a quality metric can be determined from only one aspect. After determining the quality metric, it is labeled to each corresponding sample pair. This can be done by adding the quality metric as a label field to the data of the corresponding sample pair. For example, a sample pair with a label field is characterized as follows: Where, x i It is the sample instruction in the sample pair. Based on sample instruction x i The generated prediction code text uses a quality metric q to characterize x in a sample pair. i , The degree of matching between them.

[0111] For example, the process of determining quality indicators can be implemented using a recurrent neural network. See also... Figure 6 For the code evaluation model 630 in the code processing model 600, during the training phase and the training completion phase, the predicted code text and the instructions corresponding to the predicted code text are input into the input layer 631 of the code evaluation model 630. The hidden layer 632 calls the activation function to process the input data, and the output layer 633 outputs the quality index corresponding to each predicted code text. During the training phase, the code evaluation model 630 outputs sample pairs labeled with quality indexes, and the sample pairs labeled with quality indexes are used in the training process of the code processing model 600.

[0112] In some embodiments, reference Figure 3C , Figure 3C This is a schematic diagram of the third process of the training method for the code processing model provided in this application embodiment; the "calling the first code processing model to determine the quality index corresponding to each sample pair" in step 304 can be achieved through... Figure 3C Steps 3041 to 3043 are implemented, and the details are explained below.

[0113] In step 3041, the first code processing model is invoked to perform the following processing for each sample pair: determine multiple types of matching metrics between the sample instruction and the predicted code text corresponding to the sample instruction.

[0114] Here, the matching metric is used to characterize the degree of matching between the predicted code text and the required content of the sample instructions.

[0115] In some embodiments, reference Figure 3D , Figure 3D This is a schematic diagram of the fourth step of the training method for the code processing model provided in this application embodiment; step 3041 can be achieved through... Figure 3D Steps 30411 to 30414 are implemented, and the details are explained below.

[0116] In step 30411, semantic understanding is performed on the second text features of the sample instruction to obtain the requirement content of the sample instruction.

[0117] For example, the requirement content is the text in the sample instructions used to characterize the requirements for the generated code. The types of requirement content include: the format or language type of the code text; the length of the code text; the number of code texts; the purpose or function implemented by the program corresponding to the code text, etc.

[0118] To facilitate understanding, let's illustrate with the example above. The instruction, "Please give me multiple pieces of Python code for timing," includes "multiple pieces," "timing," and "Python code." Here, "multiple pieces" refers to the number of prediction code texts, "timing" refers to the functionality implemented by the program corresponding to the prediction code texts, and "Python code" refers to the prediction code texts or the language type.

[0119] In step 30412, in response to the predicted code text satisfying the first matching condition, a first matching index is configured for the sample pair.

[0120] For example, the first matching condition includes: the similarity between the predicted code text and a portion of the requirement content is greater than a first similarity threshold. Step 30412 can be understood as follows: if the predicted code text is related to at least a portion of the sample instructions, then a first matching index can be configured for the sample pair corresponding to the predicted code text. The values ​​of the first similarity threshold and the first matching index can be set according to the needs of the actual application scenario.

[0121] Continuing with the above requirements as an example, some requirements refer to at least one or more of the following: "multi-segment," "timing," and "Python code." Assume that the predicted code text addresses at least one or more of the above requirements. Even if the predicted code text does not address any of the requirements, a score (of the first matching metric) is assigned to the sample pair corresponding to the predicted code text.

[0122] In step 30413, in response to the predicted code text satisfying the second matching condition, a second matching index is configured for the sample pair.

[0123] For example, the second matching condition includes: the similarity between the predicted code text and all the required content is greater than a second similarity threshold. Step 30413 can be understood as follows: if the predicted code text is correlated with all the content in the sample instructions, then a second matching index can be configured for the sample pair corresponding to the predicted code text. The values ​​of the second similarity threshold and the second matching index can be set according to the needs of the actual application scenario.

[0124] For example, if the predicted code text addresses the requirements of "multi-segment", "timing", and "Python code" mentioned above, then a score (second matching index) can be configured for the sample pair corresponding to the predicted code text.

[0125] In step 30414, in response to the predicted code text satisfying the second matching condition and the predicted code text having no runtime errors, a third matching metric is configured for the sample pair.

[0126] For example, the absence of runtime errors in the predicted code text means that the code is free of defects (bugs). For instance, if the predicted code text directly and comprehensively answers the instruction "Please give me multiple pieces of Python code for timing," and has no execution defects (bugs), then the sample pair corresponding to the predicted code text can be assigned 1 score (the third matching metric); or, if the predicted code text directly and comprehensively answers the instruction "Please give me multiple pieces of Python code for timing," and has no execution defects (bugs) and does not contain any redundant content unrelated to the instruction, then the sample pair corresponding to the predicted code text can be assigned 5 scores (the third matching metric).

[0127] In some embodiments, if the predicted code text does not meet the first matching condition, there is no need to determine the second matching condition for the predicted code text, nor to configure the second and third matching metrics. This saves the computational resources required to determine the predicted code text.

[0128] In this embodiment, the matching degree is quantified into corresponding matching indicators, and the matching degree is determined from multiple levels. The matching degree is combined with code defects as an evaluation standard, which improves the accuracy of evaluating the predicted code.

[0129] Continue to refer to Figure 3C In step 3042, the code specification index of the predicted code text is determined based on the code language type of the predicted code text.

[0130] For example, code languages ​​can be categorized into front-end languages ​​and back-end languages. Front-end languages ​​are primarily used for web development, including HTML, CSS, and JavaScript; while back-end languages ​​are typically used for server-side development, such as Java, Python, and PHP. Each code language type has corresponding naming conventions. Naming conventions are rules and agreements designed to improve code readability, maintainability, and reusability. These conventions include variable naming, class naming, method naming, and file naming.

[0131] In some embodiments, step 3042 can be implemented by: determining the naming convention corresponding to the code language type of the predicted code text; traversing each code character in the predicted code text and determining the number of first characters that satisfy the naming convention among the traversed code characters; and determining the code convention index of the predicted code text based on the ratio between the number of first characters and the total number of characters in the predicted code text.

[0132] For example, the ratio between the number of the first character and the total number of characters in the predicted code text can be used directly as a code style indicator. Alternatively, the ratio can be summed or multiplied by a pre-configured value. In practical applications, the pre-configured value and the method for determining the code style indicator can be set according to user needs.

[0133] In step 3043, each matching metric and code standardization metric are added together to obtain the quality metric corresponding to the sample pair.

[0134] For example, if the predicted code does not meet the corresponding matching conditions, the matching index can be 0. The quality index of the sample pair is obtained by adding each matching index of the predicted code to the code specification index.

[0135] In this embodiment, the degree of matching between the code and the sample instructions, and the degree to which the code conforms to the naming conventions are quantified, thereby improving the accuracy of determining quality indicators.

[0136] Continue to refer to Figure 3A In step 305, the first code processing model is trained based on multiple labeled sample pairs to obtain the second code processing model.

[0137] For example, quality metrics can reflect the quality of the predicted code text in each sample pair. Among the multiple labeled sample pairs, there are different quality metrics. Training the first code processing model based on the multiple labeled sample pairs can enable the second code processing model to distinguish the quality of the predicted code text during the code generation process, so as to achieve self-optimization of the model.

[0138] In some embodiments, reference Figure 3E , Figure 3E This is a schematic diagram of the fifth step in the training method of the code processing model provided in this application embodiment. Step 305 can be achieved through... Figure 3E Steps 3051 to 3053 are implemented, and the details are explained below.

[0139] In step 3051, multiple sample pairs are grouped to obtain multiple sample combinations.

[0140] For example, each sample combination includes two sample pairs, and the two sample pairs in the sample combination have different quality metrics. This difference in quality metrics is intended to train the code processing model's ability to distinguish between good and bad predicted code text during code generation.

[0141] Continuing with the example above, when the sample instructions in two sample pairs are the same, the sample combination can be simplified as follows: This saves storage space required for storing sample combinations and computational resources required for training models using sample combinations.

[0142] In step 3052, for each sample combination, the target sample pairs with higher quality indicators in the sample combination are labeled.

[0143] For example, annotation methods can be to directly annotate sample pairs within a sample combination, or to set the quality index of a sample combination by ranking the predicted code text higher. For instance, a sample combination can be characterized as... The quality indicators are superior to of, Sort in Previously, the code text was predicted. The corresponding sample pairs are considered to have higher quality indicators by default, and are used to label sample combinations. Predictive code text with high quality indicators.

[0144] In step 3053, the first code processing model is trained based on each labeled sample combination to obtain the second code processing model.

[0145] For example, the training method can be supervised training, and the loss function used in the training process can be cross-entropy loss. The purpose of cross-entropy loss is to measure the difference between the model's predicted distribution and the true distribution. The first code processing model is trained using a combination of samples generated by the model itself as the training set, so that the first code processing model acts as its own reward model and provides a reward signal to itself. In this embodiment, the reward signal is a labeled high-quality indicator.

[0146] In some embodiments, step 3053 can be implemented as follows: based on the sample instructions and predicted code text in each labeled sample combination, the first code processing model is called to perform recognition processing to obtain the predicted sample pair corresponding to each sample combination; based on the difference between each predicted sample pair and the target sample pair in the corresponding sample combination, the cross-entropy loss of the first code processing model is determined; based on the cross-entropy loss, the parameters of the first code processing model are updated to obtain the second code processing model.

[0147] For example, a predicted sample pair refers to a sample pair in a sample combination that is likely to have a higher quality metric. The difference between each predicted sample pair and the target sample pair in the corresponding sample combination can be the difference between the predicted probabilities. Updating the parameters of the first code processing model can be done through backpropagation.

[0148] In this embodiment, the training method described above enables the code processing model not only to generate code but also to evaluate the quality of the generated predicted code. The training process simulates the self-checking process of human programmers after writing code, improving the accuracy and efficiency of the code processing model in generating code.

[0149] In some embodiments, after step 305, the second code processing model is used as the first code processing model, and the first code processing model is iteratively trained based on multiple labeled sample pairs, wherein the multiple labeled sample pairs used in each training process are generated by the first code processing model obtained in the previous training process.

[0150] For example, assume the current code processing model is model M. t t is a positive integer, based on code processing model M t Generate training data for round t. The training data for round t includes the code processing model M.t The acquired labeled sample pairs, i.e., the sample pairs labeled with the higher quality index, are used to train the code processing model M based on the training data from round t. t The trained model M is obtained. t+1 Similarly, model M t+2 Based on model M t+1 The model is trained using multiple labeled sample pairs. The above process employs a direct preference optimization algorithm, which optimizes the model by comparing the preferences of different outputs (in this embodiment, preference refers to the predicted code text with higher quality indicators).

[0151] In some embodiments, after step 305, in response to an input operation, a processing instruction corresponding to the input operation is determined; a second code processing model is invoked based on the processing instruction to perform code generation processing, resulting in multiple candidate code texts; the second code processing model is invoked to determine the target code text among the multiple candidate code texts, wherein the target code text is the candidate code text that best matches the required content of the processing instruction.

[0152] For example, input operations could be the user copying existing instruction text or typing using a physical or virtual keyboard. The second code processing model is a trained code processing model capable of generating multiple different candidate code texts and identifying the highest quality code text from them. See also... Figure 6 During the training completion phase, the instructions to be processed are input into the first converter model 610 of the code processing model 600. The first converter model 610 outputs multiple candidate code texts to the code evaluation model 630. The code evaluation model 630 determines the quality index of the multiple candidate code texts and outputs the target code text with the highest quality index.

[0153] In this embodiment, the first code processing model itself is invoked to perform quality evaluation on the generated predicted code text, and the quality evaluation results are used as labels to annotate each sample pair. The first code processing model is used as its own reward model. Compared with the method of manually annotating training data in related technologies, this saves the computing resources and time consumed in the training process of the code processing model. The first code processing model is trained based on the sample pairs labeled with quality indicators, so that the first code processing model has the function of determining the better code among multiple predicted codes, realizing the self-optimization of the model and improving the accuracy of the code generated by the trained second code processing model.

[0154] The following will describe an exemplary application of the training method of the code processing model of this application in a real-world application scenario.

[0155] With the rapid development of artificial intelligence technology, large-scale language models have made significant progress in the field of natural language processing. However, the application of large-scale language models in specific areas such as code generation and optimization still faces a series of challenges. The code processing models of related technologies rely on large amounts of manually labeled data, consuming high computational and human resources, and are difficult to cover all programming languages ​​and scenarios. Furthermore, they often lack the ability to self-evaluate code quality during generation, leading to potential problems such as inefficiency and poor readability. The code generation methods of related technologies include the following:

[0156] 1. Rule-based code generation: Relies on predefined programming rules and templates to generate code through logical judgments and conditional branching. Rule-based code generation has limited flexibility and adaptability when facing complex and ever-changing programming scenarios.

[0157] 2. Template-based code generation: This method uses existing codebases to generate new code by filling in variables and parameters in templates. However, template-based code generation struggles to handle logical complexity and innovation in code.

[0158] 3. Machine Learning-Based Code Generation: This method trains machine learning models, such as decision trees, support vector machines, or neural networks, to learn the patterns and structure of code. Machine learning-based code generation often requires large amounts of labeled data and faces challenges in understanding the deep logic and semantics of the code.

[0159] 4. Deep learning-based code generation: Code generation is performed using recurrent neural networks (RNN) and transformer models. However, deep learning-based code generation still needs improvement in terms of self-evaluation and continuous optimization.

[0160] 5. Human Feedback-Based Reinforcement Learning: This method incorporates feedback from human experts and trains the model using reinforcement learning algorithms. After generating code, the model adjusts based on the evaluations of human experts. Human feedback-based reinforcement learning relies on the involvement of human experts, making it costly and difficult to scale.

[0161] 6. Code quality assessment tools: such as static code analyzers and code review systems. Code quality assessment tools usually focus on static code analysis and lack dynamic detection and optimization of the code generation process.

[0162] 7. Data Augmentation Techniques: In the field of natural language processing, data augmentation techniques such as synonym replacement, random insertion, and deletion are widely used. However, in the field of code generation, the application of data augmentation techniques is not yet mature because the syntax and semantic structure of code are more complex than those of natural language.

[0163] These related technologies have driven the development of code generation technology to some extent, but they have limitations in self-learning and continuous optimization. For code processing models based on large language models, model tuning heavily relies on manually labeled supervised fine-tuning data (SFT) and reinforcement learning from human feedback (RLHF). This not only increases manual costs and computational resource consumption, but also makes it difficult to guarantee the quality and consistency of the data. Furthermore, once trained, these models often cannot improve themselves and struggle to adapt to changes in the programming environment.

[0164] The code processing model training method in this application employs a self-reward tuning mechanism. Through iterative learning and evaluation by the model itself, it continuously improves code generation capabilities while reducing dependence on external resources. This saves computational and human resources during model training and improves code quality. The model is allowed to continuously self-evaluate and iterate during training, thereby achieving continuous learning and optimization, improving its adaptability and long-term performance. It can dynamically adjust the code generation strategy based on real-time user feedback and personalized needs, providing more accurate services that meet user requirements.

[0165] The training method of the code processing model provided in the embodiments of this application will be explained below with reference to the accompanying drawings. For easier understanding of the training method of the code processing model provided in the embodiments of this application, please refer to... Figure 5 , Figure 5 This is a schematic diagram illustrating the principle of the training method for the code processing model provided in this application embodiment; the training method in this application embodiment allows a large code model to improve its code generation capability through self-learning and evaluation without external reward signals.

[0166] In step 501, the model is fine-tuned.

[0167] The initial code processing model M0 is fine-tuned using different types of seed instructions; this is the process of pre-training the model. After fine-tuning, the code processing model M1 is obtained.

[0168] In step 502, the self-instruction is generated.

[0169] Based on the code generation (G) instructions in the seed instructions, instruction generalization is performed to obtain the generated generalized instruction set, that is, the set of new code generation instructions {x}. i}, where i is a positive integer. Let the set {x} be... i Input code processing model M tIn this context, t represents the current iteration training number. If this is the first training iteration, the code is input into the code processing model M1 to obtain the code generation result based on the code generation instructions. Where k is a positive integer greater than 1. Code processing model M t Code generation results Evaluate and generate a score.

[0170] In step 503, self-preference tuning is performed.

[0171] Generate code results set {x i} Divided into multiple preference outcome pairs l and w are positive integers greater than or equal to 1 and less than or equal to k. Each preference pair is scored based on the generated result. Sort the generated code results, for example: The evaluation results are better than of, Sort in Previously, based on multiple preference results, the code was trained using a self-preference tuning algorithm to process model M. t The trained model M is obtained. t+1 Model M t+1 This will be used as the model in the next training process.

[0172] The following provides a detailed explanation of the training method for the code processing model in the embodiments of this application. (Reference) Figure 4 , Figure 4 This is a schematic diagram of the sixth process of the training method for the code processing model provided in the embodiments of this application.

[0173] In step 401, the initial code processing model is fine-tuned using a seed instruction set to obtain a pre-trained code processing model.

[0174] For example, step 401 corresponds to Figure 5 Step 501. The initial code processing model is a large language model used to generate code. The code processing model involves the following functions: code generation (G), code evaluation (E), and self-instruction generation (SIG).

[0175] The code generation function provides a prompt describing the user's request and generates high-quality, useful (and executable) code based on that prompt. The self-instruction generation function generates instructions and code, evaluates new instructions, and adds this new instruction data to the model's own training set by following a few examples. The code evaluation function assesses the quality of generated code or existing generated code. These functions combine to enable the model to perform self-calibration; they are components of iteratively training the model using Artificial Intelligence Feedback (AIF).

[0176] For example, seed instructions are used to fine-tune the initial model. These seed instructions include code generation instructions, code evaluation instructions, and self-instruction generation instructions. Two manually constructed instruction sets (G, E, SIG) are used to perform supervised fine-tuning of the pre-trained model, enabling it to perform code generation, code evaluation, and self-instruction generation in a standardized manner. The model then possesses the ability to perform these three tasks.

[0177] For example, the code generation instructions used to train code generation functionality could be the following:

[0178] Instruction: Please generate a Python code for binary search.

[0179] refer to Figure 7 , Figure 7 This is a schematic diagram of the output content of the code processing model provided in this application embodiment. The model output generated by the code processing model according to the above instructions can be... Figure 7 The content in [the document / article].

[0180] In step 402, the pre-trained code processing model is invoked to generate generalization instructions and candidate answers for each generalization instruction.

[0181] For example, the pre-trained code processing model is invoked, and based on the self-instruction generation in the seed instruction, sample instructions for generalization and candidate answers corresponding to each sample instruction are generated (e.g., as mentioned above). Figure 7 (The code in the text). The generalized sample instructions are also known as generalized instructions. Step 402 can be implemented as follows: use the fine-tuned trained code processing model to perform instruction generalization based on the Self-instruction Generation (SIG) instructions to obtain the generated generalized instruction set.

[0182] Self-instruction generation instructions for code processing model inputs can include the following:

[0183] Instructions: You are required to generate a set of 20 different code generation task instructions. The following are the requirements for the generation task instructions:

[0184] 1. Try not to repeat verbs in every instruction to maximize variety.

[0185] 2. The language used in teaching should also be diverse. For example, you should combine interrogative and imperative sentences.

[0186] 3. The types of instructions should be diverse. The list should include various types of programming tasks, such as open generation, sorting, editing, optimization, etc.

[0187] 4. The AI ​​language model should be able to perform the instruction. For example, do not ask the assistant to create any visual or audio output. As another example, do not ask the assistant to wake you up at 5 PM or set a reminder, as it cannot perform any action.

[0188] 5. The instruction manual should be in English.

[0189] 6. The explanation should be at least one or two sentences long. Imperative and interrogative sentences are both allowed.

[0190] 7. You should generate appropriate input for the instruction. The input field should contain a specific example provided for the instruction. It should contain real data, not simple placeholders. The input should be rich in content to make the teaching challenging, but preferably no more than 100 words.

[0191] 8. Not all instructions require input. For example, when an instruction asks for general information, such as "Write a program to load a file," it is not necessary to provide specific context. In this example, this embodiment only requires "..." to be entered in the input field.<no input> ".

[0192] 9. The output should be an appropriate response to the instructions and inputs.

[0193] 10. All tasks should be related to code generation or programming.

[0194] For example, in the above self-instruction generation instructions, "you" refers to the code processing model itself. The generated generalized instruction set (the sample instruction set above) is used as a prompt to call the code processing model to generate k answers, each answer being a piece of code text, and each instruction x... i Generate k pairs of instruction-answer pairs

[0195] In step 403, the pre-trained code processing model is invoked to determine the quality index for each sample pair.

[0196] For example, in this embodiment of the application, the quality index is represented by numerical scores. The code processing model uses instruction evaluation capabilities to score the generated answers, obtaining k scores. In this embodiment of the application, for each sample pair, multiple scoring conditions described below are used, and scores are accumulated according to the degree to which each scoring condition is met to obtain the score for each instruction-answer pair:

[0197] Scoring Criterion 1: If the generated code is relevant to the user's requirements and includes some relevant information, even if the generated code is incomplete or contains some irrelevant content, add 1 point.

[0198] For example, in practical applications, "relevance to user requirements and including some relevant information" can be represented by similarity or percentage. When the similarity between the generated code and the user's requirements reaches a pre-configured threshold (e.g., 20%), it is determined that scoring condition 1 is met, and 1 point is added. The specific value of the added points can be set according to the application scenario, which will not be elaborated below.

[0199] Scoring criterion 2: If the generated code solves most of the user's problem, but some key logic is not implemented, add 1 point.

[0200] For example, in practical applications, "most of the content of the user's question" can be determined by similarity or percentage. For instance, if the generated code is similar to the user's requested content to a pre-configured threshold (e.g., 50%), then it is determined to meet scoring condition 2 and 1 point is added.

[0201] Scoring criterion 3: If the generated code answers the user's question with all the basic logic, regardless of whether it looks like it was written by an AI assistant or not, and may have some minor flaws (bugs) but does not affect actual execution, add 1 point.

[0202] For example, if the generated code is similar to the user's requested content to a pre-configured threshold (e.g., 90%) and the generated code can run, then it is determined that scoring condition 2 is met, and 1 point is added.

[0203] Scoring Criterion 4: If the generated code answers the user's question directly and comprehensively from the perspective of the AI ​​assistant and has no execution defects (bugs), and the code structure is clear and the logic is clear, and it has a clear help and effect in achieving the task or solving the problem, even if there are minor areas for improvement in code style, add 1 point.

[0204] For example, if the generated code meets the user's requirements for each section, and the generated code can run without defects, then it is determined that scoring condition 2 is met, and 1 point is added.

[0205] Scoring Criterion 5: A score of 5 is given for a code snippet that is perfectly tailored to the user’s question, provided by the AI ​​assistant, with no superfluous information, reflects expert knowledge, and demonstrates a high-quality, engaging, and insightful answer.

[0206] For example, if the generated code meets the user's requirements for each section, and the generated code is runnable, has no defects, and conforms to naming conventions, then it is determined that scoring condition 2 is met, and 1 point is added.

[0207] In some embodiments, the generated code can be evaluated from at least one dimension, including code generation specifications and the extent to which the code meets user needs. Code generation specifications may include the following:

[0208] The generated code must conform to naming conventions:

[0209] 1. Variable naming:

[0210] Use meaningful names and avoid using single characters (unless it's a loop counter, etc.).

[0211] In Python, lowercase letters and underscores (snake_case) are typically used to name variables.

[0212] In languages ​​such as JavaScript, Java, and C#, camelCase is commonly used.

[0213] 2. Function and method naming:

[0214] Use a verb followed by a noun to indicate the function or method's purpose.

[0215] To maintain consistency, for example, if a function is used to retrieve data, its name might start with "get".

[0216] The code processing model systematically assigns scores to each generated piece of code based on the aforementioned naming conventions and scoring criteria.

[0217] In this embodiment, compared to related technologies, a code processing model with stronger code generation capabilities can be trained, and the preferred direction of code generation by the code processing model can be adjusted as needed. By using manually written seed instructions for generation and evaluation, the model generalizes the generated instruction set and trains the model, enabling the model to perform n rounds of self-reward tuning, achieving automated code generation tuning, and giving the large code model self-learning and continuous optimization capabilities.

[0218] In step 404, preference sample combinations are constructed, and a pre-trained code processing model is iteratively trained based on each preference sample combination using the direct preference optimization algorithm.

[0219] For example, constructing preference sample combinations involves grouping sample pairs into multiple preference sample combinations. Each preference sample combination includes two sample pairs (instruction-answer pairs) with different quality metrics. Constructing preference sample combinations can be achieved as follows: divide multiple instruction-answer pairs into multiple groups of two, sort the scores within each group, and label the higher-scoring pair as "better," i.e., label the preference, thus obtaining the preference sample combination. Add each preference sample combination to a preference sample combination dataset. A preference sample combination includes: an instruction-answer pair labeled with a higher score, and an unlabeled instruction-answer pair with a lower score than the other.

[0220] For example: the combination of preference samples is represented as The evaluation results are better than of, Sort in Previously, based on the combination of labeled preference samples Generate code with a high score.

[0221] The process of iteratively training the code to process the model using the direct preference optimization algorithm can be achieved as follows: the model is trained based on a dataset of preference sample combinations, so that the model can distinguish the better instruction-answer pair in each preference sample combination. The training method can be supervised training, and the loss function can be cross-entropy loss.

[0222] The model itself acts as its own reward model, replacing the need for reward signals obtained from an external model. This is achieved through a large model evaluation mechanism (LLM-as-a-Judge), which assigns response evaluation as an instruction following the training task. Self-generated AI feedback preference data is used as the training set.

[0223] The overall self-rewarding optimization in this application is an iterative process, with each iteration improving upon the model from the previous iteration. Importantly, because the model can both improve its code generation capabilities and use the same generation mechanism as its reward model, the reward model itself can be improved through these iterations, whereas in standard practices of related technologies, the reward model is fixed. The model achieves self-guidance and iterative optimization through a self-rewarding mechanism. The model will be able to self-evaluate during training, continuously iterating to adapt to changes in the programming environment, enhancing its adaptability and long-term performance. Through self-generated, diverse code examples, the model will better adapt to different programming languages ​​and scenarios. Simultaneously, through iterative learning and self-evaluation, the model will gradually improve its ability to handle complex logic and innovative tasks.

[0224] In some embodiments of this application, other artificial intelligence learning methods can also be applied to improve the performance and efficiency of the trained code processing model. For example: applying reinforcement learning allows the model to learn to generate better code through interaction with the environment. Applying meta-learning methods allows the model to learn how to learn in order to quickly adapt to new tasks. Applying transfer learning accelerates the training of the model on new tasks by applying knowledge learned in one domain to another. Applying multi-task learning allows the model to learn multiple tasks simultaneously to improve its generalization ability. Applying knowledge distillation techniques can transfer knowledge from large models to smaller models to reduce resource requirements. Modular and plug-in designs provide flexibility and scalability, allowing the model to dynamically combine modules according to task requirements. Developing automated tools to evaluate code quality can help the model learn to generate higher-quality code. These solutions each have their advantages and challenges, and can be selected and integrated according to actual needs and resources to promote the advancement of code generation technology.

[0225] The code processing model training method provided in this application embodiment can be applied in the programming, development and maintenance of applications. The model trained by the code processing model training method can perform code completion, error detection, code review and automatic code generation for application code.

[0226] The beneficial effects of the training method for the code processing model provided in this application include: 1. Self-generated code examples as training data: By using a pre-trained large code model and through a specific generation strategy, the model is guided to generate new code examples, covering a variety of programming languages ​​and scenarios, simulating programming tasks in the real world, thereby providing the model with rich learning materials.

[0227] 2. Implement self-evaluation and reward for the model: The model not only generates code but also evaluates the quality of the generated code through a self-evaluation mechanism. This simulates the self-censorship of human programmers after writing code, ensuring the accuracy, efficiency, and maintainability of the generated code.

[0228] 3. An iterative training method is adopted, in which the model adjusts its generation strategy based on the self-evaluation results in each round of training. This allows the model to gradually acquire the ability to generate better code and achieve self-optimization.

[0229] 4. No external reward required: Unlike related technologies that rely on human feedback, the embodiments of this application allow the model to learn and improve itself without external reward signals, reducing the dependence on manually labeled data and improving training efficiency and scalability.

[0230] 5. Enhanced generalization ability: By self-generating and evaluating diverse code examples, the model can learn a wider range of programming knowledge and skills, thus adapting more quickly and producing high-quality code when faced with new programming tasks.

[0231] 6. Security and Reliability: When evaluating the quality of the generated code, the model follows strict programming standards to ensure that the generated code is not only functionally correct, but also easy to maintain and extend.

[0232] 7. Scalability and Personalization: It has good scalability and can be applied to different programming languages ​​and development environments. Furthermore, the model can provide personalized code generation services based on the user's cultural background, language habits, and psychological needs.

[0233] 8. Data Augmentation and Automation: This application embodiment focuses on data augmentation for the lexical rewriting instructions of large code models. By generating structured fine-tuning data through an automated process, the efficiency and accuracy of model training are improved.

[0234] The training method for the code processing model in this application embodiment enables the model to learn itself and continuously improve. With the continuous advancement of artificial intelligence technology, the training method for the code processing model in this application embodiment is expected to become an important driving force for future programming automation.

[0235] The following description continues to illustrate the exemplary structure of the training device 455 for the code processing model provided in this application embodiment as a software module. In some embodiments, such as... Figure 2 As shown, the software modules in the training device 455 of the code processing model stored in the memory 450 may include: a data acquisition module 4551, used to acquire a set of sample instructions, wherein the set of sample instructions includes multiple sample instructions; a code generation module 4552, used to call a pre-trained first code processing model based on the set of sample instructions to perform code generation processing, and obtain the predicted code text corresponding to each sample instruction; the data acquisition module 4551 is also used to form multiple sample pairs by combining multiple sample instructions and the predicted code text corresponding to the multiple sample instructions, wherein each sample pair includes a sample instruction and the predicted code text corresponding to the sample instruction; a code evaluation module 4553, used to call the first code processing model to determine the quality index corresponding to each sample pair, and to label each sample pair according to the quality index corresponding to each sample pair; and a model training module 4554, used to train the first code processing model based on the labeled multiple sample pairs to obtain a second code processing model.

[0236] In some embodiments, the code generation module 4552 is configured to, based on each sample instruction, call a pre-trained first code processing model to perform the following processing: extract text features from the sample instruction to obtain a second text feature of the sample instruction; perform code prediction processing based on the second text feature to obtain a second character sequence, wherein the second character sequence includes multiple code characters; and arrange each code character in the second character sequence according to the code format to obtain the predicted code text corresponding to the sample instruction.

[0237] In some embodiments, the code generation module 4552 is used to perform character prediction processing based on the second text features to obtain a first code character, and to use the first code character as the predicted code character; to perform code prediction processing iteratively based on the second text features and the predicted code character to obtain multiple code characters, wherein the input of each code prediction process includes: the predicted code character and the second text features, and the code prediction process stops when the number of iterations reaches a threshold; and to combine each predicted code character according to the generation order to obtain a second character sequence.

[0238] In some embodiments, the code evaluation module 4553 is used to call the first code processing model to perform the following processing for each sample pair: determine multiple types of matching indicators between the sample instruction and the predicted code text corresponding to the sample instruction, wherein the matching indicators are used to characterize the degree of matching between the predicted code text and the required content of the sample instruction; determine the code specification indicators of the predicted code text according to the code language type of the predicted code text; and add each matching indicator and the code specification indicator to obtain the quality indicator corresponding to the sample pair.

[0239] In some embodiments, the code evaluation module 4553 is configured to perform semantic understanding on the second text features of the sample instruction to obtain the requirement content of the sample instruction; in response to the predicted code text satisfying a first matching condition, configure a first matching index for the sample pair, wherein the first matching condition includes: the similarity between the predicted code text and part of the requirement content is greater than a first similarity threshold; in response to the predicted code text satisfying a second matching condition, configure a second matching index for the sample pair, wherein the second matching condition includes: the similarity between the predicted code text and all the requirement content is greater than a second similarity threshold; in response to the predicted code text satisfying the second matching condition and the predicted code text having no runtime errors, configure a third matching index for the sample pair.

[0240] In some embodiments, the code evaluation module 4553 is used to determine the naming convention corresponding to the code language type of the predicted code text; traverse each code character in the predicted code text to determine the number of first characters that satisfy the naming convention among the traversed code characters; and determine the code convention index of the predicted code text based on the ratio between the number of first characters and the total number of characters in the predicted code text.

[0241] In some embodiments, the model training module 4554 is used to group multiple sample pairs to obtain multiple sample combinations, wherein each sample combination includes two sample pairs, and the two sample pairs in the sample combination have different quality indicators; for each sample combination, the target sample pair with higher quality indicators in the sample combination is labeled; and a first code processing model is trained based on each labeled sample combination to obtain a second code processing model.

[0242] In some embodiments, the model training module 4554 is used to call the first code processing model to perform recognition processing based on the sample instructions and predicted code text in each labeled sample combination, to obtain the predicted sample pair corresponding to each sample combination; to determine the cross-entropy loss of the first code processing model based on the difference between each predicted sample pair and the target sample pair in the corresponding sample combination; and to update the parameters of the first code processing model based on the cross-entropy loss to obtain the second code processing model.

[0243] In some embodiments, the data acquisition module 4551 is used to acquire at least one first seed instruction, wherein the type of the first seed instruction is a code generation instruction; based on the at least one first seed instruction, a pre-trained first code processing model is invoked to perform instruction generation processing to obtain multiple sample instructions; and the multiple sample instructions are combined into a sample instruction set.

[0244] In some embodiments, the data acquisition module 4551 is used to call a pre-trained first code processing model to perform the following processing based on each first seed instruction: extracting text features from the seed instruction to obtain first text features; performing text prediction processing based on the first text features to obtain multiple different first character sequences; and decoding the multiple first character sequences to obtain multiple sample instructions corresponding to the seed instruction.

[0245] In some embodiments, the model training module 4554 is configured to: acquire a seed instruction set before acquiring a sample instruction set, wherein the seed instruction set includes a first seed instruction, a second seed instruction, and a third seed instruction; call an initial first code processing model based on the first seed instruction and perform fine-tuning processing corresponding to the instruction generation processing to obtain a first fine-tuned code processing model; call the first fine-tuned code processing model based on the second seed instruction and perform fine-tuning processing corresponding to the code generation processing to obtain a second fine-tuned code processing model and predicted code text corresponding to each second seed instruction; and call the second fine-tuned code processing model based on the third seed instruction and the predicted code text corresponding to the second instructions to perform fine-tuning processing corresponding to the determined quality index to obtain a pre-trained first code processing model.

[0246] In some embodiments, the first code processing model includes a converter model and a code evaluation model; wherein the converter model is used to obtain a sample instruction set and perform code generation processing; and the code evaluation model is used to determine quality indicators.

[0247] In some embodiments, the model training module 4554 is used to train a first code processing model based on multiple labeled sample pairs to obtain a second code processing model, and then use the second code processing model as the first code processing model to iteratively train the first code processing model based on multiple labeled sample pairs, wherein the multiple labeled sample pairs used in each training process are generated by the first code processing model obtained in the previous training process.

[0248] In some embodiments, the code generation module 4552 is used to train a first code processing model based on multiple labeled sample pairs to obtain a second code processing model, and then, in response to an input operation, determine the instruction to be processed corresponding to the input operation; call the second code processing model to perform code generation processing based on the instruction to be processed to obtain multiple candidate code texts; and call the second code processing model to determine the target code text among the multiple candidate code texts, wherein the target code text is the candidate code text that best matches the requirement content of the instruction to be processed.

[0249] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. An electronic device's processor reads the computer program or computer-executable instructions from the computer-readable storage medium and executes the computer program or computer-executable instructions, causing the electronic device to perform the training method of the code processing model described in this application.

[0250] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the training method of the code processing model provided in this application. For example, ... Figure 3A The training method for the code processing model is shown.

[0251] In some embodiments, the computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0252] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0253] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0254] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0255] In summary, by invoking the first code processing model itself to perform quality evaluation on the generated predicted code text through the embodiments of this application, and using the quality evaluation results as labels to annotate each sample pair, and using the first code processing model as its own reward model, compared with the method of manually annotating training data in related technologies, the computational resources and time consumed in the training process of the code processing model are saved. The first code processing model is trained based on sample pairs labeled with quality indicators, so that the first code processing model has the function of determining the better code among multiple predicted codes, realizing the self-optimization of the model, and improving the accuracy of the code generated by the trained second code processing model.

[0256] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A training method for a code processing model, characterized in that, The method includes: Obtain a set of sample instructions, wherein the set of sample instructions includes multiple sample instructions; Based on the set of sample instructions, a pre-trained first code processing model is invoked to perform code generation processing, thereby obtaining the predicted code text corresponding to each of the sample instructions. The plurality of sample instructions and the prediction code text corresponding to the plurality of sample instructions are combined into a plurality of sample pairs, wherein each sample pair includes a sample instruction and the prediction code text corresponding to the sample instruction; The first code processing model is invoked to determine the quality index corresponding to each sample pair, and each sample pair is labeled according to the quality index corresponding to each sample pair. The first code processing model is trained based on the labeled sample pairs to obtain the second code processing model.

2. The method according to claim 1, characterized in that, The step of calling a pre-trained first code processing model based on the set of sample instructions to perform code generation processing, and obtaining the predicted code text corresponding to each of the sample instructions, includes: Based on each of the sample instructions, the pre-trained first code processing model is invoked to perform the following processing: The sample instruction is subjected to text feature extraction processing to obtain the second text feature of the sample instruction; Based on the second text features, code prediction processing is performed to obtain a second character sequence, wherein the second character sequence includes multiple code characters; Based on the code format, each code character in the second character sequence is arranged to obtain the predicted code text corresponding to the sample instruction.

3. The method according to claim 2, characterized in that, The code prediction process based on the second text features to obtain the second character sequence includes: Based on the second text feature, character prediction processing is performed to obtain the first code character, and the first code character is used as the predicted code character. Based on the second text feature and the predicted code character, code prediction processing is performed iteratively to obtain multiple code characters. The input of each code prediction process includes the predicted code character and the second text feature. The code prediction process stops when the number of iterations reaches a threshold. The predicted code characters are combined according to the generation order to obtain the second character sequence.

4. The method according to claim 1, characterized in that, The step of calling the first code processing model to determine the quality index corresponding to each sample pair includes: The first code processing model is invoked to perform the following processing for each sample pair: Multiple types of matching metrics are determined between the sample instruction and the corresponding predicted code text, wherein the matching metrics are used to characterize the degree of matching between the predicted code text and the required content of the sample instruction; Determine the code style index of the predicted code text based on the code language type of the predicted code text; The quality index corresponding to each sample pair is obtained by adding each of the matching indicators and the code standardization indicators.

5. The method according to claim 4, characterized in that, The determination of multiple types of matching metrics between the sample instruction and the corresponding predicted code text includes: Semantic understanding is performed on the second textual features of the sample instruction to obtain the requirement content of the sample instruction; In response to the predicted code text satisfying a first matching condition, a first matching index is configured for the sample pair, wherein the first matching condition includes: the similarity between the predicted code text and part of the required content is greater than a first similarity threshold; In response to the predicted code text satisfying the second matching condition, a second matching index is configured for the sample pair, wherein the second matching condition includes: the similarity between the predicted code text and all the required content is greater than a second similarity threshold; In response to the predicted code text satisfying the second matching condition and the predicted code text having no runtime errors, a third matching metric is configured for the sample pair.

6. The method according to claim 4, characterized in that, The step of determining the code style index of the predicted code text based on the code language type of the predicted code text includes: Determine the naming conventions corresponding to the code language type of the predicted code text; Iterate through each code character in the predicted code text and determine the number of first characters among the iterated code characters that satisfy the naming convention; The code standardization index of the predicted code text is determined based on the ratio between the first number of characters and the total number of characters in the predicted code text.

7. The method according to claim 1, characterized in that, The process of training the first code processing model based on the labeled multiple sample pairs to obtain the second code processing model includes: The multiple sample pairs are grouped to obtain multiple sample combinations, wherein each sample combination includes two sample pairs, and the two sample pairs in the sample combination have different quality indicators; For each of the sample combinations, identify the target sample pairs with higher quality indicators within the sample combination; The first code processing model is trained based on each of the labeled sample combinations to obtain the second code processing model.

8. The method according to claim 1, characterized in that, The process of training the first code processing model based on each of the labeled sample combinations to obtain the second code processing model includes: Based on the sample instructions and the predicted code text in each labeled sample combination, the first code processing model is called to perform recognition processing to obtain the predicted sample pair corresponding to each sample combination. Based on the difference between each predicted sample pair and the target sample pair in the corresponding sample combination, the cross-entropy loss of the first code processing model is determined. The parameters of the first code processing model are updated based on the cross-entropy loss to obtain the second code processing model.

9. The method according to any one of claims 1 to 8, characterized in that, The set of instructions for obtaining samples includes: Obtain at least one first seed instruction, wherein the type of the first seed instruction is a code generation instruction; Based on the at least one first seed instruction, the pre-trained first code processing model is invoked to perform instruction generation processing to obtain the multiple sample instructions; The plurality of sample instructions are combined into the sample instruction set.

10. The method according to claim 9, characterized in that, Based on the at least one first seed instruction, the pre-trained first code processing model is invoked to perform instruction generation processing to obtain multiple sample instructions, including: Based on each of the first seed instructions, the pre-trained first code processing model is invoked to perform the following processing: The seed instruction is subjected to text feature extraction processing to obtain the first text feature; Based on the first text features, text prediction processing is performed to obtain multiple different first character sequences; The plurality of first character sequences are decoded to obtain a plurality of sample instructions corresponding to the seed instruction.

11. The method according to any one of claims 1 to 8, characterized in that, Prior to acquiring the sample instruction set, the method further includes: Obtain a set of seed instructions, wherein the set of seed instructions includes a first seed instruction, a second seed instruction, and a third seed instruction; Based on the first seed instruction, the initial first code processing model is invoked, and fine-tuning processing corresponding to the instruction generation processing is performed to obtain the first fine-tuned code processing model. Based on the second seed instruction, the first fine-tuning code processing model is invoked to perform fine-tuning processing corresponding to the code generation processing, thereby obtaining the second fine-tuning code processing model and the predicted code text corresponding to each of the second seed instructions. Based on the predicted code text corresponding to the third seed instruction and the second seed instruction respectively, the second fine-tuned code processing model is invoked to perform fine-tuning processing corresponding to the determined quality indicators, thereby obtaining the pre-trained first code processing model.

12. The method according to any one of claims 1 to 8, characterized in that, The first code processing model includes a converter model and a code evaluation model; wherein, the converter model is used to acquire a sample instruction set and execute the code generation process; and the code evaluation model is used to determine quality indicators.

13. The method according to any one of claims 1 to 8, characterized in that, After training the first code processing model based on the labeled sample pairs to obtain the second code processing model, the method further includes: The second code processing model is used as the first code processing model, and the first code processing model is trained iteratively based on the labeled multiple sample pairs, wherein the labeled multiple sample pairs used in each training process are generated by the first code processing model obtained in the previous training process.

14. The method according to any one of claims 1 to 8, characterized in that, After training the first code processing model based on the labeled sample pairs to obtain the second code processing model, the method further includes: In response to an input operation, determine the instruction to be processed corresponding to the input operation; Based on the instruction to be processed, the second code processing model is invoked to perform code generation processing, resulting in multiple candidate code texts; The second code processing model is invoked to determine the target code text among the plurality of candidate code texts, wherein the target code text is the candidate code text that best matches the required content of the instruction to be processed.

15. A training device for a code processing model, characterized in that, The device includes: A data acquisition module is used to acquire a sample instruction set, wherein the sample instruction set includes multiple sample instructions; The code generation module is used to call a pre-trained first code processing model to perform code generation processing based on the sample instruction set, so as to obtain the predicted code text corresponding to each sample instruction. The data acquisition module is further configured to form multiple sample pairs by combining the plurality of sample instructions and the prediction code text corresponding to the plurality of sample instructions respectively, wherein each sample pair includes a sample instruction and the prediction code text corresponding to the sample instruction; The code evaluation module is used to call the first code processing model to determine the quality index corresponding to each sample pair, and to label each sample pair according to the quality index corresponding to each sample pair. The model training module is used to train the first code processing model based on the labeled sample pairs to obtain the second code processing model.

16. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the training method of the code processing model according to any one of claims 1 to 14.

17. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, The computer-executable instructions or computer program, when executed by a processor, implement the training method of the code processing model according to any one of claims 1 to 14.

18. A computer program product comprising computer-executable instructions or a computer program, characterized in that, The training method for the code processing model according to any one of claims 1 to 14 is implemented when the computer-executable instructions or computer program are executed by a processor.

Images