Large language model code translation method based on prompt fine tuning
Through the large language model code translation method based on tip fine-tuning, pseudo-labeling and iterative self-training improve the performance of the translation model, the performance limitation caused by limited data volume in traditional methods is solved, and a higher quality code translation effect is achieved.
Patent Information
- Application Number
- CN202410667808.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-05-28
- Publication Date
- 2024-11-15
AI Technical Summary
Due to the limited data volume of traditional code translation methods, the performance of language models in code translation tasks is limited and cannot fully play a role in practical application scenarios.
A large language model code translation method based on tip fine-tuning is adopted. By obtaining the source language code for preliminary translation, pseudo-tagging is determined, and iterative self-training and fine-tuning is carried out to improve the performance of the translation model to adapt to specific generation tasks.
Through iterative self-training and fine-tuning, the translation quality and task completion are significantly improved, the performance of the language model in code translation tasks is enhanced, and the needs of specific generation tasks are met.
Smart Images

Figure CN118963756A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of computer natural language processing, and in particular to a large language model code translation method based on prompt fine-tuning. Background Art
[0002] With the rapid rise of large language models (LLMs) in the field of natural language processing, their application in code translation tasks has become a research hotspot. Code translation, as an important application field of LLMs, covers the conversion of source code to target code, providing a powerful tool for software developers. Past research has made significant progress, making LLMs perform well in code translation and related tasks.
[0003] Traditional code translation methods mainly translate keywords and identifiers in programming languages into Chinese, translate comments in codes into Chinese, and translate names of functions and variables into Chinese or translate names of functions and variables into Chinese.
[0004] However, in code translation tasks, the performance of language models of traditional code translation methods is limited due to the limited amount of data, and their effectiveness in actual application scenarios is also limited. Summary of the invention
[0005] In order to improve the performance of language models, the present application provides a large language model code translation method based on prompt fine-tuning.
[0006] In a first aspect, the present application provides a large language model code translation method based on prompt fine-tuning, which adopts the following technical solution: A method for code translation of large language models based on prompt fine-tuning, comprising: Get the source language code; Performing a preliminary translation of the source language code to determine a preliminary translation result; Determining a pseudo label according to the preliminary translation result; Perform iterative self-training according to the pseudo-labels and the source language codes to determine a translation model; The translation model is fine-tuned to determine a target translation model, where the target translation model is a translation model adapted to a specific generation task.
[0007] By adopting the above technical solution, the source language code for translation is obtained, thereby providing basic data for subsequent steps; then, the source language code is preliminarily translated to obtain a preliminary translation result, thereby simplifying and translating the source language code to facilitate subsequent steps; based on the preliminary translation result, a pseudo-label is determined, thereby providing additional supervision information during the training process; then, iterative self-training is performed based on the pseudo-label and the source language code to determine the translation model, and the performance of the translation model is improved through iterative self-training; the translation model is fine-tuned to obtain a target translation model, and the translation model after fine-tuning can better meet the needs of specific generation tasks, improve translation quality and task completion, thereby improving the performance of the language model.
[0008] In a possible implementation, the performing a preliminary translation of the source language code and determining a preliminary translation result may further include: Get masked language models across programming languages; Determine a pre-trained model according to the source language code and the mask language model; Based on the pre-trained model, a latent representation is determined.
[0009] In a possible implementation, performing a preliminary translation on the source language code and determining a preliminary translation result includes: Adding noise to the source language code to obtain a damaged source language code; Determining a predicted original source language code based on the damaged source language code and the latent representation; Inputting the predicted original source language code into a preset source-to-target model to determine a target language code; Determine the back-translation source language code based on the target language code and the preset target-to-source model; Determining an adaptation model according to the back-translated source language code and the target language code; Training the adaptation model to determine a training model; Determine the preliminary translation result based on the source language code and the training model.
[0010] In a possible implementation, determining a pseudo label according to the preliminary translation result includes: Determining a target language pseudo code according to the preliminary translation result and the source language code; Testing the target language pseudocode to determine that the target language pseudocode that passes the test is a pseudocode; The pseudo label is determined according to the pseudo code.
[0011] In a possible implementation, determining the pseudo label according to the preliminary translation result further includes: Determine quality assessment indicators and screening criteria based on the preliminary translation results; If the preliminary translation result successfully matches the screening criteria, determining that the preliminary translation result is a high-quality translation; Determining, based on the high-quality translation, a screening pseudo-label; The pseudo-label is determined according to the quality evaluation and the screening pseudo-label.
[0012] In a possible implementation, performing iterative self-training according to the pseudo-label and the source language code to determine the translation model includes: Determine a training set according to the pseudo-label and the source language code; The translation model is determined according to the training set.
[0013] In a possible implementation, the fine-tuning of the translation model to determine the target translation model includes: Determine an input sequence according to the translation model, wherein the input sequence is an input sequence with a prefix; Determine an output result based on the input sequence and the translation model; Determine a loss function according to the output result and the preset true label; Determine parameter information based on the loss function; The target translation model is determined according to the parameter information and the translation model.
[0014] In the second aspect, the present application provides a large language model code translation device based on prompt fine-tuning, which adopts the following technical solution: A large language model code translation device based on prompt fine-tuning includes: a source language code acquisition module, a preliminary translation result determination module, a pseudo-label determination module, a translation model determination module and a target translation model determination module, wherein: A source language code acquisition module, used to acquire source language code; A preliminary translation result determination module, used to perform preliminary translation on the source language code and determine a preliminary translation result; A pseudo-label determination module, used to determine a pseudo-label according to the preliminary translation result; A translation model determination module, used for iterative self-training based on the pseudo-labels and source language codes to determine the translation model; The target translation model determination module is used to fine-tune the translation model to determine the target translation model, where the target translation model is a translation model adapted to a specific generation task.
[0015] By adopting the above technical solution, the source language code acquisition module obtains the source language code for translation, thereby providing basic data for subsequent steps; then, the preliminary translation result determination module performs preliminary translation on the source language code to obtain preliminary translation results, thereby simplifying and translating the source language code to facilitate subsequent steps; the pseudo-label determination module determines the pseudo-label based on the preliminary translation results, thereby providing additional supervision information during the training process; thereafter, the translation model determination module performs iterative self-training based on the pseudo-label and the source language code to determine the translation model, and the performance of the translation model is improved through iterative self-training; the target translation model determination module fine-tunes the translation model to obtain the target translation model, and the translation model after fine-tuning can better meet the needs of specific generation tasks, improve translation quality and task completion, thereby improving the performance of the language model.
[0016] In a possible implementation, the large language model code translation device based on prompt fine-tuning further includes: a masked language model module, a pre-trained model determination module and a potential representation determination module, wherein: The masked language model module is used to obtain masked language models across programming languages; A pre-training model determination module, used to determine a pre-training model according to the source language code and the mask language model; The potential representation determination module is used to determine the potential representation according to the pre-trained model.
[0017] In a possible implementation, the preliminary translation result determination module includes: a damaged source language code determination unit, a predicted original source language code determination unit, a target language code determination unit, a back-translated source language code determination unit, an adaptation model determination unit, a training model determination unit, and a preliminary translation result determination unit, wherein: a damaged source language code determining unit, configured to add noise to the source language code to obtain a damaged source language code; a predicted original source language code determining unit, configured to determine a predicted original source language code according to the damaged source language code and the potential representation; a target language code determination unit, configured to input the predicted original source language code into a preset source-to-target model to determine a target language code; A back-translation source language code determination unit, used to determine the back-translation source language code according to the target language code and a preset target-to-source model; an adaptation model determination unit, configured to determine an adaptation model according to the back-translated source language code and the target language code; A training model determination unit, used to train the adaptation model and determine a training model; The preliminary translation result determination unit is used to determine the preliminary translation result according to the source language code and the training model.
[0018] In a possible implementation, the pseudo-label determination module includes: a target language pseudo-code determination unit, a pseudo-code determination unit, and a pseudo-label determination unit, wherein: A target language pseudo code determination unit, configured to determine a target language pseudo code according to the preliminary translation result and the source language code; A pseudocode determination unit, configured to test the target language pseudocode and determine that the target language pseudocode that passes the test is the pseudocode; The pseudo-label determining unit is configured to determine the pseudo-label according to the pseudo-code.
[0019] In a possible implementation, the pseudo-label determination module includes: a standard determination unit, a high-quality translation determination unit, a screening pseudo-label determination unit, and a pseudo-label determination unit, wherein: A standard determination unit, used to determine quality assessment indicators and screening criteria according to the preliminary translation results; a high-quality translation determination unit, configured to determine that the preliminary translation result is a high-quality translation if the preliminary translation result successfully matches the screening criteria; a screening pseudo-label determination unit, configured to determine a screening pseudo-label according to the high-quality translation; The pseudo-label determination unit is configured to determine the pseudo-label according to the quality evaluation and the screened pseudo-label.
[0020] In a possible implementation, the translation model determination module includes: a training set determination unit and a translation model determination unit, wherein: A training set determining unit, configured to determine a training set according to the pseudo-label and the source language code; The translation model determination unit is used to determine the translation model according to the training set.
[0021] In a possible implementation, the target translation model determination module includes: an input sequence determination unit, an output result determination unit, a loss function determination unit, a parameter information determination unit and a target translation model determination unit, wherein: An input sequence determination unit, used to determine an input sequence according to the translation model, wherein the input sequence is an input sequence with a prefix; an output result determining unit, configured to determine an output result based on the input sequence and the translation model; A loss function determination unit, used to determine a loss function according to the output result and a preset true label; A parameter information determination unit, used to determine parameter information according to the loss function; The target translation model determining unit is used to determine the target translation model according to the parameter information and the translation model.
[0022] In a third aspect, the present application provides an electronic device, which adopts the following technical solution: An electronic device, comprising: at least one processor; Memory; At least one application, wherein the at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute the above-mentioned large language model code translation method based on prompt fine-tuning.
[0023] In a fourth aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium includes: a computer program that can be loaded by a processor and execute the above-mentioned large-scale language model code translation method based on prompt fine-tuning.
[0024] In summary, this application includes the following beneficial technical effects: The source language code for translation is obtained to provide basic data for subsequent steps; then, the source language code is preliminarily translated to obtain a preliminary translation result, so that the source language code is simplified and translated to facilitate subsequent steps; based on the preliminary translation result, a pseudo-label is determined to provide additional supervision information during the training process; then, iterative self-training is performed based on the pseudo-label and the source language code to determine the translation model, and the performance of the translation model is improved through iterative self-training; the translation model is fine-tuned to obtain a target translation model, and the fine-tuned translation model can better meet the needs of specific generation tasks, improve translation quality and task completion, thereby improving language model performance. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] Figure 1 It is a flowchart of a large language model code translation method based on prompt fine-tuning of the present application; Figure 2 It is a block diagram of a large language model code translation device based on prompt fine-tuning of the present application; Figure 3 It is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0026] The following is combined with Figure 1-3 This application is described in further detail.
[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0028] The embodiment of the present application provides a large language model code translation method based on prompt fine-tuning, which is executed by an electronic device, which can be a server or a terminal device, wherein the server can be an independent physical server, a server cluster or a distributed device composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited thereto. The terminal device and the server can be directly or indirectly connected via wired or wireless communication, and the embodiment of the present application does not limit this.
[0029] Reference Figure 1 The method comprises: step S101, step S102, step S103, step S104 and step S105, wherein: Step S101: Obtain source language code.
[0030] Specifically, if the source language code is publicly accessible, the monitoring device extracts the source language code from the official website, code hosting platform or other relevant resources that need to be translated, and transmits it to the electronic device; if the source language code is not public, the monitoring device will automatically contact the maintainer or owner of the project to obtain access rights, extract the source language code, and transmit it to the electronic device.
[0031] Step S102: Perform a preliminary translation on the source language code to determine a preliminary translation result.
[0032] Specifically, after the electronic device obtains the source language code, Step S103: Determine a pseudo label according to the preliminary translation result.
[0033] Specifically, the electronic device adds noise to the source language code to obtain a damaged source language code; then the electronic device determines a predicted original source language code based on the damaged source language code and the potential representation; the electronic device inputs the predicted original source language code into a preset source-to-target model to determine the target language code; the electronic device determines a back-translated source language code based on the target language code and the preset target-to-source model; at the same time, the electronic device determines an adaptation model based on the back-translated source language code and the target language code; then the electronic device trains the adaptation model to determine a training model; the electronic device determines a preliminary translation result based on the source language code and the training model.
[0034] Step S104: Perform iterative self-training according to the pseudo-labels and the source language codes to determine the translation model.
[0035] Specifically, the electronic device determines a training set based on the pseudo-label and the source language code; then the electronic device determines a translation model based on the training set; or the electronic device introduces a cache mechanism to store previously generated samples; the model can choose to obtain previously generated samples from the cache, or generate new samples and add them to the cache; the electronic device uses the samples obtained from the cache or newly generated to perform online training on the model to obtain a translation model.
[0036] Step S105: fine-tune the translation model to determine the target translation model.
[0037] In the embodiment of the present application, the target translation model is a translation model adapted to a specific generation task.
[0038] Specifically, the electronic device determines an input sequence according to the translation model; then, the electronic device determines an output result based on the input sequence and the translation model; at the same time, the electronic device calculates a corresponding loss function according to the output result and a preset true label; the electronic device determines parameter information according to the loss function; the electronic device screens the translation model according to the parameter information and the translation model, and determines that the translation model suitable for a specific generation task is the target translation model.
[0039] The embodiment of the present application provides a large-scale language model code translation method based on prompt fine-tuning, which obtains the source language code for translation, thereby providing basic data for subsequent steps; then, the source language code is preliminarily translated to obtain a preliminary translation result, thereby simplifying and translating the source language code to facilitate subsequent steps; based on the preliminary translation result, a pseudo-label is determined, thereby providing additional supervision information during the training process; then, iterative self-training is performed based on the pseudo-label and the source language code to determine the translation model, and the performance of the translation model is improved through iterative self-training; the translation model is fine-tuned to obtain a target translation model, and the translation model after fine-tuning can better meet the needs of specific generation tasks, improve translation quality and task completion, thereby improving language model performance.
[0040] The source language code is preliminarily translated to determine the preliminary translation result, which also includes: obtaining a masked language model across programming languages; determining a pre-trained model based on the source language code and the masked language model; and determining a potential representation based on the pre-trained model.
[0041] Specifically, the electronic device pre-trains the masked language model across programming languages and the source language code transmitted by the technician to obtain a pre-trained model; then, the electronic device initializes the pre-trained model to obtain a latent representation; wherein the pre-trained model initialization ensures that code snippets with similar meanings are mapped to the same latent representation in different programming languages.
[0042] The source language code is preliminarily translated to determine the preliminary translation result, including: adding noise to the source language code to obtain a damaged source language code; determining a predicted original source language code based on the damaged source language code and the potential representation; inputting the predicted original source language code into a preset source-to-target model to determine the target language code; determining a back-translated source language code based on the target language code and the preset target-to-source model; determining an adaptation model based on the back-translated source language code and the target language code; training the adaptation model to determine a training model; and determining the preliminary translation result based on the source language code and the training model.
[0043] Specifically, the electronic device uses a pre-trained cross-programming language model as the initialization of the encoder. For the decoder, since the decoder involves a source attention mechanism, some of its parameters need to be randomly initialized; then the electronic device performs a denoising auto-encoding target, that is, the model is trained to predict the original input based on the damaged input (source code with noise added); the electronic device uses random masking, deletion and shuffling of input tags to create noise, and then the electronic device adds noise to the source language code to obtain a damaged source language code; the electronic device determines the corresponding predicted original source language code based on the damaged source language code and the potential representation; wherein the first tag input to the decoder is a special tag indicating the target programming language; the electronic device connects The source-to-target model and the preset target-to-source model are trained together, and pseudo-parallel data is generated by back translation; the electronic device inputs the predicted original source language code into the preset source-to-target model to obtain the target language code; the electronic device determines the back-translated source language code according to the target language code and the preset target-to-source model; then, the electronic device determines the adaptation model according to the back-translated source language code and the target language code; the electronic device alternately performs denoising auto-encoding and back-translation target training on the adaptation model to obtain a training model; the electronic device inputs the source language code into the training model to translate the source language code to obtain a preliminary translation result; since the decoder contains additional parameters related to the source attention mechanism, these parameters are randomly initialized after the encoder is initialized.
[0044] Determining a pseudo label according to the preliminary translation result includes: determining a target language pseudo code according to the preliminary translation result and the source language code; testing the target language pseudo code and determining the target language pseudo code that passes the test as the pseudo code; and determining a pseudo label according to the pseudo code.
[0045] Specifically, the electronic device uses an existing translation model to translate the source language code and generates a preliminary translation result in the target language; the electronic device uses a unit test between the generated target language function and the source language function for verification, and the electronic device determines the target language pseudocode that passes the test as the pseudocode; and the electronic device combines the pseudocode and regards the samples that pass the unit test as high-quality pseudo-labels; at the same time, the electronic device uses the verified pseudo-labels to build a new parallel data set, and then retrains the translation model with the parallel data set; and repeats this process until the model performance converges.
[0046] Determining the pseudo-labels according to the preliminary translation results also includes: determining quality assessment indicators and screening criteria according to the preliminary translation results; if the preliminary translation results successfully match the screening criteria, determining that the preliminary translation results are high-quality translations; determining the screening pseudo-labels according to the high-quality translation; and determining the pseudo-labels according to the quality assessment and the screening pseudo-labels.
[0047] Specifically, in each training step, the current model is used to dynamically generate a translation of the target language, that is, a preliminary translation result, without pre-generating the entire parallel data set; then, the electronic device introduces a cache mechanism to store previously generated samples, and at the same time, based on the preliminary translation result, determines the corresponding quality assessment indicators and screening criteria; then, the electronic device successfully matches the preliminary translation result with the screening criteria, and if the preliminary translation result successfully matches the screening criteria, it means that the preliminary translation result is a high-quality translation; the electronic device determines the corresponding screening pseudo-label based on the high-quality translation; the electronic device determines the pseudo-label based on the quality assessment and the screening pseudo-label.
[0048] Iterative self-training is performed according to the pseudo-labels and the source language codes to determine the translation model, including: determining a training set according to the pseudo-labels and the source language codes; and determining the translation model according to the training set.
[0049] Specifically, the electronic device constructs a new training set using the verified pseudo-labels and the source language code, and then the electronic device retrains the translation model using the training set; this process is repeated until the performance of the translation model converges.
[0050] The translation model is fine-tuned to determine the target translation model, including: determining an input sequence according to the translation model; determining an output result based on the input sequence and the translation model; determining a loss function according to the output result and a preset true label; determining parameter information according to the loss function; and determining a target translation model according to the parameter information and the translation model.
[0051] In the embodiment of the present application, the input sequence is an input sequence with a prefix.
[0052] Specifically, the electronic device introduces a prefix and adds it to the beginning of the input sequence of the language model to obtain the input sequence; the prefix is a trainable matrix whose parameters are updated through training; the electronic device determines the output result based on the input sequence and the translation model, wherein the prefix may contain information related to the task in the generation task; the electronic device calculates the loss function according to the output result and the preset true label, and the influence of the prefix needs to be considered when calculating the activation layer and the loss function of the language model; for the activation of the prefix in the sequence, the electronic device directly uses the value of the prefix matrix; for the part of the sequence other than the prefix, the electronic device uses the original calculation method of the language model; in the process of the electronic device training the model, the parameter information is determined according to the loss function, wherein the update of the parameter information only updates the parameter information of the prefix matrix, and keeps the other parameter information of the language model unchanged; the electronic device determines the target translation model according to the parameter information and the translation model.
[0053] Reference Figure 2The large-scale language model code translation device 20 based on prompt fine-tuning may specifically include: a source language code acquisition module 201, a preliminary translation result determination module 202, a pseudo-label determination module 203, a translation model determination module 204 and a target translation model determination module 205, wherein: The source language code acquisition module 201 is used to acquire the source language code; A preliminary translation result determination module 202 is used to perform a preliminary translation on the source language code and determine a preliminary translation result; A pseudo-label determination module 203 is used to determine a pseudo-label according to the preliminary translation result; A translation model determination module 204 is used to perform iterative self-training based on pseudo labels and source language codes to determine a translation model; The target translation model determination module 205 is used to fine-tune the translation model and determine the target translation model, where the target translation model is a translation model adapted to a specific generation task.
[0054] A possible implementation of the embodiment of the present application is a large language model code translation device 20 based on prompt fine-tuning, further comprising: a masked language model module, a pre-trained model determination module and a potential representation determination module, wherein: The masked language model module is used to obtain masked language models across programming languages; A pre-training model determination module, used to determine a pre-training model according to a source language code and a mask language model; The latent representation determination module is used to determine the latent representation based on the pre-trained model.
[0055] In a possible implementation of the embodiment of the present application, the preliminary translation result determination module 202 includes: a damaged source language code determination unit, a predicted original source language code determination unit, a target language code determination unit, a back-translated source language code determination unit, an adaptation model determination unit, a training model determination unit, and a preliminary translation result determination unit, wherein: A damaged source language code determining unit, used for adding noise to the source language code to obtain a damaged source language code; A predicted original source language code determination unit, used for determining a predicted original source language code according to the damaged source language code and the potential representation; a target language code determination unit, configured to input the predicted original source language code into a preset source-to-target model to determine the target language code; A back-translation source language code determination unit, used to determine the back-translation source language code according to the target language code and a preset target-to-source model; An adaptation model determination unit, used to determine an adaptation model according to the back-translated source language code and the target language code; A training model determination unit, used for training the adaptation model and determining the training model; The preliminary translation result determination unit is used to determine the preliminary translation result according to the source language code and the training model.
[0056] In a possible implementation of the embodiment of the present application, the pseudo-label determination module 203 includes: a target language pseudo-code determination unit, a pseudo-code determination unit, and a pseudo-label determination unit, wherein: A target language pseudo code determination unit, used to determine the target language pseudo code according to the preliminary translation result and the source language code; A pseudocode determination unit, used for testing the target language pseudocode and determining that the target language pseudocode that passes the test is the pseudocode; The pseudo-label determination unit is used to determine the pseudo-label according to the pseudo-code.
[0057] In a possible implementation of the embodiment of the present application, the pseudo-label determination module 203 includes: a standard determination unit, a high-quality translation determination unit, a pseudo-label screening determination unit, and a pseudo-label determination unit, wherein: The standard determination unit is used to determine the quality assessment indicators and screening criteria based on the preliminary translation results; A high-quality translation determination unit, configured to determine that the preliminary translation result is a high-quality translation if the preliminary translation result successfully matches the screening criteria; A screening pseudo-label determination unit, used for determining screening pseudo-labels based on high-quality translation; The pseudo-label determination unit is used to determine the pseudo-label according to quality evaluation and pseudo-label screening.
[0058] In a possible implementation of the embodiment of the present application, the translation model determination module 204 includes: a training set determination unit and a translation model determination unit, wherein: A training set determination unit, used for determining a training set according to the pseudo-label and the source language code; The translation model determination unit is used to determine the translation model according to the training set.
[0059] In a possible implementation of the embodiment of the present application, the target translation model determination module 205 includes: an input sequence determination unit, an output result determination unit, a loss function determination unit, a parameter information determination unit and a target translation model determination unit, wherein: An input sequence determination unit, used for determining an input sequence according to a translation model, wherein the input sequence is an input sequence with a prefix; An output result determination unit, used to determine an output result based on an input sequence and a translation model; A loss function determination unit, used to determine a loss function according to an output result and a preset true label; A parameter information determination unit, used to determine parameter information according to the loss function; The target translation model determination unit is used to determine the target translation model according to the parameter information and the translation model.
[0060] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0061] The present application also introduces an electronic device from the perspective of a physical device, such as Figure 3 As shown, Figure 3 The electronic device 30 shown includes: a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, such as through a bus 302. Optionally, the electronic device 30 may also include a transceiver 304. It should be noted that in actual applications, the transceiver 304 is not limited to one, and the structure of the electronic device 30 does not constitute a limitation on the embodiments of the present application.
[0062] The processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.
[0063] The bus 302 may include a path to transmit information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0064] The memory 303 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.
[0065] The memory 303 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the contents shown in the above method embodiment.
[0066] The electronic devices include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server, etc. Figure 3 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0067] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.
[0068] The above are only some implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A large language model code translation method based on prompt fine-tuning, characterized in that: include: Get the source language code; Performing a preliminary translation of the source language code to determine a preliminary translation result; Determining a pseudo label according to the preliminary translation result; Perform iterative self-training according to the pseudo-labels and the source language codes to determine a translation model; The translation model is fine-tuned to determine a target translation model, where the target translation model is a translation model adapted to a specific generation task.
2. A large language model code translation method based on prompt fine-tuning according to claim 1, characterized in that: The preliminary translation of the source language code to determine the preliminary translation result also includes: Get masked language models across programming languages; Determine a pre-trained model according to the source language code and the mask language model; Based on the pre-trained model, a latent representation is determined.
3. A large language model code translation method based on prompt fine-tuning according to claim 2, characterized in that: The preliminary translation of the source language code to determine the preliminary translation result includes: Adding noise to the source language code to obtain a damaged source language code; Determining a predicted original source language code based on the damaged source language code and the latent representation; Inputting the predicted original source language code into a preset source-to-target model to determine a target language code; Determine the back-translation source language code based on the target language code and the preset target-to-source model; Determining an adaptation model according to the back-translated source language code and the target language code; Training the adaptation model to determine a training model; Determine the preliminary translation result based on the source language code and the training model.
4. A large language model code translation method based on prompt fine-tuning according to claim 1, characterized in that: Determining a pseudo label according to the preliminary translation result includes: Determining a target language pseudo code according to the preliminary translation result and the source language code; Testing the target language pseudocode to determine that the target language pseudocode that passes the test is a pseudocode; The pseudo label is determined according to the pseudo code.
5. A large language model code translation method based on prompt fine-tuning according to claim 1, characterized in that: The determining of the pseudo-label according to the preliminary translation result further includes: Determine quality assessment indicators and screening criteria based on the preliminary translation results; If the preliminary translation result successfully matches the screening criteria, determining that the preliminary translation result is a high-quality translation; Determining, based on the high-quality translation, a screening pseudo-label; The pseudo-label is determined according to the quality evaluation and the screening pseudo-label.
6. A large language model code translation method based on prompt fine-tuning according to claim 1, characterized in that: The iterative self-training according to the pseudo-label and the source language code to determine the translation model includes: Determine a training set according to the pseudo-label and the source language code; The translation model is determined according to the training set.
7. A large language model code translation method based on prompt fine-tuning according to claim 1, characterized in that: The fine-tuning of the translation model to determine the target translation model includes: Determine an input sequence according to the translation model, wherein the input sequence is an input sequence with a prefix; Determine an output result based on the input sequence and the translation model; Determine a loss function according to the output result and the preset true label; Determine parameter information based on the loss function; The target translation model is determined according to the parameter information and the translation model.
8. A large language model code translation device based on prompt fine-tuning, characterized in that: include: A source language code acquisition module, used to acquire source language code; A preliminary translation result determination module, used to perform preliminary translation on the source language code and determine a preliminary translation result; A pseudo-label determination module, used to determine a pseudo-label according to the preliminary translation result; A translation model determination module, used for iterative self-training based on the pseudo-labels and source language codes to determine the translation model; The target translation model determination module is used to fine-tune the translation model to determine the target translation model, where the target translation model is a translation model adapted to a specific generation task.
9. An electronic device, characterized in that: The electronic device includes: at least one processor; Memory; At least one application, wherein at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute a large language model code translation method based on prompt fine-tuning as described in any one of claims 1 to 7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed in a computer, the computer is caused to execute the large language model code translation method based on prompt fine-tuning as described in any one of claims 1 to 7.
Citation Information
Cited By
Prompt strategy activation-based multi-language large model translation performance optimization method and system
CN119903855A
Method and system for optimizing translation performance of a multilingual large model activated based on a prompting strategy
CN119903855B
Large model code translation method and device fusing code functions and styles
CN120335814A
Heterogeneous programming language translation method and device based on large model and medium
CN121638270A
