Large model code translation reinforcement learning training method based on test example generation

Abstract

The application provides a large model code translation reinforcement learning training method based on test sample generation, and relates to the technical field of artificial intelligence. The method comprises the following steps: performing mutation processing on source language code to be translated, and generating a plurality of mutated code instances using a large model; generating a unit test sample set and target language code based on the source language code using a parameterized strategy model; applying the unit test sample set to the source language code for execution verification and defining a unit test sample reward; comparing and executing the target language code and the source language code according to standard test samples and test samples that can pass the verified source language code, and defining a target code reward; combining the rewards to form a reinforcement learning reward signal, updating the model according to a group relative strategy optimization method, and completing code translation. The application realizes the continuous reinforcement of the model generation capability by constructing a training mechanism for the collaborative optimization of test generation and code translation.

Description

Technical Field

[0001] This invention relates to the fields of large code models and reinforcement learning in artificial intelligence, and in particular to a reinforcement learning training method for large model code translation based on test sample generation. Background Technology

[0002] With the increasing scale and complexity of software systems, the demand for code migration, refactoring, and reuse across different programming languages ​​is growing, making code translation technology an important research direction in software engineering and program analysis. In recent years, large-scale pre-trained language models have made significant progress in source code understanding and generation tasks, and neural network-based code translation methods are gradually replacing traditional rule-based or template-based conversion methods. Existing methods typically model code translation as a sequence-to-sequence or encoder-decoder problem, using supervised learning on large-scale parallel code datasets to enable the model to learn the mapping relationship between the source and target languages. Some studies further introduce static analysis features, abstract syntax trees, or intermediate representations to improve the syntactic correctness and semantic consistency of the translation results.

[0003] However, existing supervised learning-based code translation methods heavily rely on test examples or manually labeled results provided in the training data. Their evaluation and optimization processes typically depend on limited and relatively simple unit test sets, making it difficult to fully characterize the program's real-world behavior under boundary conditions, abnormal inputs, or complex execution paths. To address these issues, existing techniques have attempted to introduce reinforcement learning frameworks, using code execution results or test pass rates as reward signals to optimize the target code generated by the model. For example, using whether the program passes a given test set as reward feedback. However, these methods usually only focus on whether the target code passes existing test examples, failing to effectively address the problem of insufficient discriminative power of the test examples themselves. This results in erroneous code potentially receiving high rewards during training.

[0004] Furthermore, in existing technologies, test cases are typically designed manually or derived directly from datasets, lacking systematic modeling of the "difficulty" and "discriminative power" of the test cases. This makes it difficult to distinguish between semantically equivalent correct implementations and erroneous implementations with hidden defects in data type, time complexity, or boundary handling. Therefore, how to simultaneously improve the discriminative power of test cases and the semantic consistency of the target code during code translation training remains a pressing technical problem to be solved in this field.

[0005] Current code translation technologies primarily rely on the Transformer architecture to build encoder-decoder models. These models treat source language code as sequential input, learn the syntactic structure and contextual dependencies within the code through self-attention mechanisms, and generate corresponding target language code. These methods are typically trained on large-scale parallel code datasets, utilizing maximum likelihood estimation or cross-entropy loss functions to enable the model to learn the mapping relationship between source and target code. Compared to earlier rule-based or template-based methods, these models offer advantages in syntactic diversity and cross-language adaptability, and have been widely applied in scenarios such as program migration and code refactoring.

[0006] Building upon this foundation, to further improve the accuracy of translation results, existing technologies commonly employ supervised fine-tuning of large models or the introduction of reinforcement learning for optimization. Supervised fine-tuning methods typically rely on manual annotation or reference answers provided in the dataset, improving model performance by minimizing the difference between the generated results and the reference code. Some technologies further incorporate reinforcement learning frameworks, using whether the code passes a given unit test example, or the proportion of test examples that pass, as reward signals to optimize the model's generation strategy, aiming to alleviate the limitations of relying solely on static text similarity.

[0007] However, the aforementioned existing technologies still have significant shortcomings. On the one hand, supervised fine-tuning and reinforcement learning processes heavily rely on existing test examples, which are limited in number and relatively easy to understand, making it difficult to cover complex inputs, boundary conditions, or performance-related issues, thus hindering the effective differentiation of erroneous code. On the other hand, reward signals in reinforcement learning typically only reflect whether the target code passes the test, without evaluating the discriminative ability of the test examples themselves. This can lead to the model tending to generate code that is "superficially correct but semantically flawed" during training. Therefore, existing technologies still have room for improvement in enhancing the semantic consistency and robustness of code translation. Summary of the Invention

[0008] To address the technical problems of insufficient test case discrimination ability, inadequate reward signal representation, and difficulty in effectively improving the semantic consistency of target code during existing code translation reinforcement learning training processes, this invention provides a method and apparatus for training large-scale code translation reinforcement learning based on test case generation. The technical solution is as follows: On the one hand, a reinforcement learning training method for large model code translation based on test sample generation is provided. This method is implemented by a large model code translation reinforcement learning training device, and includes: S1. Obtain the source language code to be translated, perform mutation processing on the source language code, and generate multiple mutated code instances based on the mutation processing results using a large model to obtain a mutated code set.

[0009] S2. Based on the source language code, a parameterized strategy model is used to generate a set of unit test cases and target language code.

[0010] S3. Apply the set of unit test cases to the source language code for execution verification, and define unit test case rewards based on the verification results.

[0011] S4. Obtain standard test cases from the training dataset. Based on the standard test cases and the test cases in the unit test case set that have been verified to be passable by the source language code, compare and execute the target language code with the source language code to obtain the proportion of test cases with consistent output. Define the target code reward based on the proportion.

[0012] S5. Combine the unit test case reward with the target code reward to form a reinforcement learning reward signal. Update the parameterized policy model based on the reinforcement learning reward signal and the group relative policy optimization method to complete the code translation.

[0013] Optionally, S1 includes: Obtain the source language code to be translated, perform mutation processing on the source language code based on predefined semantically related mutation rules, generate multiple mutated code instances using a large model based on the mutation processing results, and construct a mutated code set based on the multiple mutated code instances.

[0014] Among them, predefined semantically related variation rules are used to introduce potential semantic or performance defects while maintaining the overall similarity of the program structure of the source language code.

[0015] Optionally, S3 includes: The unit test case set is applied to the source language code for execution verification to determine whether the source language code cannot be compiled, executed, or produces valid output under any test case in the unit test case set.

[0016] If so, the unit test case set is determined not to meet the constraints, and the reward for the unit test case set is set to 0.

[0017] If not, the source language code is determined to pass the unit test case set, and the unit test case set is applied to each variant code in the variant code set for execution verification. The unit test case reward is defined based on the proportion of variant codes that are successfully distinguished by the test cases in the verification results of all variant codes.

[0018] Optionally, S4 includes: Obtain standard test cases from the training dataset and test cases from the unit test case set that have been verified to pass the source language code.

[0019] The target language code and the source language code are compared and executed based on the standard test cases and the passed test cases to obtain the execution results of the target language code and the source language code.

[0020] By determining whether the execution result of the target language code is consistent with the execution result of the source language code under the same test case, the proportion of test cases with consistent output is obtained among all test cases, and a reward for the target code is defined based on the proportion.

[0021] Optionally, the parameterized policy model in S5 is updated based on the reinforcement learning reward signal and the group-relative policy optimization method, including: The set of unit test cases generated based on the source language code using a parameterized strategy model, along with the target language code, will be used as candidate outputs.

[0022] The comprehensive reward for each candidate output is calculated based on the reinforcement learning reward signal. The average reward within the group is calculated based on the comprehensive reward of all candidate outputs. The average reward within the group is used as the baseline to calculate the relative advantage within the group for each candidate output, and the parameterized policy model is updated accordingly.

[0023] On the other hand, a large-model code translation reinforcement learning training device based on test sample generation is provided. This device is applied to the large-model code translation reinforcement learning training method based on test sample generation. The device includes: The source code semantic mutation construction module is used to obtain the source language code to be translated, perform mutation processing on the source language code, and generate multiple mutated code instances based on the mutation processing results using a large model to obtain a mutated code set.

[0024] The code translation and test case generation module is used to generate a set of unit test cases and target language code based on the source language code using a parameterized strategy model.

[0025] The unit test case discrimination capability evaluation module is used to apply the set of unit test cases to the source language code for execution verification, and define unit test case rewards based on the verification results.

[0026] The target code semantic consistency evaluation module is used to obtain standard test cases from the training dataset. Based on the standard test cases and the test cases in the unit test case set that have been verified to be passable by the source language code, the target language code and the source language code are compared and executed to obtain the proportion of test cases with consistent output. The target code reward is defined according to the proportion.

[0027] The reinforcement learning module based on group relative policy optimization combines unit test case rewards with target code rewards to form a reinforcement learning reward signal. The parameterized policy model is then updated based on the reinforcement learning reward signal and the group relative policy optimization method to complete the code translation.

[0028] Optionally, the source code semantic variation constructor module is further used for: Obtain the source language code to be translated, perform mutation processing on the source language code based on predefined semantically related mutation rules, generate multiple mutated code instances using a large model based on the mutation processing results, and construct a mutated code set based on the multiple mutated code instances.

[0029] Among them, predefined semantically related variation rules are used to introduce potential semantic or performance defects while maintaining the overall similarity of the program structure of the source language code.

[0030] Optionally, the unit test case discrimination capability evaluation module is further used for: The unit test case set is applied to the source language code for execution verification to determine whether the source language code cannot be compiled, executed, or produces valid output under any test case in the unit test case set.

[0031] If so, the unit test case set is determined not to meet the constraints, and the reward for the unit test case set is set to 0.

[0032] If not, the source language code is determined to pass the unit test case set, and the unit test case set is applied to each variant code in the variant code set for execution verification. The unit test case reward is defined based on the proportion of variant codes that are successfully distinguished by the test cases in the verification results of all variant codes.

[0033] Optionally, the target code semantic consistency evaluation module is further used for: Obtain standard test cases from the training dataset and test cases from the unit test case set that have been verified to pass the source language code.

[0034] The target language code and the source language code are compared and executed based on the standard test cases and the passed test cases to obtain the execution results of the target language code and the source language code.

[0035] By determining whether the execution result of the target language code is consistent with the execution result of the source language code under the same test case, the proportion of test cases with consistent output is obtained among all test cases, and a reward for the target code is defined based on the proportion.

[0036] Optionally, the reinforcement learning module based on group-relative policy optimization is further used for: The set of unit test cases generated based on the source language code using a parameterized strategy model, along with the target language code, will be used as candidate outputs.

[0037] The comprehensive reward for each candidate output is calculated based on the reinforcement learning reward signal. The average reward within the group is calculated based on the comprehensive reward of all candidate outputs. The average reward within the group is used as the baseline to calculate the relative advantage within the group for each candidate output, and the parameterized policy model is updated accordingly.

[0038] On the other hand, a large model code translation reinforcement learning training device is provided, the large model code translation reinforcement learning training device comprising: a processor; a memory, the memory storing computer-readable instructions, which, when executed by the processor, implement any of the methods in the large model code translation reinforcement learning training method based on test samples described above.

[0039] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, the at least one instruction being loaded and executed by a processor to implement any of the above-described methods for reinforcement learning training of large model code translation based on test samples.

[0040] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: This invention proposes a method and apparatus for training large-scale model code translation reinforcement learning based on test sample generation. Compared with existing technologies, this invention significantly improves the discriminative ability of test samples and the effectiveness of reward signals by introducing a joint generation and evaluation mechanism of unit test samples and target code during the code translation reinforcement learning training process, thereby enhancing the model's ability to identify complex inputs, boundary conditions, and hidden semantic errors. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart of a reinforcement learning training method for large model code translation based on test samples, provided by an embodiment of the present invention. Figure 2 This is an overall framework diagram of a large model code translation reinforcement learning training method based on test sample generation provided by an embodiment of the present invention; Figure 3 This is a schematic diagram of the intra-group relative strategy optimization principle provided in an embodiment of the present invention; Figure 4 This is a block diagram of a large model code translation reinforcement learning training device based on test sample generation provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a large model code translation reinforcement learning training device provided in an embodiment of the present invention. Detailed Implementation

[0043] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0044] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0045] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, they convey the same meaning.

[0046] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0047] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0048] This invention provides a large-model code translation reinforcement learning training method based on test sample generation. This method can be implemented using a large-model code translation reinforcement learning training device, which can be a terminal or a server. Figure 1 The flowchart shown is for a large model code translation reinforcement learning training method based on test samples. The processing flow of this method may include the following steps: S1. Obtain the source language code to be translated, perform mutation processing on the source language code, and generate multiple mutated code instances based on the mutation processing results using a large model to obtain a mutated code set.

[0049] Optionally, step S1 above may include: Obtain the source language code to be translated, perform mutation processing on the source language code based on predefined semantically related mutation rules, generate multiple mutated code instances using a large model based on the mutation processing results, and construct a mutated code set based on the multiple mutated code instances.

[0050] Among them, predefined semantically related variation rules are used to introduce potential semantic or performance defects while maintaining the overall similarity of the program structure of the source language code.

[0051] One feasible implementation method is, for example Figure 2 As shown, in the construction of semantic mutation in source code, the source language code to be translated is first obtained, denoted as... Based on predefined semantically related mutation rules, the source code is mutated, and multiple mutated code instances are generated using a large model, forming a mutated code set: , This indicates the number of mutated codes. Mutation rules are used to introduce potential semantic or performance defects while maintaining the overall similarity of the program structure. These include, but are not limited to, data type narrowing or expansion, time complexity degradation, missing boundary condition handling, integer overflow, or precision loss-related mutations.

[0052] The variant code constructed in the above manner may behave the same as the source code under simple test conditions, but it is prone to produce different outputs under complex or extreme input conditions, thus providing a basis for comparison for subsequent evaluation of the test sample discrimination capability.

[0053] S2. Based on the source language code, a parameterized strategy model is used to generate a set of unit test cases and target language code.

[0054] In one feasible implementation, a parameterized strategy model is adopted in the joint code translation and test case generation process. In the given source code Under these conditions, a set of unit test cases and target language code are generated simultaneously. The set of unit test cases is represented as follows: , This indicates the number of test cases. Each unit test case contains only input data. The generated target language code is represented as follows: This joint generation enables the model to proactively construct test cases that can distinguish between different implementations while generating the target code, providing a foundation for subsequent reward calculations.

[0055] S3. Apply the set of unit test cases to the source language code for execution verification, and define unit test case rewards based on the verification results.

[0056] Optionally, step S3 above may include: The unit test case set is applied to the source language code for execution verification to determine whether the source language code cannot be compiled, executed, or produces valid output under any test case in the unit test case set.

[0057] If so, the unit test case set is determined not to meet the constraints, and the reward for the unit test case set is set to 0.

[0058] If not, the source language code is determined to pass the unit test case set, and the unit test case set is applied to each variant code in the variant code set for execution verification. The unit test case reward is defined based on the proportion of variant codes that are successfully distinguished by the test cases in the verification results of all variant codes.

[0059] In one feasible implementation, in the evaluation of the discrimination capability of unit test cases, the sample set is first... Applied to source code Perform execution verification. If the source code fails to compile or execute under any test case, or fails to produce valid output, then the test case set is deemed not to meet the constraints, and the corresponding reward is... Once the source code passes the test case set, the test case set is further applied to the mutated code set. For each variant code in the test, its execution results are statistically analyzed. If a variant code fails to compile, encounters runtime exceptions, outputs inconsistent with the source code, times out, or exceeds resource limits during testing, it is considered successfully distinguished by the test samples. Based on the above statistical results, a unit test sample reward is defined. for: (1) In the formula, This indicates the number of mutated codes. This reward quantifies the test cases' ability to distinguish between erroneous implementations; a higher reward value indicates a stronger ability to discriminate between erroneous implementations.

[0060] S4. Obtain standard test cases from the training dataset. Based on the standard test cases and the test cases in the unit test case set that have been verified to be passable by the source language code, compare and execute the target language code with the source language code to obtain the proportion of test cases with consistent output. Define the target code reward based on the proportion.

[0061] Optionally, step S4 above may include: Obtain standard test cases from the training dataset and test cases from the unit test case set that have been verified to pass the source language code.

[0062] The target language code and the source language code are compared and executed based on the standard test cases and the passed test cases to obtain the execution results of the target language code and the source language code.

[0063] By determining whether the execution result of the target language code is consistent with the execution result of the source language code under the same test case, the proportion of test cases with consistent output is obtained among all test cases, and a reward for the target code is defined based on the proportion.

[0064] In one feasible implementation, the semantic consistency evaluation of the target code comprehensively utilizes both the standard test cases provided with the dataset and the test cases generated by the model whose source code has been verified to be passable, to evaluate the target language code. With source code A comparative execution is performed. By comparing the output results of both methods under each test case, the proportion of test cases with consistent output is calculated, and a target code reward is defined based on this proportion. : (2) This reward is used to characterize the degree of semantic consistency between the target code and the source code under the enhanced test set.

[0065] S5. Combine the unit test case reward with the target code reward to form a reinforcement learning reward signal. Update the parameterized policy model based on the reinforcement learning reward signal and the group relative policy optimization method to complete the code translation.

[0066] Optionally, updating the parameterized policy model in S5 based on the reinforcement learning reward signal and the group-relative policy optimization method may include: The set of unit test cases generated based on the source language code using a parameterized strategy model, along with the target language code, will be used as candidate outputs.

[0067] The comprehensive reward for each candidate output is calculated based on the reinforcement learning reward signal. The average reward within the group is calculated based on the comprehensive reward of all candidate outputs. The average reward within the group is used as the baseline to calculate the relative advantage within the group for each candidate output, and the parameterized policy model is updated accordingly.

[0068] In one feasible implementation, in reinforcement learning based on group-relative policy optimization, the unit test examples are rewarded. With target code reward These signals are combined to form the final reinforcement learning reward signal: Or adopt a weighted form , This indicates the reward weight for unit test cases. This represents the reward weight for the target code. During training, multiple sets of candidate outputs are generated for the same source code sample, and the relative advantage within each candidate output is calculated. A group-relative strategy optimization algorithm is used to update the model parameters, thereby ensuring training stability while guiding the model to gradually improve the semantic consistency of code translation and the ability to discriminate test cases.

[0069] Figure 3 This is a schematic diagram of the training principle of GRPO (Group Relative Policy Optimization) used in this invention. It shows that under the same source code conditions, multiple candidate outputs containing target code and unit test cases are sampled from the current policy. The comprehensive reward corresponding to each candidate output is compared within the group. The average reward within the group is used as the baseline to calculate the relative advantage, and the policy update is completed without introducing an additional value function.

[0070] This invention addresses the problems of current code translation models' heavy reliance on static and insufficiently discriminative test examples during training, leading to difficulties in effectively distinguishing erroneous code, insufficient reinforcement learning reward signals, and limited improvement in model semantic consistency. It designs a code translation reinforcement learning training method based on unit test example generation. This method introduces a joint generation mechanism of unit test examples and target code, and constructs a comprehensive reward signal based on the distinguishing ability of unit test examples to source code and its variant implementations, and the semantic consistency between the target code and source code. This signal is then combined with a group-based relative policy optimization method to train the model, forming a training framework that co-optimizes test example generation and code translation, effectively enhancing code translation capabilities. The entire training framework is optimized to improve the discriminative ability of test examples and the semantic consistency of code translation, providing more effective and stable training feedback for the code translation model without the need for additional manually labeled test data. Traditional reinforcement learning code translation methods only use whether the target code passes existing test examples as the reward criterion, making it difficult to identify erroneous implementations with defects in data types, boundary conditions, or complex execution paths. This invention guides the model to simultaneously generate unit test cases and target code, and combines the test cases to comprehensively evaluate the ability to distinguish between source code and its variant implementations, thereby constructing a more sufficient reward signal and significantly improving the accuracy and robustness of the code translation model in complex scenarios.

[0071] This invention proposes a method and apparatus for training large-scale model code translation reinforcement learning based on test sample generation. Compared with existing technologies, this invention significantly improves the discriminative ability of test samples and the effectiveness of reward signals by introducing a joint generation and evaluation mechanism of unit test samples and target code during the code translation reinforcement learning training process. This enhances the model's ability to identify complex inputs, boundary conditions, and hidden semantic errors.

[0072] Figure 4 This is a block diagram illustrating a large-model code translation reinforcement learning training apparatus based on test sample generation, according to an exemplary embodiment. The apparatus is used for a large-model code translation reinforcement learning training method based on test sample generation. (Refer to...) Figure 4The device includes a source code semantic variation construction module 310, a code translation and test case joint generation module 320, a unit test case discrimination ability evaluation module 330, a target code semantic consistency evaluation module 340, and a reinforcement learning module 350 based on group relative policy optimization. Among them: The source code semantic mutation construction module 310 is used to obtain the source language code to be translated, perform mutation processing on the source language code, and generate multiple mutated code instances based on the mutation processing results using a large model to obtain a mutated code set.

[0073] The code translation and test case joint generation module 320 is used to generate a set of unit test cases and target language code based on the source language code and using a parameterized strategy model.

[0074] The unit test case discrimination capability evaluation module 330 is used to apply the set of unit test cases to the source language code for execution verification, and to define unit test case rewards based on the verification results.

[0075] The target code semantic consistency evaluation module 340 is used to obtain standard test cases in the training dataset. Based on the standard test cases and the test cases in the unit test case set that have been verified to be passable by the source language code, the target language code and the source language code are compared and executed to obtain the proportion of test cases with consistent output. The target code reward is defined according to the proportion.

[0076] The reinforcement learning module 350 based on group relative policy optimization is used to combine unit test case rewards and target code rewards to form a reinforcement learning reward signal. The parameterized policy model is updated according to the reinforcement learning reward signal and the group relative policy optimization method to complete the code translation.

[0077] This invention proposes a method and apparatus for training large-scale model code translation reinforcement learning based on test sample generation. Compared with existing technologies, this invention significantly improves the discriminative ability of test samples and the effectiveness of reward signals by introducing a joint generation and evaluation mechanism of unit test samples and target code during the code translation reinforcement learning training process. This enhances the model's ability to identify complex inputs, boundary conditions, and hidden semantic errors.

[0078] Figure 5 This is a schematic diagram of the structure of a large model code translation reinforcement learning training device provided in an embodiment of the present invention, as shown below. Figure 5 As shown, the large model code translation reinforcement learning training device can include the above-mentioned Figure 4 The illustrated large model code translation reinforcement learning training device is generated based on test samples. Optionally, the large model code translation reinforcement learning training device 410 may include a first processor 2001.

[0079] Optionally, the large model code translation reinforcement learning training device 410 may also include a memory 2002 and a transceiver 2003.

[0080] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0081] The following is combined Figure 5 A detailed description of each component of the large model code translation reinforcement learning training device 410 is provided below: The first processor 2001 is the control center of the large model code translation reinforcement learning training device 410. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0082] Optionally, the first processor 2001 can perform various functions of the large model code translation reinforcement learning training device 410 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0083] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 5 CPU0 and CPU1 are shown in the diagram.

[0084] In a specific implementation, as one example, the large model code translation reinforcement learning training device 410 may also include multiple processors, for example... Figure 5 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor or a multi-core processor. Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (such as computer program instructions).

[0085] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0086] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected to the interface circuit of the large model code translation reinforcement learning training device 410. Figure 5 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0087] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0088] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 5 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0089] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be connected to the interface circuit of the reinforcement learning training device 410 via large model code translation. Figure 5 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0090] It should be noted that, Figure 5 The structure of the large model code translation reinforcement learning training device 410 shown in the figure does not constitute a limitation on the router. The actual knowledge structure recognition device may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0091] Furthermore, the technical effect of the large model code translation reinforcement learning training device 410 can be referred to the technical effect of the large model code translation reinforcement learning training method based on test sample generation described in the above method embodiments, and will not be repeated here.

[0092] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0093] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0094] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0095] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0096] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0097] It should be understood that, in various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0098] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0099] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0100] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0101] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0102] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0103] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0104] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A reinforcement learning training method for large model code translation based on test sample generation, characterized in that, The method includes: S1. Obtain the source language code to be translated, perform mutation processing on the source language code, and generate multiple mutated code instances based on the mutation processing results using a large model to obtain a mutated code set; S2. Based on the source language code, a set of unit test cases and target language code are generated using a parameterized strategy model; S3. Apply the set of unit test cases to the source language code for execution verification, and define unit test case rewards based on the verification results; S4. Obtain standard test cases from the training dataset. Based on the standard test cases and the verified source language code in the unit test case set, the target language code and the source language code can be compared and executed through the test cases to obtain the proportion of test cases with consistent output. Define the target code reward based on the proportion. S5. Combine the unit test case reward with the target code reward to form a reinforcement learning reward signal. Update the parameterized policy model based on the reinforcement learning reward signal and the group relative policy optimization method to complete the code translation. The S3 includes: The unit test case set is applied to the source language code for execution verification to determine whether the source language code cannot be compiled, cannot be executed, or cannot produce valid output under any test case in the unit test case set. If so, the unit test case set is determined not to meet the constraints, and the reward of the unit test case set is set to 0. If not, the source language code is determined to pass the unit test case set, and the unit test case set is applied to each variant code in the variant code set for execution verification. The unit test case reward is defined according to the proportion of variant codes that are successfully distinguished by the test cases in the verification results of all variant codes. The S4 includes: The source language code that has been validated in the standard test cases and unit test case sets of the training dataset can be obtained through the test cases; The target language code and the source language code are compared and executed according to the standard test cases and the passed test cases to obtain the execution results of the target language code and the source language code. By determining whether the execution result of the target language code is consistent with the execution result of the source language code under the same test case, the proportion of test cases with consistent output is obtained among all test cases, and a reward for the target code is defined based on the proportion.

2. The reinforcement learning training method for large model code translation based on test sample generation according to claim 1, characterized in that, S1 includes: Obtain the source language code to be translated, perform mutation processing on the source language code based on predefined semantically related mutation rules, generate multiple mutated code instances using a large model based on the mutation processing results, and construct a mutated code set based on the multiple mutated code instances; The predefined semantically related variation rules are used to introduce potential semantic or performance defects while maintaining the overall similarity of the program structure of the source language code.

3. The method for training large model code translation reinforcement learning based on test sample generation according to claim 1, characterized in that, The step S5, updating the parameterized policy model based on the reinforcement learning reward signal and the group-relative policy optimization method, includes: The set of unit test examples generated based on the source language code using a parameterized strategy model and the target language code are used as candidate outputs; The comprehensive reward for each candidate output is calculated based on the reinforcement learning reward signal. The average reward within the group is calculated based on the comprehensive reward of all candidate outputs. The average reward within the group is used as the baseline to calculate the relative advantage within the group for each candidate output, and the parameterized policy model is updated accordingly.

4. A reinforcement learning training device for large model code translation based on test sample generation, wherein the reinforcement learning training device for large model code translation based on test sample generation is used to implement the reinforcement learning training method for large model code translation based on test sample generation as described in any one of claims 1-3, characterized in that, The device includes: The source code semantic mutation construction module is used to obtain the source language code to be translated, perform mutation processing on the source language code, and generate multiple mutated code instances based on the mutation processing results using a large model to obtain a mutated code set. The code translation and test case joint generation module is used to generate a set of unit test cases and target language code based on the source language code using a parameterized strategy model; The unit test case discrimination capability evaluation module is used to apply the set of unit test cases to the source language code for execution verification, and define unit test case rewards based on the verification results. The target code semantic consistency evaluation module is used to obtain standard test cases in the training dataset. Based on the standard test cases and the verified source language code in the unit test case set, the target language code and the source language code can be compared and executed through the test cases to obtain the proportion of test cases with consistent output. The target code reward is defined according to the proportion. The reinforcement learning module based on group relative policy optimization combines unit test case rewards with target code rewards to form a reinforcement learning reward signal. The parameterized policy model is then updated based on the reinforcement learning reward signal and the group relative policy optimization method to complete the code translation.

5. A large-scale model code translation reinforcement learning training device, characterized in that, The large model code translation reinforcement learning training device includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 3.

Images