Method for automated program code translation from original language to target language
Through automated program code translation methods, multiple translation and quality evaluations are performed using language models and test tools, the problems of inefficiency and unreliability of existing code translation methods are solved, and efficient and reliable code translation and quality evaluation are achieved.
Patent Information
- Application Number
- CN202411368485.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-09-28
- Filing Date
- 2024-09-27
- Publication Date
- 2025-04-04
AI Technical Summary
Existing code translation methods have problems such as inefficiency, unreadable code, and unsafe translations, especially when using dedicated code translators and large language models.
An automated program code translation method is adopted, using the language model to translate the program code of the original language into the program code of the target language, and multiple translations are performed by changing the hyperparameters or inputs of the language model. The original and target codes are automatically compared with testing tools, evaluate the code quality, and update the language model through reinforcement learning.
It realizes efficient and reliable code translation, ensuring that the translated code is operational and maintainable in the target language, meets security requirements, and provides code quality evaluation and confidence feedback.
Smart Images
Figure CN119759364A_ABST
Abstract
Description
Technical Field
[0001] There are methods for automated translation of program code from an original language or Source language to a target language or Target language. Background Art
[0002] For example, the translation is done manually, that is, by a programmer. This action is not efficient.
[0003] In addition, there is a special code translator. But these special code translators can only be used for minority source-target language. A problem is that translation does not guarantee that code can be operated with target language. In addition, unreadable code may be produced, and the unreadable code for example does not have maintainability, because translation is carried out mechanically. Finally, there is the following risk, that is, the special code translator adopts the idiom from the source language potentially, and is not suitable for the target language.
[0004] Recent attempts have used large language models (LLMs) or code assistants for code translation. However, problems arise because LLMs and code assistants do not provide guarantees for the correctness of the translated code. Summary of the invention
[0005] A first general aspect of the present disclosure relates to a method for automated program code translation from an original language to a target language.
[0006] The method comprises:
[0007] - translating the original program code in the original language into a target program code in the target language with the aid of a language model,
[0008] - repeatedly performing the translation with changing conditions, e.g. a change in one or more hyperparameters such as a temperature parameter of the language model, a transformation in the original program code and / or a change in the input to the language model,
[0009] - comparing the source program code and the target program code with a test harness, wherein the test harness is automatically generated,
[0010] - evaluating the target program code according to a code quality metric, a test quality metric and / or a number of tests.
[0011] A second general aspect of the present disclosure relates to a method for training a language model set up for automated program code translation from original program code in an original language to target program code in a target language.
[0012] The method comprises:
[0013] - Input the original program code into the language model and generate the predicted target program code,
[0014] - automated verification of the predicted target program code based on formal verification of identical behavior of the source program code and the target program code, tests in the source language, tests on the contract in the source language and / or syntactic and stylistic tests,
[0015] - generating a reward for the language model, wherein a negative reward is generated when a test fails, wherein a positive reward is generated if all tests succeed, and wherein the magnitude of the reward is based on the number of contracts in the original language and the number of tests in the original language,
[0016] -Update the weights of the language model using the reward value.
[0017] A third general aspect of the present disclosure relates to a computer system designed to perform the method according to the first and / or second general aspect (or embodiments thereof).
[0018] A fourth general aspect of the present disclosure relates to a computer program designed to perform the method according to the first general aspect (or its embodiments).
[0019] A fifth general aspect of the present disclosure relates to a computer-readable medium or a signal storing and / or containing a computer program according to the fourth general aspect (or its embodiments).
[0020] The techniques of the first, second, third, fourth and fifth general aspects may, in some cases, have one or more of the following advantages.
[0021] A symbiosis of a language model for generating code (using the creativity of the language model) and verification and validation (V&V) techniques is proposed to safeguard the results of the language model. This powerful or complete automation increases efficiency. V&V techniques are used to ensure that the results are (likely) correct, give feedback to the user on the correctness, confidence and / or quality of the translation, and make the code translation more reliable.
[0022] The present disclosure enables guarantees for automatically translated code. By using a language model such as LLM for translating the code, the translated code is understandable and utilizes the properties of the target language. Advantageously, the resulting code meets properties such as functional equivalence and other properties, especially security requirements. In addition, the quality of the resulting, i.e., translated, code is measured and evaluated.
[0023] The present disclosure can realize code translation that meets safety requirements. Accordingly, the generated code can be run on safety-critical systems such as control devices, especially in the automotive field.
[0024] Certain terms are used in this disclosure in the following manner.
[0025] A “language model” can be, in particular, a large language model (LLM), a neural network, a recurrent neural network (RNN), a transformer model or a code model as a language model specialized for code or a completely general language model that also includes code. It also includes computer languages, such as program code of a computing device, such as a computer. The language of the model can include not only natural languages but also artificial languages such as programming languages.
[0026] "Testing", "verifying" or "comparing original program code and target program code" may include: formal verification of the same behavior of the original program code and the target program code, for example by means of bounded model checking, testing in the original language, testing on contracts in the original language and / or syntactic and stylistic testing, fuzz testing, mutation of the input of the test harness, derivation from contracts in the original language and / or target language and / or derivation from language models.
[0027] A "test harness" or test framework includes a compilation of software and test data, which is used to perform automated system testing of programs under various environmental conditions. A test harness typically includes a test execution engine, which is responsible for processing the test logic, and a test data repository or database, which contains test scripts, test programs and other test resources. Here, the test harness is automatically generated, for example, by adding differentiated tests to the database. The initiation of the test can start from a given or pre-completed test in the test database. The system can also automatically generate tests.
[0028] Here, the data may be software code, including test cases and tools plus additional (natural language) descriptions of functional modes or validity ranges. Exemplarily, C is described here as the source language and Rust is described as the target language, other combinations are possible. Translating C to Rust is interesting because Rust provides features in the field of safety-critical systems, but there is a lot of legacy code in other languages (especially C).
[0029] "Contracts" are an integral part of contract-based programming or design by contract ("Design by contract"). This is a concept of software development, the goal of which is to optimize the interaction of individual program modules by defining formal contracts for use interfaces that go beyond their static definition.
[0030] "Fuzz Testing" or "Fuzz-Testing" is the automated process of sending randomly generated inputs from a fuzzer to a target or target program and observing the target's response.
[0031] "Fuzzers" or "fuzz testing engines" are programs that automatically generate inputs. They are therefore not necessarily connected to the software to be tested and do not perform instrumentation. However, they have the ability to instrument the code, generate test cases and execute the program to be tested. Known examples are afl and libfuzzer.
[0032] A "test case" is a specific test traversal from a test harness or fuzz test and specific inputs. To ensure reproducibility, interesting traversals (new code paths found or abnormal termination) are stored. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] Figure 1 is a flow chart illustrating the disclosed technique for performing automated program code translation.
[0034] Figure 2 A system is schematically shown in which the techniques of the present disclosure may be used for automated program code translation.
[0035] Figure 3 A system is schematically shown in which the techniques of the present disclosure may be used for automated program code translation.
[0036] Figure 4 is a flow chart illustrating the disclosed technique for training a language model. DETAILED DESCRIPTION
[0037] Figure 1 is a flow chart illustrating a method 10 for automated program code translation from an original language to a target language.
[0038] In a first step, an original program code in an original language is translated 11 into a target program code in a target language with the aid of a language model. The language model is, for example, a large language model (LLM) into which data, such as here a program code, is input (prompt) together with a question, such as here a translation request.
[0039] The translation is repeatedly performed 12 with changing conditions, such as changes in one or more hyperparameters such as temperature parameters of the language model, transformations in the original program code and / or changes in inputs to the language model, such as changes in tasks or prompts. In addition, variables may also be renamed in the code.
[0040] These measures produce a difference. This difference allows verification and an improved quality assessment of the generated translations, and also allows the language model to be trained with the aid of feedback. Within the scope of the improved quality assessment, it can be determined which of the generated translations are better or worse suited.
[0041] The original program code and the one or more target program codes are then compared with a test harness 13, wherein the test harness can be automatically generated. Alternatively, an already existing test harness can also be used. The test of the test harness or the test harnesses themselves can be automatically generated, for example by fuzz testing, mutation of the input of the test harness, derivation from the original language and / or the target language contract and / or from a language model.
[0042] Comparing the target program code with the test harness can start, for example, with one or more pre-given or pre-made tests. These initial tests can belong to the code or be provided with the code. The tests can be stored in a test database. The differential tests can then be added to the test database.
[0043] Automatic construction of a test harness for comparing source code to target code may include the following steps or assumptions:
[0044] - When a comparison fails on a given test, that variant of the translation is discarded.
[0045] - When a comparison on an automatically generated test fails, that variant of the translation is discarded. The one or more failed tests are adopted because the tests are clearly differentiated. Adoption may include storage in a test database. Automatically generated tests may be formed by fuzz testing, input mutation, derivation from contracts and, if necessary, from a language model.
[0046] -When a comparison fails on a differentiation test, the variant is discarded.
[0047] Optionally, static verification can be performed in parallel with testing using the test harness.
[0048] -Contracts are either given in advance or extracted from the system environment.
[0049] - In the case of Rust, contracts can also be provided by the compiler, in other languages, for example, by linters, etc.
[0050] - Bounded model checking and / or abstract interpretation, implemented in commercial tools such as Astrée or open source tools such as CBMC.
[0051] -Automatically build a bounded model checking setup that statically compares the target program code against a given contract.
[0052] -Automatically build an abstract interpretation setup that statically compares the target program code to a given contract.
[0053] - Automatically build bounded model checking settings to check source and target code for functional equivalence.
[0054] The target program code is evaluated 14 according to code quality metrics such as source code length, number of loops and or branch depth, test quality metrics such as branch coverage and / or number of available or executed tests.
[0055] According to one specific embodiment, the method further comprises carrying out steps 11 to 14 of the claims multiple times, so that multiple evaluations of the target program code are generated, and wherein a quality value is respectively assigned to the multiple evaluations.
[0056] According to one specific embodiment, the method further comprises generating a quality value based on code and test metrics, the number of available tests, the number of additionally executed tests and / or the type and / or number of formal checks of the contract in the target language.
[0057] The evaluation or quality value allows the comparability and prioritization of the generated target program codes. The method returns only correct solutions, ie code translations, and additionally provides information about the quality and correctness confidence.
[0058] Figure 2 A computer system 20 is schematically shown in which the techniques of the present disclosure may be used for automated program code translation. The computer system 20 is designed to execute Figure 1 Method 10 and according to Figure 4 The computer system 20 may be implemented in hardware and / or software. Figure 2 The system shown in can be viewed as a computer program designed to execute the Figure 1 Method 10 and according to Figure 4 Method 40.
[0059] Original program code 21 in an original language such as C is provided to a language model 22 such as a Large Language Model (LLM) for translation into a target language such as Rust. Language model 22 generates target program code 23 as a translation of original program code 21. This area of computer system 20 may be referred to as a production area.
[0060] Another input 24 to the system 20 is the test and optionally the test harness in the original language. Alternatively or optionally, the test and / or the test harness may be in the target language. These inputs are fed to the test harness 25. The test harness 25 records the function or test in the target language.
[0061] Optionally, the contract 26 can be fed to a contract unit 27 that works in a target language such as Rust. The contract is managed there for later verification of the target program code 23.
[0062] The input of the first checking unit 28 is connected to the language model 22 for inputting the target program code 23 and to the test tool 25 for inputting the test routine. In the first checking unit 28 , the target program code 23 is compared with the original program code 21 by means of the test tool 25 .
[0063] The input of the second checking unit 29 is connected to the language model 22 for inputting the target program code 23 and to the contract unit 27 for inputting the contract. In the second checking unit 29, the original program code 21 is checked with the aid of the contract 26.
[0064] If the checks in the first check unit 28 and the second check unit 29 are successfully completed, a status message 30 is output: the target program code 23 runs normally. This area of the computer system 20 can be called a check area.
[0065] The target program code 31 is evaluated in view of its quality according to the metric 32. The metric 32 may include a code quality metric, a test quality metric and / or the number of tests. When the evaluation is successful, the target program code and its quality or quality are output as output 33. This area of the computer system 20 may be referred to as a quality evaluation area.
[0066] Based on the evaluation, the quality can be calculated. If multiple target program codes have been generated, the solutions can be provided to the user in an orderly manner according to the quality.
[0067] Figure 3 A computer system 20 is schematically shown in which the techniques of the present disclosure may be used for automated program code translation. The computer system 20 may correspond to Figure 2 The computer system 20 is designed to execute the Figure 1 Method 10 and according to Figure 4 Method 40, especially Figure 3 The computer system 20 is designed to execute according to Figure 4 The computer system 20 may be implemented in hardware and / or software. Figure 3The system shown in can be viewed as a computer program designed to execute the Figure 1 Method 10 and according to Figure 4 Method 40.
[0068] exist Figure 3 2 shows the error handling mechanism of the computer system 20. If the target program code 31 cannot be generated without error, this leads to an error in the first inspection unit 28. Accordingly, an output message 34 is used to reduce the trust in the translation. In addition, the error and the associated target program code 31 are stored in an error module 35.
[0069] From the error module 35, the best target program code to date with errors still present is fed back to the language model 22 as information 36 in order to thereby generate a better, ideally error-free target program code. This reduces reliability and is taken into account in the quality determination.
[0070] Optionally, this may not only relate to errors in the inspection region, but similarly also to errors in the quality assessment region.
[0071] Figure 4 is a flow chart illustrating a method 40 for training a language model. The method 40 for training a language model is set up for automated program code translation from an original program code in an original language to a target program code in a target language.
[0072] In a first step, the original program code is input 41 into a language model and a predicted target program code is generated. The language model may have been pre-trained and may also have been fine-tuned if necessary. Alternatively, a new, untrained language model may also be used to start. Here, training is based on reinforcement learning. Training occurs in a training environment, for example, using PPO (Proximal Policy Optimization).
[0073] Further, the automated checking 42 of the predicted target program code is based on formal checking of identical behavior of the original program code and the target program code, testing in the original language, contractual testing in the original language, and / or syntactic and stylistic testing.
[0074] A reward is then generated 43 for the language model, wherein a negative reward is generated when a test fails. The more negative the value of the negative reward becomes, the more wrong the test or its result is. When all tests succeed, a positive reward is generated, and the size of the value of the reward is based on the number of contracts in the original language and the number of tests in the original language.
[0075] According to one embodiment, the method further comprises, in the case of a positive reward, settling the value of the reward with the code quality metric.
[0076] Finally, the weights of the language model are updated 44 with the value of the reward. The result of this method is a language model that is better trained on new, unlabeled data (here, for example, C code from an engine control unit), ie, provides more reliable translations.
[0077] According to one embodiment, the method further comprises approximating the reward by executing only one of the tests of the automated verification. This allows the training to be accelerated.
Claims
1. A method (10) for automated program code translation from a source language to a target language, comprising the steps of: - translating (11) an original program code (21) in an original language into a target program code (23) in a target language by means of a language model, - repeatedly performing (12) the translation with changing conditions, for example a change in one or more hyperparameters such as a temperature parameter of the language model, a transformation in the original program code (21) and / or a change in the input (24) to the language model (22), - comparing (13) the source program code (21) and one or more target program codes (23) with a test harness (25), wherein the test harness (25) is automatically generated, - evaluating (14) the target program code (23) according to a code quality metric, a test quality metric and / or a number of tests.
2. The method according to claim 1, wherein the steps (11-14) of the claim are performed multiple times, so that multiple evaluations of the target program code (23) are generated, and wherein the multiple evaluations are each assigned a quality value.
3. The method according to claim 2, wherein the quality value is generated based on code and test metrics, the number of available tests, the number of additionally executed tests and / or the type and / or number of formal checks of the contract (26) of the target language.
4. The method according to any one of claims 1 to 3, wherein the conditions for the change of the translation include renaming of variables in the code and / or changes to tasks and / or commands in the input to the language model (22).
5. A method according to any one of claims 1 to 4, wherein the tests of the test tool (25) are automatically generated by fuzz testing, mutation of the input of the test tool (25), derivation from contracts (26) of the original language and / or target language and / or from a language model (22).
6. The method according to claim 1, wherein, if the comparison (13) and / or the evaluation (14) is unsuccessful, information about errors in the translated target program code (23) is fed back to the language model.
7. A method (40) for training a language model, which is set up for automated program code translation from an original program code (21) in an original language to a target program code (23) in a target language, the method comprising the following steps: - input (41) the original program code (21) into the language model (22) and generate a predicted target program code (23), - automatically checking (42) the predicted target program code (23) based on formal verification of identical behavior of the source program code and the target program code, testing in the source language, testing on the contract (26) in the source language and / or syntactic and stylistic testing, - generating (43) a reward for the language model (22), wherein a negative reward is generated when a test fails, wherein a positive reward is generated if all tests succeed, and wherein the magnitude of the reward is based on the number of contracts (26) in the original language and the number of tests in the original language, - Update (44) the weights of the language model using the value of the reward.
8. The method according to claim 7, wherein in case of a positive reward, the value of the reward is settled using a code quality metric.
9. A method according to claim 7 or 8, wherein the reward is approximated by performing only one of the tests of the automated verification.
10. A computer system (20) designed to carry out the method (10; 40) according to any one of claims 1 to 9.
11. A computer program designed to carry out the method (10; 40) according to any one of claims 1 to 9.
12. A computer readable medium or signal storing and / or containing a computer program according to claim 11.