Method for transforming a code by a language model

Abstract

The invention relates to a method (100) for transforming a code using a language model (50), comprising the following steps: - extracting (101) a code segment to be transformed from a code, - transforming (102) the extracted code segment using the language model (50), - checking (103) the transformed code segment based on at least one defined requirement, - integrating (104) the transformed code segment into the code if a result of the check (103) indicates that the at least one defined requirement is met. The invention further relates to a computer program, a device, and a storage medium for this purpose.

Description

[0001] The invention relates to a method for transforming a code using a language model. Furthermore, the invention relates to a computer program, a device, and a storage medium for this purpose. State of the art

[0002] Large Language Models (LLMs) can perform tasks on existing program code using appropriate text prompts. For example, code can be improved, bugs fixed, or translated from one programming language to another. However, the results delivered by the LLM are often incorrect or do not meet desired quality criteria (keyword: hallucination). For instance, refactoring is intended to improve the quality of a code construct (e.g., in terms of readability or maintainability) while preserving its behavior. However, if refactoring is performed by an LLM, the behavior often changes—so much so that it is no longer refactoring. In this way, subtle errors may be introduced that later lead to problems and require costly correction.The larger the input – in this case, the code to be processed – the more frequently an LLM produces such incorrect results.

[0003] The challenge of guaranteeing correctness is particularly important in safety-critical systems. Without sound safeguards, such technologies cannot be used in the development of such systems.

[0004] For a linear language manager (LLM) to correctly perform the tasks described above, a certain amount of context is required. This includes, for example, declarations of the program constructs used in the corresponding code. Therefore, it is not enough to simply provide the function in question; type, variable, and other declarations used within the function must also be included. This significantly increases the required context input size, potentially by an order of magnitude. As a result, the LLM reaches its limits even more quickly and cannot focus on the truly relevant part, which often comprises only a few lines of code. This, in turn, increases the error rate.

[0005] Secondly, functions often contain complex control constructs (e.g., nested loops and branches) that cause scaling problems when checked with formal methods. In many cases, the formal method cannot therefore deliver a verification result within an acceptable timeframe (e.g., a few minutes). Disclosure of the invention

[0006] The invention relates to a method with the features of claim 1, a computer program with the features of claim 10, a device with the features of claim 11, and a computer-readable storage medium with the features of claim 12. Further features and details of the invention will become apparent from the respective dependent claims, the description, and the drawings. Features and details described in connection with the method according to the invention naturally also apply in connection with the computer program, the device, and the computer-readable storage medium according to the invention, and vice versa, so that mutual reference is always possible with regard to the disclosure of the invention.

[0007] The invention relates in particular to a method for transforming a code by means of a language model (English: "Large Language Model"), comprising the following steps: Extracting a code snippet to be transformed from a code, wherein the code snippet to be transformed is determined manually or automatically, e.g., due to an error during code execution or to perform refactoring of the code snippet, for example, to improve the readability or maintainability of the code snippet; transforming the extracted code snippet using the language model, wherein the extracted code snippet is improved during the transformation, for example, with respect to at least one property such as scarcity or maintainability, and / or at least one error in the extracted code snippet is fixed, and / or the extracted code snippet is translated from one programming language to another; checking the transformed code snippet based on at least one defined requirement.wherein preferably at least one or more test procedures can be carried out, wherein, for example, it can be checked whether an error in the original extracted code snippet still exists and / or whether the transformed code snippet leads to an error message and / or whether the transformed code snippet produces the same output as the original extracted code snippet, integrating the transformed code snippet into the code, i.e., in particular into the original code, if a result of the check indicates that the at least one defined requirement is met.

[0008] When extracting the code segment to be transformed from the code, it is preferably not simply a matter of extracting, for example, the affected lines, but also the necessary context. As a result, the extracted code segment must contain the relevant code and be translatable. According to the invention, the extracted code segment can thus be isolated into a translatable form and transformed separately by the language model. This is particularly advantageous when the code from which the code segment is extracted is very extensive. This isolation allows for more targeted processing and error reduction, as the focus is on this specific area. Formal verification ensures, in particular, that the transformation meets at least one defined requirement and generates correct code.Only if the test is successful is the transformed code snippet preferably integrated into the original code, thereby improving the accuracy of the entire transformation process.

[0009] The at least one defined requirement can, for example, include semantic consistency and / or equivalence, i.e., in particular, the same behavior as the originally extracted code snippet, syntactic correctness, and / or at least a rule-based restriction of the code, for example, based on a standard or other context of the code.

[0010] Furthermore, if the result of the check indicates that at least one defined requirement is not met, the transformation and check steps can be repeated until the check result indicates that at least one defined requirement is met. Thus, by repeatedly performing the transformation and check, it can be ensured that the transformed code snippet meets at least one defined requirement. This significantly increases the reliability of the resulting code and can reduce errors that might arise from inadequate transformation results.

[0011] It is also conceivable, optionally, that each subsequent transformation could further include the following step: Determining a text request comprehensively involves determining each previous result of transformation and verification, whereby a correction of the previous result of transformation is initiated by the language model based on the text request.

[0012] This ensures, in particular, that the language model iteratively improves its output. Furthermore, the combination of the transformed code snippet and the result of the verification allows for stepwise fine-tuning of the language model. This can lead to greater accuracy and reliability in the transformation process.

[0013] Furthermore, within the scope of the invention, it is conceivable that the extraction process comprises the following steps: Determining an abstract syntax tree (AST) of the code, analyzing the determined abstract syntax tree, particularly with regard to context information in the code for the extracted code snippet, inserting at least one addition into the extracted code snippet based on a result of the analysis, in particular to provide the context information that enables isolated translation, testing and / or execution of the extracted code snippet in accordance with the code.

[0014] An abstract syntax tree is, in particular, a data structure that can be used to represent an abstract syntactic structure of program code. It is preferably a tree structure that represents the code in a hierarchical form and allows the code to be analyzed and processed at an abstract level. The abstract syntax tree is generated, for example, by a compiler or interpreter parser and includes, in particular, all information about the structure of the code, including the arrangement of expressions, statements, and operators.

[0015] Optionally, the analysis of the identified abstract syntax tree may include the following steps: Analyzing the abstract syntax tree to determine nodes in the abstract syntax tree that are associated with the extracted code snippet, identifying a parent node of the specific nodes of the extracted code snippet, and identifying nodes of the abstract syntax tree that are below the parent node and do not belong to the specific nodes of the extracted code snippet.

[0016] This ensures, in particular, that the language model precisely identifies the code segment to be transformed, thereby improving the quality of the transformation. Furthermore, analyzing the abstract syntax tree can provide a better understanding of the context of the code segment to be transformed, which can also improve the accuracy of the transformation.

[0017] It is also conceivable, optionally, that inserting at least one addition into the extracted code snippet includes at least one of the following steps: Inserting artificially generated code to adapt the runtime behavior of the extracted code snippet to the code, i.e., in particular the original code; inserting declarations from the code that affect the extracted code snippet.

[0018] This allows the extracted code snippet to be advantageously and precisely adapted to an original behavior in the code's environment, enabling more precise transformation by the language model and allowing analysis by static or dynamic methods.

[0019] Another possibility is that the process is automated and that at least one requirement includes a requirement from a standard, in particular MISRA-C. MISRA-C is a C programming standard from the automotive industry, developed by the Motor Industry Software Reliability Association (MISRA). The inclusion of standards like MISRA-C ensures, in particular, that the resulting code also meets common safety and quality specifications. This improves the reliability and security of the resulting code.

[0020] Furthermore, extraction can be performed based on an error in a technical system. In this case, the at least one defined requirement can relate to fixing the error in the technical system. In other words, a corresponding code snippet that leads to the error in the technical system is extracted, and the at least one defined requirement can be used to check whether the error has been fixed.

[0021] The invention also relates to a computer program, in particular a computer program product, comprising instructions which, when executed by a computer, cause the computer to execute the method according to the invention. Thus, the computer program according to the invention offers the same advantages as those described in detail with reference to a method according to the invention.

[0022] The invention also relates to a data processing device configured to execute the method according to the invention. The device can, for example, be a computer that executes the computer program according to the invention. The computer can have at least one processor for executing the computer program. Alternatively, a non-volatile data storage device can be provided in which the computer program is stored and from which the computer program can be read by the processor for execution.

[0023] The invention may also relate to a computer-readable storage medium which contains the computer program according to the invention and / or includes instructions which, when executed by a computer, cause the computer to execute the method according to the invention. The storage medium is, for example, designed as a data storage device such as a hard drive and / or non-volatile memory and / or a memory card. The storage medium can, for example, be integrated into the computer.

[0024] Furthermore, the method according to the invention can also be implemented as a computer-implemented method. Alternatively or additionally, at least one of the disclosed method steps can be computer-implemented and / or carried out automatically.

[0025] Further advantages, features and details of the invention will become apparent from the following description, which refers to the drawings.

[0026] Exemplary embodiments of the invention are described in detail. The features mentioned in the claims and in the description can be essential to the invention individually or in any combination. The following are shown: Fig. 1 a schematic visualization of a method, a technical system, a language model, a device, a storage medium and a computer program according to embodiments of the invention, Fig. 2 a schematic representation of an abstract syntax tree according to embodiments of the invention, Fig. 3 a schematic representation of an abstract syntax tree according to embodiments of the invention, Fig. 4 a schematic representation of an abstract syntax tree according to embodiments of the invention, Fig. 5 a schematic representation of an abstract syntax tree according to embodiments of the invention.

[0027] In Fig. 1 A method 100, a technical system 11, a language model 50, a device 10, a storage medium 15 and a computer program 20 are schematically represented according to exemplary embodiments of the invention.

[0028] Fig. 1 In particular, a method 100 for transforming code using a language model 50 is shown. In a first step 101, a code segment to be transformed is extracted from the code. In a second step 102, the extracted code segment is transformed using the language model 50. In a third step 103, the transformed code segment is checked against at least one defined requirement. In a fourth step 104, the transformed code segment is integrated into the code if a result of the check 103 indicates that the at least one defined requirement is met.

[0029] Fig. 2 This shows how to determine node 2 of the abstract syntax tree 1 corresponding to the relevant lines of the code. A set K comprises these determined node 2s.

[0030] Fig. 3 shows how to determine a common parent node p of the determined nodes 2.

[0031] Fig. 4 This schematically shows how to find nodes 2' whose entire subtree is not in K. A set N contains these nodes 2'.

[0032] Fig. 5 shows mappings between subtrees p and q as well as q and r. In particular, p is the subtree of the original code, q is the subtree of the isolated code, and r is the subtree of the transformed code.

[0033] Fig. 6 This shows how to determine a resulting code based on the subtrees. Specifically, `pr` is inserted in place of `pr`, and `y` is inserted in place of `x`. Similarly, nodes from `r` are preferably replaced by subtrees from `p` (not shown).

[0034] On the Figuren 2 bis 6 This will be referred to again in the detailed description below.

[0035] According to exemplary embodiments of the invention, the code segments relevant for the modification are isolated before the actual processing. Then, in particular, the language model-based transformation is performed. The result is then preferably checked based on at least one defined requirement, i.e., validated using formal methods. Finally, the modification, i.e., in particular the transformed code segment, is preferably incorporated into the original code.

[0036] The problem (i.e., the code) is reduced to a code snippet relevant to the change. This allows the language model to focus more effectively on this code snippet and thus deliver better results. The formal method, i.e., the verification process, is particularly advantageous due to the low complexity of the code snippet, avoiding scaling problems.

[0037] The input data preferably consists of translatable code and a modification task to be performed by a language model on a specific part of the code, i.e., the code snippet. Furthermore, a formal method or executable tool that implements this method and can verify the quality of the result is preferably provided.

[0038] The affected lines of the code snippet are preferably extracted and supplemented by analyzing the abstract syntax tree 1 (AST) of the code so that valid code is created that includes these lines. For this purpose, the abstract syntax tree 1 for the given (total) input code can first be determined. Subsequently, nodes 2 of the abstract syntax tree 1 that belong to the lines of the code snippet can be determined. These, in particular, form the set K (see...). Fig. 2 ).

[0039] Subsequently, a (first) common parent node p of all nodes in K is preferably determined (see below). Fig. 3 ), which is preferably a statement (i.e., expressions are preferably preserved). Subsequently, nodes 2' of the abstract syntax tree below the parent node p are preferably determined, from which the entire subtree is not in K (cf. Fig. 4 These elements, or nodes 2', form a set N and are preferably replaced by artificially generated code in a step described below. A function with corresponding interfaces is then preferably generated around the code attached to the parent node p. The code belonging to the nodes 2 attached to p can then be inserted into N up to the nodes 2'. At each node, artificially generated code is preferably inserted that marks the point reached at runtime, for example, by setting a variable to a unique constant value. Furthermore, necessary declarations can be inserted that appear before the function in the generated code.

[0040] Then, preferably, the language model-based transformation is performed on the isolated, extracted code snippet.

[0041] The result of the language model-based transformation is then preferably checked. If the check fails, this is preferably reported back to language model 50, and a correction is initiated. The language model-based transformation can then be performed again using the response from language model 50.

[0042] If the check was successful, the change made by language model 50 to the isolated code snippet is preferably applied to the original code, i.e., integrated into it. This preferably involves computing the abstract syntax tree 1 of the original function, node p, the abstract syntax tree 1' of the extracted function, and the abstract syntax tree 1" of the function modified by language model 50 (see...). Fig. 5 Furthermore, preferably a node q in the abstract syntax tree 1' is determined that corresponds to the node p in the abstract syntax tree 1, as well as a node r in the abstract syntax tree 1", that corresponds to the node q in the abstract syntax tree 1' (see Fig. 5 , yellow nodes). This is done, for example, via a node type and a position in the abstract syntax tree. Subsequently, similarities and differences between the subtrees p and q can be determined (see below). Fig. 5 , left pair). This allows a mapping Tba of the inserted placeholders, i.e., the artificially generated code, to the original code (from the abstract syntax tree 1) to be determined. Furthermore, similarities and differences between the subtrees q and r can be determined (see below). Fig. 5 , right pair). This determines, in particular, a mapping Tcb of the placeholders in abstract syntax tree 1" to the placeholders in abstract syntax tree 1'. The transformed code can then be generated from abstract syntax tree 1 by traversing and unparsing the individual nodes n using the function Tcb (Tba (n)), provided it is defined there (see...). Fig. 6 ).

[0043] The method according to exemplary embodiments of the invention will be described below using an example.

[0044] Given the following C code: void func(unsigned int val) { / / more code A if (val != 0) { / / more code B} else { / / more code C} / / more code D}

[0045] MISRA-C requires, for example, that the types on both sides of the operator must be the same for mathematical operations. In the example, `val` is an unsigned `int`, but the literal `0` is an `int` (by default). To fix this problem, it's preferable to isolate the relevant line. Starting with the expression `val != 0`, the `if` statement can be identified as node `p`. The entire `then` block as well as the `else` block ("more code B" and "more code C") can be identified as set `N`. A new function is then preferably generated that contains exactly this code: unsigned int val; int foo() { int ret = 0; if (val != 0) { ret = 1;} else { ret = 2;} return ret;}

[0046] The instructions regarding `ret` are preferably generated for the nodes in N. They are particularly helpful for the formal examination to characterize the behavior. Furthermore, the declaration of `val` can be generated. The language model-based transformation on this code could lead to the following result: unsigned int val; int foo() { int ret = 0; if (val ! = Ou) { ret = 1;} else { ret = 2;} return ret;}

[0047] It can now be verified and concluded that this code no longer addresses the original problem. As a next step, the code change can be incorporated into the original code. void func(unsigned int val) { / / more code A if (val != Ou) { / / more code B} else { / / more code C} / / more code D}

[0048] The preceding explanation of the embodiments describes the present invention solely by way of examples. Naturally, individual features of the embodiments can be freely combined with one another, provided this is technically feasible, without departing from the scope of the present invention.

Claims

1. A method (100) for transforming a code by a language model (50), comprising the following steps: - Extracting (101) a code segment to be transformed from a code, - Transforming (102) the extracted code segment by the language model (50), - Checking (103) the transformed code segment on the basis of at least one defined requirement, - Integrating (104) the transformed code segment into the code if a result of the check (103) indicates that the at least one defined requirement is met.

2. Method (100) according to claim 1, characterized by that , if the result of checking (103) indicates that at least one defined requirement is not met, the steps of transforming (102) and checking (103) are repeated until the result of checking (103) indicates that at least one defined requirement is met.

3. Method (100) according to claim 2, characterized by that The transformation (102) performed again in each instance further includes the following step: - Determining a text request comprising a previous result of the transformation (102) and the checking (103), whereby a correction of the previous result of the transformation (102) is initiated by the language model (50) on the basis of the text request.

4. Method (100) according to any one of the preceding claims, characterized by that which includes at least one defined requirement: semantic consistency and / or equivalence, syntactic correctness and / or at least a rule-based restriction of the code.

5. Method (100) according to any one of the preceding claims, characterized by thatThe extraction (101) comprises the following steps: - Determining an abstract syntax tree (1) of the code, - Analyzing the determined abstract syntax tree (1), - Inserting at least one addition into the extracted code snippet based on a result of the analysis.

6. Method (100) according to claim 5, characterized by that Analyzing the determined abstract syntax tree (1) comprises the following steps: - Analyzing the abstract syntax tree (1) to determine nodes (2) in the abstract syntax tree (1) that are associated with the extracted code snippet, - Determining a parent node (p) of the specific nodes (2) of the extracted code snippet, - Determining nodes (2) of the abstract syntax tree (1) that are below the parent node (p) and do not belong to the specific nodes (2) of the extracted code snippet.

7. Method (100) according to claim 5 or 6, characterized by thatThe insertion of at least one addition into the extracted code snippet includes at least one of the following steps: - Inserting artificially generated code to adapt the runtime behavior of the extracted code snippet to the code, - Inserting declarations from the code that affect the extracted code snippet.

8. Method (100) according to any one of the preceding claims, characterized by that the procedure (100) is carried out automatically and the at least one requirement includes at least one requirement from a standard, in particular MISRA-C.

9. Method (100) according to any one of the preceding claims, characterized by that the extraction (101) is carried out on the basis of a fault in a technical system (11) and at least one defined requirement relates to at least the correction of the fault in the technical system (11).

10. Computer program (20), comprising instructions which, when the computer program (20) is executed by a computer (10), cause it to execute the method (100) according to one of the preceding claims.

11. Device (10) for data processing, which is configured to carry out the method (100) according to any one of claims 1 to 9.

12. Computer-readable storage medium (15) comprising instructions which, when executed by a computer (10), cause it to perform the steps of the method (100) according to any one of claims 1 to 9.

Images