Large language model code translation error detection

Abstract

Large language model code translation error detection includes receiving a code portion of a first programming language, and converting the code portion to a second programming language. A first accuracy is calculated to convert the code portion into a second programming language. A difference between the first accuracy and a historical accuracy of a transition from the first programming language to the second programming language is determined. A potential error in the code portion of the first programming language is indicated based on a difference between the first accuracy and the historical accuracy being greater than a predetermined value.

Description

Technical Field

[0001] The present disclosure relates to methods, apparatuses, and products for detecting errors in code translation of large language models. Background Art

[0002] Migrating the functionality of legacy source code to a more modern programming language can increase the maintainability and readability of the source code, as well as improve system performance. However, such migration is a daunting task, which can include writing, testing, validating, and debugging a large amount of code. If the modernized source code is deployed with errors into a complex software system, the expected operations may not be completed, the brute-force retry logic may consume resources, transactions may start to fail, and all components of the stack may fail, resulting in possible downtime. This can be catastrophic, especially in core production systems. Summary of the Invention

[0003] According to embodiments of the present disclosure, various methods, apparatuses, and products for detecting errors in code translation of large language models are described herein. In some aspects, a method for detecting errors in code translation of a large language model includes receiving a code portion in a first programming language; converting the code portion into a second programming language; calculating a first accuracy of converting the code portion into the second programming language; determining a difference between the first accuracy and a historical accuracy of the conversion from the first programming language to the second programming language; and indicating a potential error in the code portion of the first programming language based on the difference between the first accuracy and the historical accuracy being greater than a predetermined value.

[0004] In an embodiment, calculating the first accuracy of converting the code portion into the second programming language further includes: calculating a first code structure representation of the code portion in the first programming language; calculating a second code structure representation of the converted code portion in the second programming language; and calculating the first accuracy based on a comparison between the first code structure representation and the second code structure representation.

[0005] In an embodiment, calculating the first code structure representation of the code portion in the first programming language further includes calculating the first code structure representation of the code portion in the first programming language based on one or more software metrics. In an embodiment, calculating the second code structure representation of the converted code portion in the second programming language further includes calculating the second code structure representation of the converted code portion in the second programming language based on the one or more software metrics. In an embodiment, the one or more software metrics include code complexity metrics.

[0006] In an embodiment, calculating the first code structure representation of the code portion of the first programming language based on one or more software metrics further includes calculating the first code structure representation of the code portion of the first programming language based on a weighted combination of a plurality of the one or more software metrics.

[0007] In an embodiment, the method includes determining a historical accuracy based on an accuracy of at least one previous conversion of the first programming language to the second programming language for another code portion.

[0008] In an embodiment, converting the code portion to a second programming language includes using a generative artificial intelligence model to convert the code portion to the second programming language. In an embodiment, the generative artificial intelligence model includes a large language model.

[0009] In an embodiment, indicating the potential error in the code portion based on the difference between the first accuracy and the historical accuracy being greater than a predetermined value further includes providing an indication of the potential error in the code portion to the generative artificial intelligence model.

[0010] In some aspects, an apparatus may include a processing device; and a memory operably coupled to the processing device, where the memory stores computer program instructions that, when executed, cause the processing device to perform the method. In some aspects, a computer program product including a computer-readable storage medium may store computer program instructions that, when executed, perform the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A block diagram illustrating an example computing environment for large language model code translation error detection in accordance with some embodiments of the present disclosure.

[0012] Figure 2 A flowchart illustrating an example method for large language model code translation error detection in accordance with some embodiments of the present disclosure.

[0013] Figure 3 A flowchart illustrating another example method for large language model code translation error detection in accordance with some embodiments of the present disclosure.

[0014] Figure 4 A flowchart illustrating another example method for large language model code translation error detection in accordance with some embodiments of the present disclosure.

[0015] Figure 5 A flowchart illustrating another example method for large language model code translation error detection in accordance with some embodiments of the present disclosure. DETAILED DESCRIPTION

[0016] The emergence of large language models and other improvements to artificial intelligence (AI) technologies have enabled AI to generate source code. For example, large language models (LLMs) are trained on vast datasets of source code to provide generative AI that outputs source code based on an input or prompt. As will be discussed in more detail below, the migration of an application from its original source code to new source code can be assisted by such AI. That is, AI can be used to generate new source code based on an input of the original source code. For example, an LLM can be given a prompt such as "generate Java code that achieves the same goal as the following COBOL code", where legacy COBOL source code is provided as the input. In response, the LLM can, at least ideally, output AI-generated Java source code that performs the same function as the legacy version and produces the same output as the legacy version. Of course, any new source code, whether generated by humans or AI, is prone to introducing defects, execution errors, or other faults.

[0017] Embodiments according to the present disclosure advantageously utilize historical accuracy data to detect potential defects, execution errors, or other faults in the original source code of a first programming language (e.g., COBOL), which is subsequently translated into transformed source code of a second programming language (e.g., Java). LLMs have significantly advanced the process of translating code from one programming language to another. However, challenges arise when the output code of the translation deviates from an acceptable standard. In such cases, it is difficult to determine whether an error in the translated source code is caused by a translation error or a latent error in the original source code. Various embodiments recognize that this difference between an accurate translation of good input and a sub-optimal result of bad input can serve as an indicator that the original source code should be corrected before translating the source code again.

[0018] In a specific example, an LLM used to convert COBOL to Java produces a specific translation accuracy that is acceptable within a standard deviation assuming the COBOL is written correctly without defects. However, if the COBOL contains defects or errors, the accurate translation of the "defective" COBOL will be "defective" Java. As a result, the accuracy of the code translation may drop sharply below the standard deviation, potentially indicating an error in the input COBOL. A decrease in the translation accuracy from the historical accuracy of translating COBOL to Java may lead to the hypothesis that the LLM is translating code that has not been seen before or that there is an error in the source code. In one example, a 20% decrease in accuracy from the expected accuracy results in an 80% certainty hypothesis that there is a defect in the COBOL code.

[0019] By identifying potential errors in the original source code, the original source code can be reviewed to identify potential errors. Once the errors are corrected, the original source code can be translated again using an LLM to produce more accurate transformed code. This process can be iterated until an acceptable translation accuracy is achieved.

[0020] Now refer to Figure 1 , which shows an example computing environment in accordance with aspects of the present disclosure. Computing environment 100 includes an example of an environment for executing at least some of the computer code involved in performing the various methods described herein, such as code analysis module 107. In addition to code analysis module 107, computing environment 100 includes, for example, computer 101, wide area network (WAN) 102, end user device (EUD) 103, remote server 104, public cloud 105, and private cloud 106. In this embodiment, computer 101 includes a set of processors 110 (including processing circuitry 120 and cache 121), communication structure 111, volatile memory 112, persistent storage 113 (including operating system 122 and code analysis module 107, as described above), a set of peripheral devices 114 (including a set of user interface (UI) devices 123, storage 124, and a set of Internet of Things (IoT) sensors 125), and network module 115. Remote server 104 includes remote database 130. Public cloud 105 includes gateway 140, cloud orchestration module 141, a set of host physical machines 142, a set of virtual machines 143, and a set of containers 144.

[0021] Computer 101 can take the form of a desktop computer, laptop computer, tablet computer, smart phone, smart watch or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device now known or hereafter developed that is capable of running programs, accessing a network, or querying a database such as remote database 130. As is well known in the computer art and depending on the technology, the execution of computer-implemented methods can be distributed among multiple computers and / or multiple locations. On the other hand, in this presentation of computing environment 100, the discussion focuses on a single computer, specifically computer 101, to keep the presentation as simple as possible. Computer 101 can be located in the cloud, even though Figure 1 it is not shown in the cloud in

[0022] The processor set 110 includes one or more computer processors of any type now known or to be developed in the future. The processing circuitry 120 may be distributed across multiple packages, such as multiple cooperative integrated circuit chips. The processing circuitry 120 may implement multiple processor threads and / or multiple processor cores. The cache 121 is a memory located within the processor chip package and is generally used for data or code that should be made available for rapid access by the threads or cores running on the processor set 110. Cache memory is typically organized into multiple levels based on its relative proximity to the processing circuitry. Alternatively, some or all of the cache in the processor set may be located "off-chip". In some computing environments, the processor set 110 may be designed to work with qubits and perform quantum computing.

[0023] Computer-readable program instructions are typically loaded onto the computer 101 so that the processor set 110 of the computer 101 executes a series of operational steps to implement a computer-implemented method such that the instructions so executed will instantiate the method specified in the flowchart and / or narrative description of the computer-implemented method included in this document. These computer-readable program instructions are stored in various types of computer-readable storage media, such as the cache 121 and other storage media discussed below. The program instructions and associated data are accessed by the processor set 110 to control and direct the execution of the computer-implemented method. In the computing environment 100, at least some of the instructions for performing the computer-implemented method may be stored in the code analysis module 107 in the persistent storage 113.

[0024] The communication structure 111 is a signal conduction path that allows the various components of the computer 101 to communicate with each other. Generally, this structure consists of switches and conductive paths, such as switches and conductive paths that make up a bus, a bridge, a physical input / output port, etc. Other types of signal communication paths may be used, such as fiber optic communication paths and / or wireless communication paths.

[0025] The volatile memory 112 is any type of volatile memory now known or to be developed in the future. Examples include dynamic random access memory (RAM) or static RAM. Generally, the volatile memory 112 is characterized by random access, but this is not required unless specifically stated. In the computer 101, the volatile memory 112 is located within a single package and inside the computer 101, but, alternatively or additionally, the volatile memory may be distributed across multiple packages and / or located externally relative to the computer 101.

[0026] The persistent storage 113 is any form of non-volatile storage for a computer that is known now or developed in the future. The non-volatility of this storage means that the stored data is maintained regardless of whether power is supplied to the computer 101 and / or directly to the persistent storage 113. The persistent storage 113 can be read-only memory (ROM), but typically at least a portion of the persistent storage allows for writing of data, deletion of data, and re-writing of data. Some common forms of persistent storage include disk and solid-state storage devices. The operating system 122 can take several forms, such as various known proprietary operating systems or operating systems of the open-source portable operating system interface type that employ a kernel. The code included in the code analysis module 107 typically includes at least some of the computer code involved in performing the computer-implemented methods described herein.

[0027] The set of peripheral devices 114 includes a collection of the peripheral devices of the computer 101. The data communication connection between the peripheral devices and other components of the computer 101 can be implemented in various ways, such as a Bluetooth connection, a near-field communication (NFC) connection, a connection made by a cable (such as a universal serial bus (USB)-type cable), a plug-in connection (e.g., a secure digital (SD) card), a connection made through a local communication network, and even a connection made through a wide area network such as the Internet. In various embodiments, the set of UI devices 123 can include components such as a display screen, a speaker, a microphone, wearable devices (such as glasses and smartwatches), a keyboard, a mouse, a printer, a touchpad, a game controller, and a haptic device. The storage 124 is an external storage device, such as an external hard drive, or a pluggable storage device, such as an SD card. The storage 124 can be persistent and / or volatile. In some embodiments, the storage 124 can take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where the computer 101 needs to have a large amount of storage (e.g., in the case where the computer 101 locally stores and manages a large database), the storage can be provided by a peripheral storage device designed to store a very large amount of data, such as a storage area network (SAN) shared by multiple geographically distributed computers. The set of IoT sensors 125 consists of sensors that can be used in Internet of Things applications. For example, one sensor can be a thermometer, and another sensor can be a motion detector.

[0028] The network module 115 is a collection of computer software, hardware, and firmware that allows the computer 101 to communicate with other computers via the WAN 102. The network module 115 may include hardware such as a modem or a Wi-Fi signal transceiver, software for packetizing and / or depacketizing data transmitted over the communication network, and / or web browser software for transmitting data over the Internet. In some embodiments, the network control function and the network forwarding function of the network module 115 are executed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing software-defined networking (SDN)), the control function and the forwarding function of the network module 115 are executed on physically separate devices such that the control function manages several different network hardware devices. Computer-readable program instructions for performing computer-implemented methods can generally be downloaded to the computer 101 from an external computer or an external storage device via a network adapter or a network interface included in the network module 115.

[0029] The WAN 102 is any wide area network (e.g., the Internet) capable of transmitting computer data over non-local distances via any technology known now or developed in the future for transmitting computer data. In some embodiments, the WAN 102 may be replaced and / or supplemented by a local area network (LAN) designed to transmit data between devices located in a local area, such as a Wi-Fi network. The WAN and / or LAN typically include computer hardware such as copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.

[0030] The end-user device (EUD) 103 is any computer system used and controlled by an end user (e.g., a customer of an enterprise operating the computer 101) and may take any form discussed above in connection with the computer 101. The EUD 103 typically receives helpful and useful data from the operation of the computer 101. For example, in the hypothetical case where the computer 101 is designed to provide recommendations to an end user, the recommendation will typically be transmitted from the network module 115 of the computer 101 to the EUD 103 via the WAN 102. In this way, the EUD 103 can display or otherwise present the recommendation to the end user. In some embodiments, the EUD 103 may be a client device such as a thin client, a thick client, a mainframe computer, a desktop computer, etc.

[0031] The remote server 104 is any computer system that provides at least some data and / or functionality to the computer 101. The remote server 104 may be controlled and used by the same entity operating the computer 101. The remote server 104 represents a machine that collects and stores helpful and useful data used by other computers such as the computer 101.

[0032] A public cloud 105 is any computer system available for use by multiple entities, which provides on-demand availability of computer system resources and / or other computing capabilities (notably data storage (cloud storage) and computing power), without direct active management by the user. Cloud computing typically exploits the sharing of resources to achieve economies of scale and consistency. The direct and active management of the computing resources of the public cloud 105 is performed by the computer hardware and / or software of the cloud orchestration module 141. The computing resources provided by the public cloud 105 are typically implemented by virtual computing environments running on various computers of a set of host physical machines 142, which is the universe of physical computers in and / or available for the public cloud 105. A virtual computing environment (VCE) typically takes the form of virtual machines from a set of virtual machines 143 and / or containers from a set of containers 144. It should be understood that these VCEs can be stored as images and can be transferred between various physical machine hosts as images or after instantiation of the VCE. The cloud orchestration module 141 manages the transfer and storage of images, deploys new instantiations of the VCE, and manages the active instantiations of the VCE deployment. The gateway 140 is a collection of computer software, hardware, and firmware that allows the public cloud 105 to communicate via the WAN 102.

[0033] Some further explanations of the virtualized computing environment (VCE) will now be provided. A VCE can be stored as an "image". New active instances of the VCE can be instantiated from this image. Two common types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to an operating system feature where the kernel allows for the existence of multiple isolated user space instances, called containers. From the perspective of the programs running within them, these isolated user space instances typically appear as actual computers. A computer program running on a normal operating system can utilize all the resources of that computer, such as connected devices, files and folders, network shares, CPU capabilities, and quantifiable hardware capabilities. However, a program running within a container can only use the contents of the container and the devices allocated to the container, which is a feature known as containerization.

[0034] The private cloud 106 is similar to the public cloud 105, except that the computing resources are only available for use by a single enterprise. Although the private cloud 106 is depicted as communicating with the WAN 102, in other embodiments, the private cloud can be completely disconnected from the Internet and only accessible through a local / private network. A hybrid cloud is a combination of multiple clouds of different types (e.g., private, community, or public cloud types) that are typically implemented by different vendors. Each of the multiple clouds remains a separate and discrete entity, but the larger hybrid cloud architecture is bound together by standardized or proprietary technologies that enable orchestration, management, and / or data / application portability between the multiple constituent clouds. In this embodiment, the public cloud 105 and the private cloud 106 are both part of the larger hybrid cloud.

[0035] For further explanation, Figure 2 A flowchart of an example method for large language model code translation error detection according to some embodiments of the present disclosure is shown. Figure 2 The method can be performed by a code analysis module 201 such as Figure 1 the code analysis module 107. In some embodiments, the code analysis module 201 can be implemented as a process or service separate from the application or software that implements code analysis. For example, the code analysis module 201 can be implemented by an operating system or other software that monitors the behavior and execution of the application that implements code analysis. As another example, in some embodiments, the code analysis module 201 can be implemented as a process or service that applies modifications to the application that implements code analysis.

[0036] Figure 2 The method includes receiving 202 a code portion in a first programming language and converting 204 the code portion into a second programming language. Although Figure 2 the execution of an application is described, it should be understood that the solutions described herein can be applied to any software or module capable of implementing code analysis, including applications, operating systems, libraries, etc. In an example, the code portion can include legacy source code written in an older programming language (e.g., COBOL), and the converted code portion can include code written in a modern programming language (e.g., Java). In a specific example, both the code portion and the converted code portion are intended to achieve the same goal, provide the same interface, and produce the same output. In a specific example, the code portion can include an application, subroutine, function, or driver. In an embodiment, a generative AI such as an LLM is used to translate the code portion from the first programming language to the second programming language. In some examples, an AI language model is utilized to generate some or all of the converted code. For example, some or all of the original source code can be provided as input to the AI language model, and it is requested (e.g., prompted) to generate source code that implements the same goal as the original source code in a different programming language.

[0037] In an embodiment, the code analysis module 201 calculates 206 a first accuracy of converting a code portion into a second programming language. In a particular embodiment, the first accuracy indicates the accuracy of the translation from the original source code to the converted source code. In one or more embodiments, the first accuracy of converting a code portion into a second programming language is based on a similarity comparison of the code structures of the original code portion and the converted code portion within a predetermined acceptable threshold. In a particular embodiment, a first code structure representation of the code portion in the first programming language and a second code structure representation of the converted code portion in the second programming language are calculated. The first code structure representation and the second code structure representation are compared to calculate the first accuracy.

[0038] In one or more embodiments, calculating the first code structure representation of the code portion in the first programming language and the second code structure representation of the converted code portion in the second programming language is based on one or more software metrics. In a particular embodiment, an Abstract Syntax Tree (AST) is used to calculate the first accuracy of converting a code portion into the converted code portion. An AST is a data structure for representing the structure of a code portion as a tree representation of the abstract syntactic structure of the text of a programming language. Each node of the tree represents a construct that appears in the text. The syntax is "abstract" in the sense that it does not represent every detail that appears in the true syntax but only structural or content-related details. Compared to the source code, an AST typically does not include non-essential punctuation and delimiters. By comparing the ASTs of the original code portion and the converted code portion, a measure of the conversion accuracy is calculated.

[0039] In another embodiment, one or more code complexity metrics are used to calculate the respective code complexities of each of the original code portion and the converted code portion. A correctly translated source code is expected to exhibit a similar degree of complexity as the original source code. In one or more embodiments, comparing the code complexity of the original source code portion and the converted code portion is a potential indicator of logical errors, bugs, defects, or other programming errors in the original source code portion.

[0040] In an example, a cyclomatic complexity metric is used to calculate the complexities of the original code portion and the converted code portion. The cyclomatic complexity metric represents the number of linearly independent paths through the source code. A path is linearly independent if there exists a subset of one or more paths where the symmetric difference of their sets of edges is empty. For example, if the source code does not contain control flow statements, the complexity is 1 because there is only a single path through the code. If the source code includes a single conditional statement, there are two paths through the code, indicating a complexity of 2.

[0041] In another example, Halstead complexity metrics are used to calculate the complexity of the original code portion and the transformed code portion. Halstead complexity metrics are calculated statistically without program execution and take into account factors such as the number of different operators, the number of different operands, the total number of operators, and the total number of operands. In another example, maintainability index metrics are used to calculate the complexity of the original code portion and the transformed code portion. Maintainability index metrics are calculated to represent the relative ease of maintaining a code portion and are calculated based on the number of statements in the code, cyclomatic complexity, and Halstead volume (calculated as a function of the code length, the number of different operators, and the number of different operands).

[0042] In another embodiment, calculating a first code structure representation of a code portion in a first programming language and a second code structure representation of the transformed code portion in a second programming language is based on a combination of one or more software metrics. In a particular embodiment, calculating a first code structure representation of a code portion in a first programming language and a second code structure representation of the transformed code portion in a second programming language is based on a weighted combination of a plurality of software metrics among one or more software metrics. For example, when calculating the corresponding code structure, one or more of the software metrics may be weighted differently than other software metrics.

[0043] In one embodiment, the code analysis module 201 determines 208 a difference between a first accuracy and a historical accuracy of a transformation from a first programming language to a second programming language. In an embodiment, the historical accuracy is based on an accuracy of at least one previous transformation of another code portion from the first programming language to the second programming language. In a particular embodiment, the accuracy determination of a previous transformation of code from the first programming language to the second programming language is used to determine an expected accuracy within a specific deviation of a translation of the transformation of code programmed in the first programming language to the transformed code in the second programming language. In a particular embodiment, previously determined accuracy information is used to construct an accuracy model for code translation from the first programming language to the second programming language.

[0044] In one embodiment, the code analysis module 201 indicates 210 a potential error in a code portion of a first programming language based on the difference between a first accuracy and a historical accuracy being greater than a predetermined value. In a specific example, if the difference between the first accuracy and the historical accuracy is greater than 20%, the code analysis module 201 indicates 210 a potential error in the code portion of the first programming language. In the example, the indication of the potential error in the code portion is used to check for errors in the code portion, correct the errors, and perform a conversion of the code portion of the first programming language to a second programming language using the corrected code. In a specific embodiment, the process is iteratively repeated until an acceptable accuracy is obtained. In another embodiment, the indication further includes the probability of an error in the code portion.

[0045] In another embodiment, the code analysis module 201 determines that the difference between the first accuracy and the historical accuracy is less than a predetermined value, determines that the converted code portion is not executable, and indicates that the converted code portion is not executable. In a specific embodiment, determining that the converted code portion is not executable includes determining that the converted code portion is not compilable or includes one or more errors.

[0046] For further explanation, Figure 3 FIG. shows a flowchart of another example method for large language model code translation error detection according to some embodiments of the present disclosure. Figure 3 The method of Figure 2 extends the method of Figure 3 in that calculating 206 a first accuracy for converting a code portion to a second programming language includes calculating 302 a first code structure representation of the code portion of the first programming language, and calculating 304 a second code structure representation of the converted code portion of the second programming language. The method of also includes calculating 306 the first accuracy based on a comparison of the first code structure representation and the second code structure representation.

[0047] In an embodiment, calculating the first code structure representation of the code portion of the first programming language is based on one or more software metrics. In an embodiment, calculating the second code structure representation of the converted code portion of the second programming language is also based on one or more software metrics. In one or more embodiments, the one or more software metrics include code complexity metrics. In an embodiment, calculating the first code structure representation of the code portion of the first programming language is based on a weighted combination of multiple software metrics among one or more software metrics.

[0048] For further explanation, Figure 4 FIG. shows a flowchart of another example method for large language model code translation error detection according to some embodiments of the present disclosure. Figure 4 The method of Figure 2The method lies in determining the difference between the first accuracy and the historical accuracy of the conversion from the first programming language to the second programming language, including determining 402 the historical accuracy based on the accuracy of at least one previous conversion from the first programming language to the second programming language for another code portion.

[0049] For further explanation, Figure 5 FIG. shows a flowchart of another example method for large language model code translation error detection according to some embodiments of the present disclosure. Figure 5 The method extends Figure 2 The method lies in converting 204 a code portion to the second programming language, including using a generative artificial intelligence model to convert 502 the code portion to the second programming language. In a particular embodiment, the generative artificial intelligence model includes a large language model.

[0050] Figure 5 The method further extends Figure 2 The method lies in indicating 210 a potential error in a code portion of the first programming language based on the difference between the first accuracy and the historical accuracy being greater than a predetermined value, including providing 504 an indication of the potential error in the code portion to the generative artificial intelligence model. In a particular embodiment, the generative artificial intelligence model is further trained based on the indication of the potential error in the code portion of the first programming language. In a particular example, the generative artificial intelligence model ignores the code portion and does not use it for training based on the likelihood of the error indicated in the code portion.

[0051] The above examples are provided in the context of translation error detection when migrating an application from an original source code to a new source code. It should be understood that the term "original source code" as used herein refers to an instance of the application for which the new source code is being verified for accuracy, and should not be construed as limiting the term to mean the earliest implementation of the application. Although the embodiments are useful for migrating or porting an application from one programming language to a different programming language and from legacy code to a more modern programming language, it can be further understood that in some examples, the original source code and the new source code can be written in the same programming language.

[0052] In view of the above, the large language model code translation error detection according to the present disclosure provides many advantages. The identification of errors within the original source code provides a greater chance of success in producing an accurate conversion of the original source code to a new source code in a different programming language. Such errors can be corrected so that the conversion of the original source code to the new source code can increase the accuracy of the new source code and prevent or reduce execution errors in the converted code.

[0053] Aspects of the present disclosure are described by narrative text, flowcharts, block diagrams of computer systems, and / or block diagrams of machine logic included in computer program product (CPP) embodiments. With respect to any flowchart, depending on the technology involved, operations may be performed in an order different from that shown in a given flowchart. For example, again depending on the technology involved, two operations shown in consecutive flowchart blocks may be performed in reverse order, as a single integrated step, simultaneously, or in a manner that at least partially overlaps in time.

[0054] A computer program product embodiment (“CPP embodiment” or “CPP”) is a term used in the present disclosure to describe any collection of one or more storage media (also referred to as “media”) jointly included in a set of one or more storage devices, the set of one or more storage devices jointly including machine-readable code corresponding to instructions and / or data for performing computer operations specified in a given CPP claim. A “storage device” is any tangible device that can hold and store instructions used by a computer processor. By way of non-limitation, computer-readable storage media may be electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, mechanical storage media, or any suitable combination of the foregoing. Some known types of storage devices that include these media include: magnetic disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded devices such as punched cards or pits / lands formed in the main surface of a disc, or any suitable combination of the foregoing. Computer-readable storage media, as the term is used in the present disclosure, should not be construed to store in the form of a transitory signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, optical pulses propagating through an optical fiber cable, electrical signals transmitted through a wire, and / or other transmission media. As will be understood by those skilled in the art, data is typically moved at certain incidental points in time during the normal operation of a storage device, such as during access, defragmentation, or garbage collection, but this does not make the storage device transitory because the data is not transitory when it is stored.

[0055] The description of the various embodiments of the present disclosure has been presented for purposes of illustration, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein were chosen to best explain the principles of the embodiments, the practical application, or technical improvements made to the technologies that are available in the market, or to enable other ordinary skilled artisans in the art to understand the embodiments disclosed herein.

Claims

1. A method, comprising: receiving a code portion in a first programming language; converting the code portion into a second programming language; calculating a first accuracy of converting the code portion into the second programming language; determining a difference between the first accuracy and a historical accuracy of conversions from the first programming language to the second programming language; as well as Based on the difference between the first accuracy and the historical accuracy being greater than a predetermined value, a potential error in the code portion in the first programming language is indicated.

2. The method of claim 1 , wherein calculating the first accuracy of converting the code portion into the second programming language further comprises: computing a first code structure representation of the code portion in the first programming language; computing a second code structure representation of the converted code portion in the second programming language; as well as The first accuracy is calculated based on a comparison of the first code structure representation and the second code structure representation. 3 . The method of claim 2 , wherein computing the first code structure representation of the code portion of the first programming language further comprises computing the first code structure representation of the code portion of the first programming language based on one or more software metrics.

4. The method of claim 3, wherein computing the second code structure representation of the converted code portion of the second programming language further comprises computing the second code structure representation of the converted code portion of the second programming language based on the one or more software metrics. The method of claim 4 , wherein the one or more software metrics include a code complexity metric.

6. The method of claim 3, wherein calculating the first code structure representation of the code portion of the first programming language based on one or more software metrics further comprises calculating the first code structure representation of the code portion of the first programming language based on a weighted combination of multiple software metrics among the one or more software metrics.

7. The method according to claim 1, further comprising: The historical accuracy is determined based on the accuracy of at least one previous conversion of another code portion from the first programming language to the second programming language.

8. The method of claim 1, wherein converting the code portion to a second programming language comprises converting the code portion to a second programming language using a generative artificial intelligence model.

9. The method according to claim 8, wherein: The generative artificial intelligence model includes a large language model.

10. The method of claim 8, wherein indicating the potential error in the code portion based on the difference between the first accuracy and the historical accuracy being greater than a predetermined value further comprises: An indication of the potential error in the code portion is provided to the generative artificial intelligence model.

11. The method according to claim 1, further comprising: determining that the difference between the first accuracy and the historical accuracy is less than the predetermined value; determining that the converted code portion is non-executable; as well as Indicates that the converted code portion is not executable.

12. An apparatus comprising: Processing equipment; as well as A memory operatively coupled to the processing device, wherein the memory stores computer program instructions which, when executed, cause the processing device to perform a method according to any one of claims 1-11.

13. A computer program product comprising computer program instructions executable by a processor to cause the processor to perform the method according to any one of claims 1 to 11.

Images