Method, device, and computer program (checking code completeness with hapax legomenon)

Abstract

To solve the problem that, even though a code may be compiled and executed correctly, the presence of single-use programming constituents may affect performance and memory usage.SOLUTION: A method of checking code completeness with hapax legomenon is provided, comprising: receiving code generated by a generative artificial intelligence (AI) model; determining whether the code includes one or more single-use programming constituents; and flagging the code in response to the code including the one or more single-use programming constituents.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a method, apparatus, and product for checking code completeness using a hapax legomenon. Generative artificial intelligence (AI) can be used to generate code such as application source code. Such code may be based on a prompt provided to a generative AI model that describes the function the code is to perform. Such code may also include a conversion from one programming language implemented by the generative AI model to another. When generating code, the generative AI model may introduce single-use programming components into the code, which are programming components such as functions or variables that are defined but never used. The code can be compiled and executed correctly, but the presence of these single-use programming components can affect performance and memory usage.

Summary of the Invention

Problems to be Solved by the Invention

[0002] The code can be compiled and executed correctly, but the presence of these single-use programming components can affect performance and memory usage.

Means for Solving the Problems

[0003] According to embodiments of the present disclosure, various methods, apparatuses, and products for checking code completeness using hapax legomenon are described herein. In some aspects, checking code completeness using hapax legomenon includes receiving code generated by a generative artificial intelligence (AI) model; determining whether the code includes one or more single-use programming components; and flagging the code in response to the code including the one or more single-use programming components. This provides the advantage of identifying single-use programming components within AI-generated code, the significance of whose presence is recognized for the first time by this disclosure. This may enable the removal of identified single-use programming components to improve the performance of AI-generated code and may enable the enhancement of generative AI models for future code generation. In some aspects, the apparatus may comprise a processing device; and a memory operably coupled to the processing device, the memory storing computer program instructions that, when executed, cause the processing device to perform this method. In some aspects, a computer program product comprising a computer-readable storage medium can store computer program instructions that, when executed, perform this method.

[0004] In some embodiments, determining whether the code includes the one or more single-use programming components may involve identifying any single-use programming components within the code. This provides the advantage of identifying and flagging any single-use programming components within the AI-generated code, regardless of whether these single-use programming components were introduced by the generative AI model or existed within some base code. In some embodiments, determining whether the code includes the one or more single-use programming components involves identifying single-use programming components within the transformed code that do not have corresponding single-use programming components within the base code. This provides the advantage of identifying and flagging only those single-use programming components introduced by the generative AI model, excluding those that originally existed within the base code.

Brief Description of the Drawings

[0005]

Figure 1

[0006]

Figure 2

[0007]

Figure 3

[0008]

Figure 4

DETAILED DESCRIPTION OF THE INVENTION

[0009] Generative artificial intelligence (AI) uses models such as neural networks to generate content such as text, code, graphics, animations, videos, audiovisual representations, audio, speech, etc. in response to a prompt. Such a prompt may include natural language input that describes the content to be generated by the generative AI model. Such a prompt may also include other data that may be used by the generative AI model when generating the requested content. For example, a generative AI model may be used to generate code, such as source code, based on a description of the function that the generated code is to implement. As another example, a generative AI model may be used to convert code from one programming language to another by providing the base code as input to the generative AI model.

[0010] Depending on the specific model used and how that model was trained, the code generated by a generative AI model may contain certain bugs, anomalies, or other undesirable characteristics. In particular, some AI-generated code may contain a "hapax legomenon," which is a word or expression that occurs in some context. In the context of source code, a hapax legomenon may include single-use programming components. Programming components may include, for example, functions or variables that can be defined within the code. Such programming components may be considered "single-use" in that they are defined but not used. For example, a single-use variable may include a variable that is defined or instantiated but not used in any particular function or operation. As another example, a single-use function may include a function that is defined but not called by any other part of the code and thus never executed.

[0011] Existing benchmark or test techniques for AI-generated code may not be affected by including single-use programming components. For example, AI-generated code may compile and execute correctly despite including single-use programming components. However, including them can still affect the performance and memory usage of programs that use the AI-generated code. Therefore, it can be beneficial to identify where single-use programming components exist within AI-generated code.

[0012] Referring now to FIG. 1, an exemplary computing environment according to an aspect of the present disclosure is shown. Computing environment 100 is an example of an environment for executing at least a portion of the computer code involved in implementing the various methods described herein, for example, including code analysis module 107. In addition to block 107, computing environment 100 includes, for example, computer 101, wide area network (WAN) 102, end user device (EUD) 103, remote server 104, public cloud 105, and private cloud 106. In the present embodiment, computer 101 includes a processor set 110 (including processing circuit 120 and cache 121), communication fabric 111, volatile memory 112, persistent storage 113 (including the operating system 122 and block 107 identified above), a set of peripheral devices 114 (including user interface (UI) device set 123, storage 124, and Internet of Things (IoT) sensor set 125), and network module 115. Remote server 104 includes remote database 130. Public cloud 105 includes gateway 140, cloud orchestration module 141, host physical machine set 142, virtual machine set 143, and container set 144.

[0013] Computer 101 may take the form of a desktop computer, laptop computer, tablet computer, smartphone, smartwatch or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device now known or later to be developed that can execute a program, access a network, or query a database such as remote database 130. As is well understood in the technical field of computer technology and as appropriate to the technology, the implementation of a computer-implemented method may be distributed among multiple computers and / or among multiple locations. On the other hand, in this presentation of computing environment 100, for purposes of keeping the presentation as simple as possible, the detailed description focuses on a single computer, specifically computer 101. Although not shown in the cloud of FIG. 1, computer 101 may be located within the cloud. On the other hand, computer 101 is not required for the cloud, except for any scope that may be affirmatively shown.

[0014] Processor set 110 includes one or more computer processors of any type now known or later to be developed. Processing circuitry 120 may be distributed across multiple packages, such as multiple tuned integrated circuit chips. Processing circuitry 120 may implement multiple processor threads and / or multiple processor cores. Cache 121 is memory located within a processor chip package and is typically used for data or code that should be available for rapid access by a thread or core executing on processor set 110. Cache memory is typically organized into multiple levels according to its relative proximity to the processing circuitry. Alternatively, some or all of the cache for the processor set may be located "off chip". In some computing environments, processor set 110 may be designed to cooperate with qubits to perform quantum computing.

[0015] Computer-readable program instructions are typically loaded onto computer 101 and cause a series of operational steps to be performed by processor set 110 of computer 101, thereby executing a computer-implemented method, and thus the instructions so executed will instantiate the method specified in the flowchart and / or description of the computer-implemented method included in this document. These computer-readable program instructions are stored in various types of computer-readable storage media such as cache 121 and other storage media discussed below. The program instructions and associated data are accessed by processor set 110 to control and direct the implementation of the computer-implemented method. In computing environment 100, at least some of the instructions for implementing the computer-implemented method may be stored within block 107 in persistent storage 113.

[0016] Communication fabric 111 is a signal conduction path that enables various components of computer 101 to communicate with each other. Typically, this fabric is made up of switches and conductive paths such as buses, bridges, physical input / output ports, and switches and conductive paths that make up the like. Other types of signal communication paths such as optical fiber communication paths and / or wireless communication paths may be used.

[0017] Volatile memory 112 is any type of volatile memory known now or to be developed in the future. Examples include dynamic random access memory (RAM) or static RAM. Typically, volatile memory 112 is characterized by random access, although this is not required unless affirmatively indicated. In computer 101, volatile memory 112 is located within a single package and is internal to computer 101, although alternatively or additionally, the volatile memory may be distributed across multiple packages and / or located external to computer 101.

[0018] The persistent storage 113 is any form of non-volatile storage for a computer, whether currently known or to be developed in the future. The non-volatility of this storage means that the stored data is retained regardless of whether power is being supplied to the computer 101 and / or directly to the persistent storage 113. The persistent storage 113 can be read-only memory (ROM), but is typically at least part of the persistent storage that allows for writing of data, deletion of data, and rewriting of data. Some well-known forms of persistent storage include magnetic disks and solid-state storage devices. The operating system 122 can take multiple forms, such as various known proprietary operating systems that employ a kernel or open-source portable operating system interface-type operating systems. The code contained within block 107 typically includes at least some of the computer code involved in implementing the computer-implemented methods described herein.

[0019] The peripheral device set 114 includes a set of peripheral devices of the computer 101. The data communication connection between the peripheral devices and other components of the computer 101 can be implemented in various ways, such as a Bluetooth (registered trademark) connection, a Near Field Communication (NFC) connection, a connection by a cable (such as a Universal Serial Bus (USB) type cable), a plug-in connection (e.g., Secure Digital (SD) card), a connection by a local area communication network, and even a connection by a wide area network such as the Internet. In various embodiments, the UI device set 123 can include components such as a display screen, a speaker, a microphone, wearable devices (such as goggles and smartwatches), a keyboard, a mouse, a printer, a touchpad, a game controller, and a haptic device. The storage 124 is an external storage such as an external hard drive or an insertable storage such as an SD card. The storage 124 can be persistent and / or volatile. In some embodiments, the storage 124 can take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where the computer 101 is required to have a large amount of storage (e.g., the computer 101 locally stores and manages a large database), this storage can be provided by a peripheral storage device designed to store a very large amount of data, such as a storage area network (SAN) shared by a plurality of geographically dispersed computers. The IoT sensor set 125 is composed of sensors that can be used in Internet of Things applications. For example, one sensor may be a thermometer, and another sensor may be a motion detector.

[0020] The network module 115 is an aggregation of computer software, hardware, and firmware that enables the computer 101 to communicate with other computers through the WAN 102. The network module 115 may include hardware such as a modem or a Wi-Fi (registered trademark) signal transceiver, software for packetizing and / or depacketizing data for communication network transmission, and / or web browser software for communicating data through the Internet. In some embodiments, the network control function and the network transfer function of the network module 115 are implemented on the same physical hardware device. In other embodiments (e.g., embodiments that utilize software-defined networking (SDN)), the control function and the transfer function of the network module 115 are implemented on physically separate devices, and as a result, the control function manages multiple different network hardware devices. Computer-readable program instructions for implementing a computer-implemented method may typically be downloaded to the computer 101 from an external computer or an external storage device through a network adapter card or a network interface included within the network module 115.

[0021] The WAN 102 is any wide area network (e.g., the Internet) capable of communicating computer data over non-local distances by any technology for communicating computer data, whether known now or to be developed in the future. In some embodiments, the WAN 102 can be replaced and / or supplemented by a local area network (LAN) designed to communicate data between devices located in a local area, such as a Wi-Fi network. A WAN and / or a LAN typically includes computer hardware such as copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and edge servers.

[0022] The end - user device (EUD) 103 is any computer system that is used and controlled by an end - user (e.g., a customer of an enterprise operating the computer 101), and can take any of the forms discussed above in relation to the computer 101. The EUD 103 typically receives useful data from the operation of the computer 101. For example, in a virtual case where the computer 101 is designed to provide recommendations to the end - user, these recommendations are typically communicated from the network module 115 of the computer 101 to the EUD 103 via the WAN 102. Thus, the EUD 103 can display or otherwise present the recommendations to the end - user. In some embodiments, the EUD 103 can be a client device such as a thin - client, a thick - client, a mainframe computer, a desktop computer, etc.

[0023] The remote server 104 is any computer system that provides at least some data and / or functionality to the computer 101. The remote server 104 can be controlled and used by the same entity that operates the computer 101. The remote server 104 represents a machine that collects and stores useful data for use by other computers such as the computer 101. For example, in a virtual case where the computer 101 is designed and programmed to provide recommendations based on historical data, this historical data can be provided from the remote database 130 of the remote server 104 to the computer 101.

[0024] The public cloud 105 is any computer system that provides on-demand availability of computer system resources and / or other computing capabilities, particularly data storage (cloud storage) and computing power, for use by multiple entities without direct and active management by the user. Cloud computing typically leverages resource sharing to achieve coherence and economies of scale. The direct and active management of the computing resources of the public cloud 105 is performed by the computer hardware and / or software of the cloud orchestration module 141. The computing resources provided by the public cloud 105 are typically implemented by a virtual computing environment that runs on various computers that make up the host physical machine set 142, which is the population of physical computers that are within and / or available to the public cloud 105. The virtual computing environment (VCE) typically takes the form of virtual machines from a virtual machine set 143 and / or containers from a container set 144. It is understood that these VCEs can be stored as images and transferred among and between various physical machine hosts either as images or after instantiation of the VCE. The cloud orchestration module 141 manages the transfer and storage of images, deploys new instantiations of the VCE, and manages the active instantiation of the VCE deployment. The gateway 140 is an aggregate of computer software, hardware, and firmware that enables the public cloud 105 to communicate through the WAN 102.

[0025] Next, some further explanation is provided regarding a virtualized computing environment (VCE). A VCE can be stored as an "image". A new active instance of a VCE can be instantiated from the image. Two well-known types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to a feature of the operating system where the kernel enables the existence of multiple isolated instances of user space, called containers. These isolated instances of user space typically behave as actual computers from the perspective of the programs running within them. A computer program running on a normal operating system can utilize all the resources of that computer, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, a program running within a container can only use the contents of the container and the devices assigned to the container. This is a feature known as containerization.

[0026] The private cloud 106 is similar to the public cloud 105, except that computing resources are available only for use by a single enterprise. The private cloud 106 is shown as communicating with the WAN 102, while in other embodiments, the private cloud may be completely disconnected from the Internet and accessible only via a local / private network. A hybrid cloud is a composition of multiple clouds of different types (e.g., private, community, or public cloud types), often implemented by different vendors. Each of the multiple clouds remains a separate and discrete entity, but the larger hybrid cloud architecture is tied together by standardized or proprietary technologies that enable orchestration, management, and / or data / application portability between the multiple constituent clouds. In this embodiment, both the public cloud 105 and the private cloud 106 are part of a larger hybrid cloud.

[0027] For further illustration, FIG. 2 depicts a flowchart of an exemplary method for checking code completeness using a hapax legomenon, according to some embodiments of the present disclosure. The method of FIG. 2 may be implemented, for example, by the code analysis module 107 of FIG. 1. As an example, in some embodiments, the method of FIG. 2 may be implemented in response to a request to compile some AI-generated code or during the compilation of such code. As another example, in some embodiments, the method of FIG. 2 may be implemented in response to receiving some code from an AI model generated based on some request or prompt.

[0028] The method of FIG. 2 includes receiving (202) code generated by a generative AI model. In some embodiments, receiving (202) code generated by a generative AI model includes receiving the code as output by the generative AI model. For example, the generative AI model may be configured to output code and provide the code to a process or service implementing the method of FIG. 2. In some embodiments, receiving (202) code generated by a generative AI model includes loading the code from storage where the code was stored after being generated by the generative AI model.

[0029] In some embodiments, the code generated by a generative AI model includes code generated based on a prompt that includes a natural language description of some function that the code is to implement or perform. For example, the prompt to the generative AI model may request code to perform a particular task (e.g., opening a network connection to a particular destination, instantiating a database table with particular features, or any other task that can be understood).

[0030] In some embodiments, the code generated by a generative AI model includes code in some programming language that has been converted from other code in a different programming language. For example, the generative AI model may be provided with some Cobol code and be prompted to convert the Cobol code to Java®. In the examples regarding code conversion using the generative AI model described below, the code input to the generative AI model is hereinafter referred to as "base code", while the code output by the generative AI model is hereinafter referred to as "converted code".

[0031] The method of FIG. 2 also includes determining (204) whether the code includes one or more single-use programming components. As described above, single-use programming components are programming components such as variables or functions that are defined in the code but not used. For example, a single-use programming component may include a variable that is defined but not otherwise used or accessed within the code. As another example, a single-use programming component may include a function that is defined but not called in another part of the code and thus never executed.

[0032] In some embodiments, determining (204) whether the code includes one or more single-use programming components may include identifying (206) any single-use programming components within the code. Thus, any single-use programming components present within the code may be included within one or more single-use programming components. Embodiments in which determining (204) whether the code includes one or more single-use programming components includes identifying (206) any single-use programming components within the code provide the advantage of identifying any or all single-use programming components within the code, regardless of whether they existed within some base code from which the AI-generated code was derived. This may enable these originally existing single-use programming components to be removed in order to further optimize the AI-generated code with respect to the base code. In some embodiments, the code may be parsed to generate a table or other data structure of programming components. The programming components may be added to such a table or data structure in response to detecting their definitions and / or their use within the code.

[0033] Accordingly, in some embodiments, identifying any single-use programming component in the code (206) may include accessing a table or data structure of programming components and determining whether an entry indicates that the programming component is not being used. In some embodiments, such a table or data structure may maintain the number of times a given program component is used in the code. In some embodiments, such a table or data structure may maintain a binary indication of whether a given program component is being used in the code. In some embodiments, such a table structure may be generated during compilation of the code. For example, such a table or data structure may be embodied as an extension of a symbol table generated during compilation of the code. In some embodiments, such a table structure may be generated independently of any compilation of the code. One of ordinary skill in the art will understand that other techniques that do not use such a table or data structure may also be used to identify single-use programming components.

[0034] In embodiments where the code includes transformed code generated from some base code, the base code may itself include single-use programming components. The transformed code may also include single-use programming components by virtue of their inclusion in the base code. The presence of single-use programming components in the transformed code may not indicate any faults or program errors in the generated AI model. Thus, identifying any single-use programming components in the code (206) may cause single-use programming components to be misidentified or treated as errors.

[0035] Accordingly, in some embodiments where the code is the converted code, determining (204) whether the code includes one or more single-use programming components may include identifying (208) in the converted code single-use programming components that do not have corresponding single-use programming components in the base code. Thus, if such single-use programming components in the converted code are not found in the base code, it may be determined (204) that the code includes one or more single-use programming components (e.g., for the purposes of flagging described below). Embodiments in which determining (204) whether the code includes one or more single-use programming components includes identifying (208) in the converted code single-use programming components that do not have corresponding single-use programming components in the base code enable identifying only those single-use programming components introduced by the generative AI model and ignoring those that originally existed in the base code. This can be used to identify and correct defects or faults in the generative AI model.

[0036] For example, in some embodiments, the generative AI model may be trained or configured to use the same function names and / or variable names across the base code and the converted code. Accordingly, in some embodiments, both the base code and the converted code may be parsed as described above to identify their programming components and whether these programming components are being used within their respective codes. Identifying (208) single-use programming components in the converted code that do not have corresponding single-use programming components in the base code using shared names or other identifiers may include identifying single-use programming components in a table or data structure for the converted code that do not have corresponding (e.g., matching identifier) entries in a table or data structure for the base code.

[0037] In some embodiments, determining whether the code includes one or more single-use programming components (204) may include (e.g., mutually exclusively) either identifying any single-use programming components within the code (206) or identifying single-use programming components within the converted code that do not have corresponding single-use programming components in the base code (208). In some embodiments, determining whether the code includes one or more single-use programming components (204) may include both identifying any single-use programming components within the code (206) and identifying single-use programming components within the converted code that do not have corresponding single-use programming components in the base code (208). For example, determining whether the code includes one or more single-use programming components (204) may include identifying any single-use programming components within the code (206). Next, single-use programming components that do not have corresponding single-use programming components in the base code may be identified (208) from these identified (206) single-use programming components. As described in more detail below, the code and / or single-use programming components may be flagged differently depending on whether they were originally included in the base code.

[0038] The method of FIG. 2 also includes flagging code (210) in response to the code including one or more single-use programming components. Flagging the code (210) can include various actions indicating that the code included one or more single-use programming components. For example, in some embodiments, flagging the code (210) can include storing (212) log data identifying one or more single-use programming components. Such log data can include metadata of the code or other data that can be understood. In some embodiments, such log data can indicate where a particular single-use programming component exists within the code. In some embodiments, such log data can indicate the type of the identified single-use programming component (e.g., variable or function). In some embodiments where the code includes transformed code, such log data can indicate whether the corresponding single-use programming component existed in the base code and, if so, where it was located.

[0039] In some embodiments, flagging the code (210) may include causing an exception (214). Causing an exception (214) may halt an operation applied to the code, such as a code compilation operation or a code execution operation, or otherwise cause the exception to be handled. In some embodiments, the exception may include attributes similar to those described above with respect to log data, including where a single-use programming component was identified, the type of the identified single-use programming component, and the like, or may encode them. In some embodiments where the code includes transformed code, causing an exception (214) may be based on whether a single-use programming component was present in the base code, thereby enabling different types of exceptions to be handled differently. For example, a warning may be generated if a single-use programming component was present in the base code, while an error may be generated if a single-use programming component was not present in the base code.

[0040] Other actions may also be taken in response to flagging the code (210). For example, in some embodiments, a identified single-use programming component may be automatically removed from the code. As another example, in some embodiments, a generative AI model may be provided with an indication as feedback that the generated code included one or more single-use programming components.

[0041] In some embodiments, the code can be flagged (210) based on criteria other than including single-use programming components within the code. For example, in some embodiments, the code can be flagged according to a technique similar to that described above if the programming components are included in the converted code a number of times that is not equal compared to the base code (210). This would include the situation described above where single-use programming components are included in the converted code but not in the base code. This would also include situations where the programming components are used one or two times (e.g., some amount, some amount below a threshold) within the converted code and are used a different amount or not at all within the base code.

[0042] The techniques described herein enable the identification of single-use programming components within AI-generated code. This can enable the removal of single-use programming components that can affect program performance or memory usage. This also provides a feedback mechanism regarding the code generation AI model to improve its code generation to avoid including single-use programming components.

[0043] The techniques described above have been described with respect to code received from a code generation AI model, but the techniques described herein can also be implemented by a code generation AI model as part of the code generation process. For example, after generating some code, the code generation AI model can identify and potentially remove single-use programming components prior to outputting the code. Additionally, the techniques described herein can be applied to code that is not generated by a code generation AI model.

[0044] For further explanation, FIG. 3 depicts a flowchart of another exemplary method for checking code completeness using a haplogroup in accordance with some embodiments of the present disclosure. The method of FIG. 3 also includes: receiving code generated by a generative artificial intelligence (AI) model (202); determining whether the code includes one or more single-use programming components (204), including identifying any single-use programming components within the code (206); or identifying single-use programming components within the transformed code that do not have corresponding single-use programming components in the base code (208); and flagging the code in response to the code including one or more single-use programming components (210), including storing log data identifying the one or more single-use programming components (212); or generating an exception (214). In this regard, the method of FIG. 3 is similar to FIG. 2 in that it includes flagging the code in response to the code including one or more single-use programming components (210).

[0045] The method of FIG. 3 differs from that of FIG. 2 in that it includes determining (204) whether the code includes one or more single-use programming components, and determining (302) whether the tendency regarding the degree of use of programming components in the converted code is different from the tendency regarding the degree of use of programming components in the base code. In some embodiments, the degree of use regarding programming components in the code can be tracked and counted by parsing the code. For example, for each identified programming component in a piece of code, the number of times the programming component is used (e.g., each time a function is called, each time a variable is used, and the like) can be counted. Using these counts, a tendency regarding the degree of use of programming components can be calculated for a piece of code. As an example, the tendency can explain or plot how many programming components are used a given number of times (e.g., N programming components are used 5 times and M programming components are used 4 times). The tendency regarding the base code can be compared with the tendency regarding the converted code as this plot approaches the number of programming components that are used only 0 times (e.g., the number of single-use programming components). The difference between these tendencies (e.g., the difference in the rate of change in the tendency exceeding some threshold) can indicate that the converted code includes a significant number of single-use programming components compared to the base code. Accordingly, this can trigger the determination (204) that the code includes one or more single-use programming components and can trigger flagging (210) as described above. In some embodiments, this can trigger flagging of the entire code (210) (e.g., without identifying specific single-use programming components), or if the code is flagged (210), it can trigger identification of specific single-use programming components.

[0046] For further explanation, FIG. 4 depicts a flowchart of another exemplary method for checking code completeness using a hapax legomenon, according to some embodiments of the present disclosure. The method of FIG. 4 also includes: receiving code generated by a generative artificial intelligence (AI) model (202); determining whether the code includes one or more single-use programming components (204), including identifying any single-use programming components within the code (206); or identifying single-use programming components within the transformed code that do not have corresponding single-use programming components in the base code (208); and flagging the code in response to the code including one or more single-use programming components (210), including storing log data identifying the one or more single-use programming components (212); or generating an exception (214). In this regard, the method of FIG. 4 is similar to FIG. 2 in that it includes flagging the code in response to the code including one or more single-use programming components (210).

[0047] The method of FIG. 4 differs from that of FIG. 2 in that the method of FIG. 4 also includes providing (402) the generative AI model with an indication that the code is flagged for containing one or more single-use programming components. In other words, an indication that the code generated by the generative AI model contains single-use programming components is provided to the generative AI model. In some embodiments, the indication to the generative AI model may include the stored (212) log data and / or the generated (214) exceptions described above. In some embodiments, the indication to the generative AI model may include any of the information described as potentially included within the log data or exceptions described above. For example, the indication may identify specific single-use programming components within the code, the type of single-use programming components, whether the single-use programming components were present within the base code input to the generative AI model, and the like. The indication may also include other information that may be understood.

[0048] The generative AI model may use the indication in various ways. For example, in some embodiments, the indication is provided to the generative AI model as part of a prompt such that code without single-use programming components can be regenerated. As another example, in some embodiments, the indication may be provided as part of the training data for retraining the generative AI model. This may cause the generative AI model to produce less single-use programming components within the code after retraining.

[0049] Various aspects of the present disclosure are described by the block diagrams of machine logic included in the description, flowcharts, block diagrams of computer systems, and / or embodiments of computer program products (CPP). For any flowchart, depending on the relevant technology, operations can be performed in an order different from that shown in a given flowchart. For example, again depending on the relevant technology, two operations shown in successive flowchart blocks can be performed in reverse order, as a single integrated step, simultaneously, or at least partially in a temporally overlapping manner.

[0050] An embodiment of a computer program product (referred to as "CPP embodiment" or "CPP") in the present disclosure is a term used to describe any set of one or more storage media (also referred to as "media") collectively included in a set of one or more storage devices, which collectively includes machine-readable code corresponding to instructions and / or data for performing computer operations specified in a given CPP claim. A "storage device" is any tangible device that can hold and store instructions for use by a computer processor. A computer-readable storage medium can be, but is not limited to, an electronic storage medium, a magnetic storage medium, an optical storage medium, an electromagnetic storage medium, a semiconductor storage medium, a mechanical storage medium, or any suitable combination of the foregoing. Some known types of storage devices including these media are floppy disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded devices (such as punched cards or pits / lands formed on the major surface of a disk), or any suitable combination of the foregoing. The term computer-readable storage medium as used in the present disclosure should not be construed as storage in the form of its own transient signals, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, optical pulses passing through an optical fiber cable, electrical signals communicated by a wire, and / or other transmission media. As will be understood by those skilled in the art, data typically moves occasionally during the normal operation of a storage device, such as during access, defragmentation, or garbage collection, but the data is not transient while it is stored, so the storage device is not transient.

[0051] The descriptions of the various embodiments of the present disclosure have been presented for purposes of illustration, but are not intended to be exhaustive or limiting of the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein has been chosen to best explain the principles of the embodiments, the practical application, or technical improvements in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. Receiving code generated by an artificial intelligence (AI) model; Determining whether the code includes one or more single-use programming components; and Flagging the code in response to the code including the one or more single-use programming components A method comprising:

2. The method of claim 1, wherein the determining whether the code includes one or more single-use programming components includes identifying any single-use programming components within the code.

3. The method of claim 1, wherein the code generated by the generating AI model comprises converted code in a first programming language converted from base code in a second programming language by the generating AI model.

4. The method of claim 3, wherein the determining whether the code includes one or more single-use programming components includes identifying single-use programming components having no corresponding single-use programming components in the base code within the converted code.

5. The method of claim 3, wherein the determining whether the code includes one or more single-use programming components includes determining whether a trend regarding the occurrence degree of programming components within the converted code is different from a trend regarding the occurrence degree of programming components within the base code.

6. The method according to any one of claims 1 to 5, wherein the one or more single-use programming components have defined variables that are not used.

7. The method according to any one of claims 1 to 5, wherein the one or more single-use programming components have defined functions that are not called.

8. The method according to any one of claims 1 to 5, further comprising providing an indication to the generating AI model that the code has been flagged for including the one or more single-use programming components.

9. The step of flagging the code in response to the code including the one or more single-use programming components includes storing log data identifying the one or more single-use programming components, the method according to any one of claims 1 to 5.

10. The step of flagging the code in response to the code including the one or more single-use programming components includes generating an exception, the method according to any one of claims 1 to 5.

11. A processing device; and A memory operably coupled to the processing device, wherein the memory, when executed, causes the processing device to: Receive code generated by a generative artificial intelligence (AI) model; Determine whether the code includes one or more single-use programming components; Flag the code in response to the code including the one or more single-use programming components Store computer program instructions. An apparatus comprising.

12. The procedure for determining whether the code includes the one or more single-use programming components includes a procedure for identifying any single-use programming components within the code, the apparatus according to claim 11.

13. The code generated by the generative AI model has converted code in a first programming language converted from base code in a second programming language by the generative AI model, the apparatus according to claim 11.

14. The procedure for determining whether the code includes the one or more single-use programming components includes a procedure for identifying single-use programming components having no corresponding single-use programming components in the base code within the converted code, the apparatus according to claim 13.

15. The procedure for determining whether the code includes the one or more single-use programming components includes a procedure for determining whether a tendency regarding the occurrence degree of programming components within the converted code is different from a tendency regarding the occurrence degree of programming components within the base code, the apparatus according to claim 13.

16. The apparatus according to any one of claims 11 to 15, wherein the one or more single-use programming components have defined variables that are not being used.

17. The apparatus according to any one of claims 11 to 15, wherein the one or more single-use programming components have defined functions that are not being called.

18. The apparatus according to any one of claims 11 to 15, wherein the computer program instructions, when executed, further cause the processing device to provide an indication to the generated AI model that the code is flagged for including the one or more single-use programming components.

19. The apparatus according to any one of claims 11 to 15, wherein the procedure for flagging the code in response to the code including the one or more single-use programming components has a procedure for storing log data identifying the one or more single-use programming components.

20. The apparatus according to any one of claims 11 to 15, wherein the procedure for flagging the code in response to the code including the one or more single-use programming components has a procedure for generating an exception.

21. A computer program for causing a computer to: receive code generated by a generated artificial intelligence (AI) model; determine whether the code includes one or more single-use programming components; and flag the code in response to the code including the one or more single-use programming components.

22. The computer program according to claim 21, wherein the procedure for determining whether the code includes the one or more single-use programming components has a procedure for identifying any single-use programming components within the code.

23. The computer program according to claim 21, wherein the code generated by the generated AI model has converted code in a first programming language converted from base code in a second programming language by the generated AI model.

24. The procedure for determining whether the code includes the one or more single-use programming components comprises a procedure for identifying, within the converted code, single-use programming components that are not present in the base code, the computer program according to claim 23.

25. The procedure for determining whether the code includes the one or more single-use programming components comprises a procedure for determining whether a tendency regarding the degree of occurrence of programming components within the converted code is different from a tendency regarding the degree of occurrence of programming components within the base code, the computer program according to claim 23.

Images