Combination code generation for training artificial intelligence systems

Abstract

Combination code generation systems and methods for training an artificial intelligence system include reducing a plurality of code portion combinations to a code portion combination subset that satisfies one or more constraints using a combination reduction, generating one or more composition programs using the code portion combination subset, and training the artificial intelligence system using the composition programs.

Description

Technical Field

[0001] The present disclosure relates to methods, apparatuses, and products for generating combined code for training an artificial intelligence system. Background Art

[0002] Modern computer environments may involve the use of artificial intelligence systems, which require a large amount of configuration for use. For example, some artificial intelligence systems include artificial intelligence or machine learning models composed of various algorithms. These algorithms need to be trained on specific data sets for purposes such as prediction, analysis, etc. Training data sets are usually large, complex, and difficult to generate or maintain. For different types of data, such as data that is not raw text and is structured in other ways, the way data is generated or acquired and the amount of data that can be used by artificial intelligence or machine learning models may be limited. In addition, obtaining training data may also pose other challenges, especially when the data cannot be freely provided to any user and requires permission or authorization to obtain. Summary of the Invention

[0003] According to embodiments of the present disclosure, various methods, apparatuses, and products for generating combined code for training an artificial intelligence system are described herein. In some aspects, generating combined code for training an artificial intelligence system includes using combined reduction to combine and reduce multiple code parts into a subset of combined code parts that meet one or more constraints, using the subset of combined code parts to generate one or more synthetic programs, and using the synthetic programs to train the artificial intelligence system. Brief Description of the Drawings

[0004] Figure 1 A block diagram of an example computing environment for generating combined code for training an artificial intelligence system according to some embodiments of the present disclosure is listed.

[0005] Figure 2 A flowchart of an example method for generating combined code for training an artificial intelligence system according to some embodiments of the present disclosure is listed.

[0006] Figure 3 A flowchart of another example method for generating combined code for training an artificial intelligence system according to some embodiments of the present disclosure is listed.

[0007] Figure 4 A flowchart of another example method for generating combined code for training an artificial intelligence system according to some embodiments of the present disclosure is listed.

[0008] Figure 5 A flowchart of another example method for generating combined code for training an artificial intelligence system according to some embodiments of the present disclosure is listed. Detailed Description

[0009] The present disclosure includes systems and methods for training artificial intelligence systems for combinatorial code generation. The generation of synthetic programs can include generating combinations of code portions of computer code. In many cases, the combinatorial set will quickly include a very large number of possible combinations. Thus, the combinatorial set can be reduced to another combinatorial set, where the reduced set can have specific characteristics. The reduced combinatorial set can then be combined or recombined in various ways to generate a synthetic program or code file. In particular, the program can be used as training data for an artificial intelligence system, such as for training a large language model (LLM). The LLM can be an LLM that is trained to output other synthetic code, e.g., synthetic code generated as a conversion of code from one programming language to another programming language.

[0010] Figure 1 Examples of computing environments in accordance with aspects of the present disclosure are listed. Computing environment 100 includes an example of an environment for executing at least some of the computer code involved in performing the various methods described herein, such as synthetic program module 107. In addition to synthetic program module 107, computing environment 100 also includes, for example, computer 101, wide area network (WAN) 102, end user device (EUD) 103, remote server 104, public cloud 105, and private cloud 106. In this embodiment, computer 101 includes a set of processors 110 (including processing circuitry 120 and cache 121), communication fabric 111, volatile memory 112, persistent storage 113 (including operating system 122 and synthetic program module 107, as identified above), a set of peripheral devices 114 (including a set of user interface (UI) devices 123, storage device 124, and a set of Internet of Things (IoT) sensors 125), and network module 115. Remote server 104 includes remote database 130. Public cloud 105 includes gateway 140, cloud orchestration module 141, a set of host physical machines 142, a set of virtual machines 143, and a set of containers 144.

[0011] Computer 101 can take the form of a desktop computer, laptop computer, tablet computer, smartphone, smartwatch, or other wearable computer, mainframe computer, quantum computer, or any other form of computer or mobile device now known or later developed that is capable of running programs, accessing a network, or querying a database, such as remote database 130. As is well known in the computer art, and depending on the technology, the execution of computer-implemented methods can be distributed among multiple computers and / or multiple locations. On the other hand, in this introduction to computing environment 100, the detailed discussion will focus on a single computer, specifically computer 101, to keep the introduction as simple as possible. Computer 101 can be located in the cloud, althoughFigure 1 It is not shown to be located in the cloud. On the other hand, computer 101 is not required to be located in the cloud, except within any scope that is explicitly indicated.

[0012] The processor set 110 includes one or more computer processors of any type that are currently known or will be developed in the future. The processing circuit 120 can be distributed across multiple packages, such as multiple coordinated integrated circuit chips. The processing circuit 120 can implement multiple processor threads and / or multiple processor cores. The cache 121 is a memory located within the processor chip package and is typically used for data or code that should be quickly accessible to the threads or cores running on the processor set 110. Cache memory is typically divided into multiple levels depending on its relative proximity to the processing circuit. Alternatively, some or all of the cache for the processor group can be located "off-chip". In some computing environments, the processor group 110 can be designed to process qubits and perform quantum computing.

[0013] Computer-readable program instructions are typically loaded onto the computer 101 to cause a series of operational steps to be performed by the processor set 110 of the computer 101 and thereby implement a computer-implemented method such that the instructions thus executed will instantiate the methods specified in the flowchart and / or narrative description of the computer-implemented methods contained in this document. These computer-readable program instructions are stored in various types of computer-readable storage media, such as the cache 121 and other storage media discussed below. The program instructions and associated data are accessed by the processor set 110 to control and direct the execution of the computer-implemented method. In the computing environment 100, at least some of the instructions for performing the computer-implemented method can be stored in the synthetic program module 107 in the persistent storage device 113.

[0014] The communication fabric 111 is a signal conduction path that allows the various components of the computer 101 to communicate with each other. Typically, this fabric is composed of switches and conductive paths, such as switches and conductive paths that make up a bus, a bridge, a physical input / output port, etc. Other types of signal communication paths can also be used, such as fiber optic communication paths and / or wireless communication paths.

[0015] The volatile memory 112 is any type of volatile memory that is currently known or will be developed in the future. For example, dynamic random access memory (RAM) or static RAM. Typically, the volatile memory 112 is characterized by random access, but this is not required unless explicitly stated. In the computer 101, the volatile memory 112 is located within a single package and is located inside the computer 101, but alternatively or in addition, the volatile memory can be distributed across multiple packages and / or located outside the computer 101.

[0016] The persistent storage device 113 is any form of non-volatile storage device for a computer, known now or developed in the future. The non-volatility of such a storage device means that the stored data remains unchanged whether or not power is supplied to the computer 101 and / or directly to the persistent storage device 113. The persistent storage device 113 can be a read-only memory (ROM), but typically at least a portion of the persistent device allows data to be written, deleted, and rewritten. Some familiar forms of persistent storage devices include magnetic disks and solid-state storage devices. The operating system 122 can take various forms, such as various known proprietary operating systems or open-source portable operating system interface type operating systems that employ a kernel. The code included in the synthetic program module 107 typically includes at least some of the computer code involved in performing the computer-implemented methods described herein.

[0017] The peripheral device set 114 includes the peripheral device set of the computer 101. Data communication connections between the peripheral devices and other components of the computer 101 can be achieved in various ways, such as via a Bluetooth connection, a near-field communication (NFC) connection, a connection established via a cable (such as a universal serial bus (USB)-type cable), a plug-in connection (e.g., a Secure Digital (SD) card), a connection established via a local communication network, and even a connection established via a wide area network (such as the Internet). In various embodiments, the UI device group 123 can include components such as a display screen, speakers, a microphone, wearable devices (such as goggles and smartwatches), a keyboard, a mouse, a printer, a touchpad, a game controller, and a haptic device. The storage device 124 is an external storage device, such as an external hard drive, or a plug-in storage device, such as an SD card. The storage device 124 can be persistent and / or volatile. In some embodiments, the storage device 124 can take the form of a quantum computing storage device for storing data in the form of qubits. In embodiments where the computer 101 needs to have a large amount of storage space (e.g., the computer 101 locally stores and manages a large database), such a storage device can be provided by a peripheral storage device designed to store an extremely large amount of data, such as a storage area network (SAN) shared by multiple computers distributed in different geographical locations. The IoT sensor set 125 consists of sensors that can be used for Internet of Things applications. For example, one sensor can be a thermometer, and another sensor can be a motion detector.

[0018] The network module 115 is a collection of computer software, hardware, and firmware that allows the computer 101 to communicate with other computers via the WAN 102. The network module 115 can include hardware such as a modem or a Wi-Fi signal transceiver, software for packetizing and / or depacketizing data for communication network transmission, and / or web browser software for communicating data over the Internet. In some embodiments, the network control function and the network forwarding function of the network module 115 are executed on the same physical hardware device. In other embodiments (e.g., embodiments utilizing software-defined networking (SDN)), the control function and the forwarding function of the network module 115 are executed on physically separate devices such that the control function manages multiple different network hardware devices. Computer-readable program instructions for performing computer-implemented methods can generally be downloaded to the computer 101 from an external computer or an external storage device via a network adapter card or network interface included in the network module 115.

[0019] The WAN 102 is any wide area network (e.g., the Internet) that is capable of communicating computer data over non-local distances via any technology for computer data communication known now or developed in the future. In some embodiments, the WAN 102 can be replaced and / or supplemented by a local area network (LAN) that is designed to communicate data between devices located within a local area (such as a Wi-Fi network). The WAN and / or LAN generally includes computer hardware such as copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and edge servers.

[0020] The end-user device (EUD) 103 is any computer system used and controlled by an end user (e.g., a customer of an enterprise operating the computer 101) and can take any form discussed above in relation to the computer 101. The EUD 103 typically receives helpful and useful data from the operation of the computer 101. For example, in the hypothetical case where the computer 101 is designed to provide advice to an end user, the advice typically travels from the network module 115 of the computer 101 via the WAN 102 to the EUD 103. In this way, the EUD 103 can display or otherwise present the advice to the end user. In some embodiments, the EUD 103 can be a client device such as a thin client, a thick client, a mainframe computer, a desktop computer, etc.

[0021] The remote server 104 is any computer system that provides at least some data and / or functionality to the computer 101. The remote server 104 can be controlled and used by the same entity that operates the computer 101. The remote server 104 represents a machine that collects and stores helpful and useful data for use by other computers, such as the computer 101. For example, in the hypothetical case where the computer 101 is designed and programmed to provide recommendations based on historical data, then the historical data can be provided to the computer 101 from the remote database 130 of the remote server 104.

[0022] The public cloud 105 is any computer system that is available for use by multiple entities, which provides on-demand availability of computer system resources and / or other computer functionality, particularly data storage (cloud storage) and computing power, without the user having to directly perform active management. Cloud computing typically utilizes resource sharing to achieve consistency and economies of scale. The direct and active management of the computing resources of the public cloud 105 is performed by the computer hardware and / or software of the cloud coordination module 141. The computing resources provided by the public cloud 105 are typically implemented by a virtual computing environment that runs on various computers that make up the set of host physical machines 142, which is the complete set of physical computers that are in and / or available to the public cloud 105. The virtual computing environment (VCE) typically takes the form of virtual machines from the set of virtual machines 143 and / or containers from the set of containers 144. It can be understood that these VCEs can be stored as images and can be transferred among and between various physical machine hosts either as images or after instantiation of the VCE. The cloud coordination module 141 manages the transfer and storage of the images, deploys new instances of the VCE, and manages the active instances of the VCE deployment. The gateway 140 is a collection of computer software, hardware, and firmware that allows the public cloud 105 to communicate over the WAN 102.

[0023] Some further explanations of the virtualized computing environment (VCE) will now be provided. The VCE can be stored as an "image". New active instances of the VCE can be instantiated from the image. Two familiar types of VCEs are virtual machines and containers. A container is a VCE that uses operating system-level virtualization. This refers to an operating system feature where the kernel allows for the existence of multiple isolated user space instances (called containers). From the perspective of the programs running within them, these isolated user space instances generally appear as real computers. A computer program running on a normal operating system can use all the resources of that computer, such as connected devices, files and folders, network shares, CPU power, and quantifiable hardware capabilities. However, a program running within a container can only use the contents of the container and the devices allocated to the container, and this functionality is called containerization.

[0024] The private cloud 106 is similar to the public cloud 105, except that the computing resources are only available for a single enterprise. Although the private cloud 106 is described as communicating with the WAN 102, in other embodiments, the private cloud may be completely disconnected from the Internet and can only be accessed through a local / private network. A hybrid cloud is a combination of multiple clouds of different types (e.g., private cloud, community cloud, or public cloud type), typically implemented by different vendors separately. Each of the multiple clouds remains an independent and discrete entity, but the larger hybrid cloud architecture is combined through standardized or proprietary technologies that enable coordination, management, and / or data / application portability between the multiple constituent clouds. In this embodiment, the public cloud 105 and the private cloud 106 are both part of a larger hybrid cloud.

[0025] Figure 2 The flowchart of an example method for generating combined code for training an artificial intelligence system according to some embodiments of the present disclosure is listed. For example, Figure 2 The method of Figure 1 can be executed by the synthesis program module 107 of

[0026] Figure 2 The method includes using combinatorial reduction to combinatorially reduce 202 multiple code portions into a subset of code portion combinations that satisfy one or more constraints. In some embodiments, the synthesis program module 107 obtains code files for generating code combinations. For example, the code files can be part of a software suite or platform (e.g., an operating system). The code files can represent a complete program, such as a memory management program that is part of an operating system.

[0027] In some embodiments, a code file is split into multiple code segments or code portions. The splitting can be done in a variety of ways. For example, a code file can be decomposed by individual functions, individual code blocks, classes, methods, or the like. As another example, certain characters or code bits can be used as delimiters (such as the brace characters "{" or "}") to split the code file into code portions. The synthesis program module 107 can be configured to take a code file as input and break it down into various code portions. The synthesis program module 107 can also assign metadata to each code portion. For example, the code portion metadata can include a code portion identifier, a parent code file identifier, a functional description of the code portion, the code author, and so on.

[0028] As an example, a memory management code file can be divided into multiple code portions based on the individual functions performed by different parts of the memory management code file. For example, different code portions can be generated for the various parts of the memory management code file, such as the memory allocation function, the memory deallocation function to free up memory, the garbage collection function, and so on. As a more specific example, the memory management code file can be a COBOL code file.

[0029] In some embodiments, the resulting code portions are recombined into different combinations of code portions that can be designed into a new synthetic program. Each code portion or group of code portions can form a constituent part of the code portion combination, which can then become an executable program. The code portions can be combined in different ways. Generally speaking, it can be understood that a set of x elements can be combined in x! ways or the factorial of x. Additionally, if p elements are selected from a set of q elements, the number of combinations can be the Cartesian product, or p q (which can also be expressed as p^q, or p raised to the power of q). The reader will understand that as the number of elements increases, the factorial of the set of elements or even the Cartesian product will result in numbers growing rapidly. As an example, in the case of 8 elements or code portions, the possible number of combinations can be 8! or 40,320. For example, in the case of selecting 2 elements from a set of 8 elements, the Cartesian product is 256, and in the case of selecting only 2 elements from a set of 20 elements, the result of the Cartesian product is over a million possible combinations. Even though a large number of code portion combinations may be generated, using such a large number of combinations as synthetic programs for training an artificial intelligence system may be impractical. Additionally, if new code portions are added each time and combinations need to be regenerated, this can result in a process of constantly regenerating a large number of combinations that is both inefficient and cumbersome.

[0030] In some embodiments, the Cartesian product of the code portion combinations is reduced in one or more ways. Figure 2The method includes performing 203, an n-wise reduction of the number of combinations of a Cartesian product. In this embodiment, n corresponds to the subset of elements to be selected from a complete set of elements for inclusion in a combination of code portions. As an example, n can be equal to 2, resulting in a 2-wise or pair-wise reduction of the number of combinations of the Cartesian product. Purely as an example, consider a set of 8 elements, each set having a pair of elements. For example, there can be 8 sets of code portions, where each set includes 2 different code portions. As a more specific example, each code portion in a set can correspond to a different variation of a particular function or operation. For example, a set of code portions can correspond to two different functions that each perform memory allocation, such as memalloc1 and memalloc2. The two memory allocation functions can be derived from the same source code file, or from different source code files (e.g., the same or different COBOL source code files). In other words, the variations of the memory allocation functions can each form a component of a synthetic program, which can then be used as a memory management program.

[0031] Continuing with the above example, recall that for a set of 8 elements, each element set having two elements, the Cartesian product can be 256. In the case of pair-wise reduction, the following combination formula can be applied:

[0032]

[0033] In the formula shown above, n represents the total number of sets (in this example, 8 pairs, or a = 8), and b represents the number of elements selected for each combination (in this example, b = 2). Using these values, the result of the formula can be a subset of 28 combinations of code portions, thereby reducing a plurality of combinations of code portions (such as 256) to a subset of combinations of code portions (e.g., 28). Additionally, the synthetic program module 107 can also be configured to further reduce the subset of 28 combinations of code portions using other techniques (such as using binary decision diagrams). The reader will understand that adding additional constraints can result in a slightly larger set, but still smaller than the set given by the Cartesian product. In the absence of constraints, arriving at the set (e.g., 28) can be considered an np-hard problem, while in the presence of specific constraints, the resulting set may be the result of an np-complete process.

[0034] Figure 2The method further includes generating one or more synthetic programs 204 using a subset of the combined code portions. Generating one or more synthetic programs using a subset of the combined code portions may include generating a complete synthetic program or an executable code file using code portion combinations from the subset of the combined code portions generated in step 202. For example, the complete synthetic program may be a recombination of code portions that produces another memory management program that includes different code portions compared to the originally used memory management program that was previously decomposed into code portions. For example, different versions of memory allocation functions (such as memalloc1 and memalloc2) may be used as part of different combinations of code portions, thereby producing different recombined synthetic programs.

[0035] Figure 2 The method further includes training 206 an artificial intelligence system using the synthetic programs. Training the artificial intelligence system using the synthetic programs may include using the synthetic programs as a training data set to train an artificial intelligence model, a machine learning model, a deep learning model, or a generative artificial intelligence model, such as a large language model (LLM). The synthetic program module 107 may be configured to train the artificial intelligence system in different ways. For example, training the artificial intelligence system may include providing a set of training data sets to the artificial intelligence system and causing the artificial intelligence system to make decisions or produce outputs based on the training data sets. In an initial stage of training, data with metadata tags or labels may be provided to the artificial intelligence system, which helps the artificial intelligence system understand the data types and produce outputs. The reader will understand that an example artificial intelligence system may be used to obtain code written in one programming language and generate similar code in a second programming language. In such an example, initially, the synthetic program module 107 may provide a training data set of combined code portions that are tagged in a specific manner, such as by the programming language used to generate the code in the combined code portions. Then, the synthetic program module 107 may prompt the artificial intelligence system to generate another program based on the training data set, but provide an identifier different from the programming language used for the training data set in the prompt or through other metadata. In later stages of training, the tags (such as the identification of the programming language) may be removed.

[0036] Figure 3 FIG. lists a flowchart of another example method for combined code generation for training an artificial intelligence system according to some embodiments of the present disclosure. Figure 3 The method of Figure 2 is similar to the method of Figure 3 in that the method of also includes reducing 202 a plurality of combined code portions to a subset of combined code portions that meet one or more constraints using combined reduction, generating 204 one or more synthetic programs using the subset of combined code portions, and training 206 an artificial intelligence system using the synthetic programs.

[0037] Figure 3 The method of Figure 2 differs from the method of Figure 3 in that the method of also includes identifying one or more combinations of code portions that compile successfully at 302. Identifying one or more combinations of code portions that compile successfully at 302 may include using a compiler to process a synthetic program formed by the combination of code portions and determining whether the program compiles successfully. For example, as previously described, the code portions may be COBOL code portions, and thus the synthetic program generated from the code portions may also be a COBOL program. Since COBOL is a compiled language, a COBOL compiler can be used to compile the program, and the synthetic program module 107 may be configured to determine whether the program compiles successfully. In some embodiments, the successfully compiled program can be used for LLM model training, while other programs created using other combinations of code portions may not be usable for LLM training. Additionally, only a small subset of samples within the combination of code portions may be compiled. Based on the compilation results of this small subset of samples, the synthetic program module 107 can decide whether to compile the remaining combinations within the subset.

[0038] Figure 3 The method of also includes identifying one or more combinations of code portions that produce one or more runnable programs at 304. The reader will understand that certain code may not produce errors during compilation but may produce errors during execution. Therefore, the synthetic program module 107 may be configured to execute the synthetic program generated by one or more of the combinations of code portions and check for successful execution before using the synthetic program as part of the training dataset for the LLM. For example, an integrated development environment (IDE) or other execution environment can be used to execute the combination.

[0039] Figure 3 The method of also includes identifying one or more combinations of code portions that produce one or more expected results during execution at 306. The reader will understand that certain code may execute successfully but produce unexpected results. Therefore, the synthetic program module 107 can be configured to execute one or more programs generated using the combination of code portions and check whether the resulting execution results are consistent with the expected results of the program.

[0040] Figure 4 Lists a flowchart of another example method for generating combined code for training an artificial intelligence system according to some embodiments of the present disclosure. Figure 4 The method of Figure 2 is similar to the method of Figure 4The method also includes using combinatorial reduction to combinatorially reduce a plurality of code portions 202 into a subset of combinations of code portions that satisfy one or more constraints, generating 204 one or more synthetic programs using the subset of combinations of code portions, and training 206 an artificial intelligence system using the synthetic programs.

[0041] Figure 4 The method of Figure 2 differs from the method of Figure 4 in that the method of

[0042] also includes selecting 402 combinations of code portions for the subset such that each code portion is included in at least one combination of code portions in the subset. In some embodiments, the synthetic program module 107 may be configured to include combinations of code portions in the subset such that each code portion is part of at least one combination of code portions so that all code portions are used. The reader will understand that a code portion may be part of more than one combination of code portions. To ensure coverage, the synthetic program module 107 may be configured to check whether each code portion is part of at least one combination of code portions. For example, a code portion may have an associated identifier. The synthetic program module 107 may be configured to store a record of the identifiers of each code portion (e.g., in some data structure like a table) that is included in a combination of code portions as part of the combinatorial operation of generating the combination of code portions. As an example, the synthetic program module 107 may ensure that each n-wise configuration (e.g., a 2-wise configuration for pairwise reduction, or a 3-wise configuration for 3-wise reduction) is included in at least one combination of code portions.

[0043] Figure 4The method also includes combinations of code portions having a specific set of code portions in the 404 subset. Readers will understand that certain code portions may constitute important components of synthetic programs generated for certain purposes. For example, a specific memory allocation function may be an important component of the memory management program being generated. Thus, the synthetic program module 107 can be configured to generate combinations of code portions that produce a synthetic program containing a specific memory allocation function. Additionally, in some cases, the synthetic program module 107 can be configured to use a combination engine to generate combinations of code portions. In such a case, the synthetic program module 107 can provide the combination engine with a set of code portions and certain constraints, such as the constraint that certain code portions will be included in any combination of code portions generated by the combination engine.

[0044] Figure 5 FIG. lists a flowchart of another example method for generating combined code for training an artificial intelligence system according to some embodiments of the present disclosure. Figure 5 The method of Figure 2 is similar to the method of Figure 5 in that the method of

[0045] Figure 5 also includes using combination reduction to reduce a combination of multiple code portions 202 to a subset of combinations of code portions that satisfy one or more constraints, generating 204 one or more synthetic programs using the subset of combinations of code portions, and training 206 an artificial intelligence system using the synthetic programs. Figure 2 The method of Figure 5 differs from the method of

[0046] Figure 5The method also includes training the artificial intelligence system using a subset of the code part combinations from the set of code parts while excluding one or more other code part combinations from the training. As described above, a large number of code part combinations can be created from the set of code parts (e.g., the Cartesian product of a set of 8 pairs of code parts can be 256 code part combinations). However, in some embodiments, only a subset of the code part combinations resulting from a combinatorial reduction of the total set of code part combinations is used for artificial intelligence training. For example, only a subset of the code part combinations resulting from an n-wise (e.g., pairwise) combinatorial reduction of the total set of code part combinations is used for artificial intelligence training.

[0047] Aspects of the present disclosure are described by narrative text, flowcharts, computer system block diagrams, and / or machine logic block diagrams included in computer program product (CPP) embodiments. For any flowchart, depending on the technology involved, operations may be performed in an order different from that shown in a given flowchart. For example, again depending on the technology involved, two operations shown in consecutive flowchart blocks may be performed in reverse order, as a single integrated step, simultaneously, or in a manner that at least partially overlaps in time.

[0048] The term "Computer Program Product Example" ("CPP Example" or "CPP") as used in this disclosure describes any collection of one or more storage media (also referred to as "media") that are substantially included in a collection of one or more storage devices that collectively include machine-readable code corresponding to instructions and / or data for performing the computer operations specified in a given CPP claim. A "storage device" refers to any tangible device that can retain and store instructions for use by a computer processor. Without limitation, computer-readable storage media can be electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, mechanical storage media, or any suitable combination of the foregoing. Some known types of storage devices that include these media include: floppy disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or pits / lands formed on the major surfaces of optical discs, or any suitable combination of the foregoing. As used in this disclosure, the term computer-readable storage media should not be construed to store in the form of transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide, optical pulses propagating through an optical fiber cable, electrical signals communicated through a wire, and / or other transmission media. Those skilled in the art will understand that during the normal operation of a storage device, such as during access, defragmentation, or garbage collection, data typically moves at certain incidental points in time, but this does not make the storage device a transitory device because the data is not transitory when stored.

[0049] The description of the various embodiments of this disclosure is presented for illustrative purposes, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are chosen to best explain the principles of the embodiments, the practical application, or technical improvements over technologies found in the marketplace, or to enable those of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for combinatorial code generation for training an artificial intelligence system, comprising: Combinatorially reducing the plurality of code portions to a combinatorial subset of code portions that satisfy one or more constraints using combinatorial reduction; generating one or more composite programs using the combined subsets of code portions; as well as An artificial intelligence system is trained using the synthetic program.

2. The method according to claim 1, wherein: Combinatorially reducing the plurality of code portions to the combined subset of code portions using the combinatorial reduction further comprises: An n-wise reduction of the plurality of code portion combinations is performed.

3. The method according to claim 1, wherein: Combinatorially reducing the plurality of code portions to the combined subset of code portions using the combinatorial reduction further comprises: Identify one or more code section combinations that compile successfully.

4. The method according to claim 1, wherein: Reducing the plurality of code portion combinations to the code portion combination subset further comprises: One or more combinations of code portions that produce one or more executable programs are identified.

5. The method according to claim 1, wherein: Reducing the plurality of code portion combinations to the code portion combination subset further comprises: One or more combinations of code sections that, when executed, produce one or more expected results are identified.

6. The method according to claim 1, wherein: The plurality of code portion combinations are generated using a plurality of code portions, and wherein reducing the plurality of code portion combinations to the subset further comprises: Code portion combinations are selected for the subset, whereby each code portion is included in at least one code portion combination in the subset.

7. The method according to claim 1, wherein: Reducing the plurality of code portion combinations to the code portion combination subset further comprises: A code portion combination having a specific set of code portions is included in the subset.

8. The method according to claim 1, wherein: Training the artificial intelligence system includes training a large language model used by the artificial intelligence system.

9. The method according to claim 1, wherein: Training the artificial intelligence system includes training the artificial intelligence system using the code portion combinations in the code portion combination subset while excluding one or more other code portion combinations from the training.

10. A computer program product, comprising computer program instructions, which, when executed, perform the method according to any one of claims 1 to 9.

11. An apparatus comprising: Processing equipment; as well as A memory operatively coupled to the processing device, wherein the memory stores computer program instructions which, when executed, cause the processing device to perform the method of any one of claims 1-9.

Images