Training data expansion through program simplification

Abstract

Techniques are provided that relate to expanding one or more training datasets to train one or more AI models. For example, one or more embodiments described herein may have a system that may include a memory that may store computer-executable components. The system may also include a processor operatively coupled to the memory and capable of executing the computer-executable components stored in the memory. The computer-executable components may include a training expansion component that may generate an expanded training dataset for training an artificial intelligence model by extracting abbreviated source code samples from source code samples included in the training dataset.

Description

Technical Field

[0001] The present disclosure relates to augmenting training data for one or more artificial intelligence (“AI”) models, and more specifically, to augmenting one or more AI model training data sets by simplifying source code samples.

Background Art

[0002] AI models are utilized in a variety of source code understanding tasks such as defect detection, code summarization, code completion, bug fixing, naming of functions and variables, code recommendations, software development workflows, and / or the like. However, training an AI model for source code understanding tasks can face various reliability concerns including data duplication bias, quality of labeling, low robustness and / or generalization, overfitting of the model (e.g., due to limitations in the size of the data set), class imbalance (e.g., due to a discrepancy between the counts of samples belonging to different classes), and / or the like. For example, the quality of the performance of an AI model can be affected by whether the model has learned task-related aspects of the source code.

[0003] For example, an AI model trained in a conventional manner can learn a separator between defective source code samples and sound source code samples using non-representative signals for a given task such as an unexpected correlation between code samples. Thereby, the AI model is trained to learn dataset nuances, which can result in a performance impairment when applied in settings outside of the theoretical context.

Summary of the Invention

[0004] The following outline provides a basic understanding of one or more embodiments of the present invention. This outline is not intended to identify any important or essential elements or to define any scope of any particular embodiment or any scope of any claim. Its sole purpose is to present the concepts in a simplified form as a prelude to the more detailed description presented later. One or more embodiments described herein describe a system, computer implementation method, apparatus and / or computer program product capable of generating one or more extended training datasets for training one or more artificial intelligence models.

[0005] According to the embodiment, a system is provided. The system may include memory capable of storing computer executable components. The system may also include a processor operably coupled to the memory and capable of executing the computer executable components stored in the memory. The computer executable components may include a training extension component that generates an extended training dataset for training an artificial intelligence model by extracting simplified source code samples from source code samples contained within the training dataset. An advantage of such a system may be the generation of additional training samples for training the artificial intelligence model.

[0006] In some examples, the system may also include a verification component capable of verifying the accuracy of the simplified source code samples. Furthermore, the system may include a labeler component capable of labeling the simplified source code samples via a task-specific labeling scheme. An advantage of such a system may be the implementation of quality control using newly generated training samples.

[0007] According to another embodiment, a system is provided. The system may include memory capable of storing computer executable components. The system may also include a processor operably coupled to the memory and capable of executing the computer executable components stored in the memory. The computer executable components may include a model training component capable of training an artificial intelligence model on an extended training dataset which includes simplified source code samples of source code samples from a training dataset.

[0008] In some examples, the system may include a simplification component that can execute a simplification algorithm to generate the simplified source code sample by reducing the source code sample. An advantage of such a system may be the generation of new training samples that may contain less noisy data.

[0009] According to the embodiment, a computer implementation method is provided. The computer implementation method may include a step in which a system operably coupled to a processor generates an augmented training dataset for training an artificial intelligence model by extracting simplified source code samples from source code samples contained within a training dataset. An advantage of such a computer implementation method may be improved performance by the artificial intelligence model by enabling the model to focus on task-relevant aspects of the training samples.

[0010] In some examples, a computer implementation method may include a step in which the system generates a plurality of simplified source code samples from the source code sample through a series of iterative reductions of the source code sample. The simplified source code samples may be derived from the plurality of simplified source code samples. An advantage of such a computer implementation method may be the generation of a plurality of simplified source code samples from a single parent source code sample.

[0011] According to another embodiment, a computer implementation method is provided. The computer implementation method may include a step in which a system operably coupled to a processor trains an artificial intelligence model on an augmented training dataset which includes simplified source code samples of source code samples from the training dataset.

[0012] In some examples, a computer implementation may include a step in which the system verifies the accuracy of a simplified source code sample. The computer implementation may also include a step in which the system labels the simplified source code sample via a task-based labeling scheme. Furthermore, the computer implementation may include a step in which, following verification and labeling, the system adds the simplified source code sample to a training dataset. An advantage of such a computer implementation may be the training of an artificial intelligence system on an expanded training dataset containing both the original and simplified source code samples.

[0013] According to one embodiment, a computer program product for training an artificial intelligence model is provided. The computer program product may include a computer-readable storage medium having embodied program instructions. The program instructions may be executable by a processor to cause the processor to generate an augmented training dataset for training the artificial intelligence model by extracting simplified source code samples from source code samples contained within the training dataset. An advantage of such a computer program product may be the reduction of overfitting and / or class imbalance during training of the artificial intelligence model.

[0014] In some examples, program instructions may further cause the processor to generate multiple simplified source code samples from a source code sample through a series of iterative reductions of the source code sample. A simplified source code sample may be derived from multiple simplified source code samples. Furthermore, iterations of the series of iterative reductions may generate new simplified source code samples that may be simpler than previous simplified source code samples generated by the series of iterative reductions. Additionally, program instructions may cause the processor to utilize a delta debugging algorithm to perform the series of iterative reductions. The advantage of such a computer program product may be the more extensible generation of simplified source code samples through a more efficient source code token reduction scheme. [Brief explanation of the drawing]

[0015] [Figure 1] A block diagram of an exemplary and non-limiting system that can facilitate the training of one or more AI models on one or more extended training datasets according to one or more embodiments described herein is shown.

[0016] [Figure 2] To extend one or more training datasets that may be used to train one or more AI models according to one or more embodiments described herein, a simplified source code sample may be verified and / or labeled. An exemplary and non-limiting system block diagram is provided.

[0017] [Figure 3] The following is a diagram of an exemplary and non-limiting table showing exemplary certification and / or labeling of a simplified source code sample that may be performed according to one or more embodiments described herein.

[0018] [Figure 4]A diagram of an exemplary and non-limiting system that can generate one or more extended training datasets for training one or more AI models in accordance with one or more embodiments described herein is shown.

[0019] [Figure 5] A diagram of an exemplary and non-limiting system that can train one or more AI models on one or more extended training datasets in accordance with one or more embodiments described herein is shown.

[0020] [Figure 6] A diagram of an exemplary and non-limiting training scheme that can be implemented for training one or more AI models on one or more extended training datasets in accordance with one or more embodiments described herein is shown.

[0021] [Figure 7] A diagram of an exemplary and non-limiting graph that can show the effectiveness of the extension of one or more training datasets in accordance with one or more embodiments described herein is shown. [Figure 8] A diagram of an exemplary and non-limiting graph that can show the effectiveness of the extension of one or more training datasets in accordance with one or more embodiments described herein is shown.

[0022] [Figure 9] A flowchart of an exemplary and non-limiting computer-implemented method that can facilitate the generation of one or more extended training datasets and / or the training of one or more AI models on one or more extended training datasets in accordance with one or more embodiments described herein is shown.

[0023] [Figure 10] A cloud computing environment is shown in accordance with one or more embodiments described herein.

[0024] [Figure 11] An abstraction model layer is shown according to one or more embodiments described herein.

[0025] [Figure 12] A block diagram of an exemplary and non-limiting operating environment in which one or more embodiments described herein may be facilitated is shown. [Modes for carrying out the invention]

[0026] The following detailed description is illustrative and not intended to limit the embodiments and / or the application or use of the embodiments. Furthermore, it is not intended to be limited by any express or implied information presented in the preceding background art or summary section of the invention or in the section on embodiments for carrying out the invention.

[0027] Herein, one or more embodiments are described with reference to the drawings, and throughout, similar reference numerals are used to refer to similar elements. In the following description, many specific details are included for illustrative purposes to provide a more complete understanding of one or more embodiments. However, it will be apparent that in various cases one or more embodiments can be carried out without these specific details.

[0028] Given the problems with other implementations of AI model training, this disclosure may be implemented to generate solutions to one or more of these problems by expanding the training dataset through one or more simplification processes to create a distribution of source code samples with varying code complexity. Advantageously, one or more embodiments described herein may utilize data-driven techniques to assist one or more AI models in focusing on task-relevant aspects of a given source code during training. For example, one or more embodiments described herein may incorporate code complexity into AI model learning. By utilizing data-driven techniques, one or more embodiments described herein may be applicable to task agnostic, programming language agnostic, and / or a wide variety of AI model types (including, for example, conventional machine learning models such as linear regression, support vector machines, decision trees, principal component analysis, and Naive Bayesian classifiers; neural network architectures such as convolutional neural networks ("CNNs"), recurrent neural networks ("RNNs"), graph neural networks ("GNNs"), combinations thereof, and / or similar).

[0029] Various embodiments of the present invention may relate to computer processing systems, computer implementations, apparatus, and / or computer program products that facilitate efficient, effective, and autonomous (e.g., without direct human guidance) AI model training. For example, one or more embodiments described herein may extend one or more training datasets by simplifying one or more source code samples. For example, simplified program samples may be generated by utilizing simplification algorithms such as delta debugging. Thereafter, the various embodiments described herein may extend one or more training datasets using smaller and potentially less noisy source code samples while maintaining the task profile of the training datasets.

[0030] Computer processing systems, computer implementation methods, devices, and / or computer program products utilize hardware and / or software to solve inherently highly technical problems (e.g., AI model training) that are not abstract and cannot be performed as a set of mental activities by humans. For example, one or more individuals cannot easily expand a training dataset through a program simplification scheme while maintaining the task profile of the training dataset. Furthermore, one or more embodiments described herein may constitute a technical improvement over conventional AI model training by expanding the training dataset to create diverse training samples with varying code complexity. Moreover, one or more embodiments described herein may have practical applications by improving AI model training through a data-driven method that is task-agnostic and applicable to a wide variety of AI model architectures. For example, the various embodiments described herein may enable task-related learning for one or more AI models by incorporating complexity awareness of the code. One or more embodiments described herein may control the training protocol of one or more AI models by expanding one or more training datasets to generate multiple simplified source code samples.

[0031] Figure 1 shows an exemplary and non-limiting block diagram of System 100, which can extend one or more training datasets for training one or more AI models. Repeated descriptions of similar elements used in other embodiments described herein are omitted for brevity. Embodiments of systems (e.g., System 100), apparatus, or processes in various embodiments of the present invention may constitute one or more machine-executable components, embodied in one or more machines, for example, in one or more computer-readable media associated with one or more machines. When executed by one or more machines (e.g., computers, computing devices, virtual machines, combinations thereof, and / or similar), such components can cause the machines to perform the operations described.

[0032] As shown in Figure 1, the system 100 may include one or more servers 102, one or more networks 104, and / or one or more input devices 106. The server 102 may include a training extension component 110. The training extension component 110 may further include a communication component 112 and / or a simplification component 114. The server 102 may also include or be associated with at least one memory 116. The server 102 may further include a system bus 118 that can be coupled to various components such as, for example, the training extension component 110 and associated components, the memory 116, and / or a processor 120, etc. Although the server 102 is shown in Figure 1, in other embodiments, various types of multiple devices may be associated with or include the features shown in Figure 1. Furthermore, the server 102 may communicate with one or more cloud computing environments.

[0033] One or more networks 104 may include, but are not limited to, wired and wireless networks, including cellular networks, wide area networks (WANs) (e.g., the Internet), or local area networks (LANs). For example, server 102 may communicate with one or more input devices 106 using virtually any desired wired or wireless technology, including, but not limited to, cellular, WAN, Wireless Fidelity (Wi-Fi®), WiMAX®, WLAN, Bluetooth® technology, combinations thereof, and / or similar (and vice versa). Furthermore, while the training extension component 110 may be provided on one or more servers 102 in the shown embodiments, it should be understood that the architecture of system 100 is not so limited. For example, training extension component 110, or one or more components of training extension component 110, may reside on another computer device, such as another server device, client device, and / or similar.

[0034] One or more input devices 106 may include, but are not limited to, one or more computerized devices such as: a personal computer, a desktop computer, a laptop computer, a mobile phone (e.g., a smartphone), a computerized tablet (e.g., including a processor), a smartwatch, a keyboard, a touchscreen, a mouse, a combination thereof, and / or similar. One or more input devices 106 may be used to input one or more training datasets 122 and / or AI models 124 into the system 100, thereby sharing the above data with the server 102 (e.g., via a direct connection and / or via one or more networks 104). For example, one or more input devices 106 may transmit data to a communication component 112 (e.g., via a direct connection and / or via one or more networks 104). In addition, one or more input devices 106 may include one or more displays that can present one or more outputs generated by the system 100 to the user. For example, one or more displays may include, but are not limited to, a cathode ray tube display ("CRT"), a light-emitting diode display ("LED"), an electron-emitting diode display ("ELD"), a plasma display panel ("PDP"), a liquid crystal display ("LCD"), an organic light-emitting diode display ("OLED"), a combination thereof, and / or similar.

[0035] In various embodiments, one or more input devices 106 and / or one or more networks 104 can be used to input one or more settings and / or commands to the system 100. For example, in various embodiments described herein, one or more input devices 106 can be used to operate and / or control the server 102 and / or related components. In addition, one or more input devices 106 can be used to display one or more outputs (e.g., displays, data, visualizations, and / or similar) generated by the server 102 and / or related components. Furthermore, in one or more embodiments, one or more input devices 106 can be provided within a cloud computing environment and / or operably coupled to said cloud computing environment.

[0036] For example, in one or more embodiments, one or more input devices 106 may be used to load one or more training datasets 122 (e.g., including one or more source code samples 123) into the system 100, which can be stored in one or more memories 116. Furthermore, the training extension component 110 may utilize one or more training datasets to train one or more AI models 124 (e.g., one or more source code comprehension tasks). For example, one or more training datasets 122 may include one or more source code samples 123. In one or more embodiments, one or more input devices 106 may also be used to load one or more AI models 124 into the system 100, which can be stored in one or more memories 116 and / or trained by the training extension component 110. Furthermore, in various embodiments, one or more input devices 106 may be used to define one or more settings and / or parameters for training the AI ​​models 124. In various embodiments, the communication component 112 may share one or more inputs (e.g., entering the system 100 via one or more input devices 106) with the relevant components of the training extension component 110, according to one or more embodiments described herein.

[0037] In various embodiments, one or more AI models 124 can be used to facilitate one or more machine learning tasks. In one or more embodiments, the AI ​​model 124 can simulate several interconnected processing units that may be analogous to abstract versions of neurons. For example, the processing units may be arranged in multiple layers (e.g., one or more input layers, one or more hidden layers, and / or one or more output layers) connected by fluctuating connection strengths (e.g., which may be commonly referred to as “weights” in the art). The AI ​​model 124 can learn through training, where data with known outcomes is input to the computer model, the output on the data is compared to the known outcomes, and / or the weights of the computer model are autonomously adjusted based on the comparison to replicate the known outcomes. As one or more AI models 124 are trained, the computer model can become increasingly accurate, and thus the trained AI model 124 can accurately analyze data with unknown outcomes based on lesson learning from the training data to facilitate one or more machine learning tasks.Examples of AI Model 124 include, but are not limited to, perceptrons ("P"), feedforward ("FF"), radial basis networks ("RBF"), deep feedforward ("DFF"), recurrent neural networks ("RNN"), long / short-term memory ("LSTM"), gated recurrent units ("GRU"), autoencoders ("AE"), variational AEs ("VAE"), denoising AEs ("DAE"), sparse AEs ("SAE"), Markov chains ("MC"), Hopfield networks ("HN"), Boltzmann machines ("BM"), and deep billy This may include F Networks ("DBN"), Deep Convolutional Networks ("DCN"), Deconvolutional Networks ("DN"), Deep Convolutional Inverse Graphics Networks ("DCIGN"), Generative Adversarial Networks ("GAN"), Liquid State Machines ("LSM"), Extreme Learning Machines ("ELM"), Echo State Networks ("ESN"), Deep Residual Networks ("DRN"), Kohonen Networks ("KN"), Support Vector Machines ("SVM"), and / or Neural Turing Machines ("NTM").

[0038] In one or more embodiments, the training extension component 110 may extend one or more training datasets 122 to facilitate the training of one or more AI models 124. For example, the training extension component 110 may include a simplification component 114 that can generate simplified source code samples 126 from one or more training datasets 122 by executing one or more simplification algorithms. As shown in Figure 1, in one or more embodiments, the simplified source code samples 126 generated by the simplification component 114 may be stored, for example, in one or more memories 116. In various embodiments, one or more simplified source code samples 126 may be smaller samples derived from parent source code samples 123 contained in one or more training datasets 122, thereby increasing the number of training samples used to train one or more AI models 124 from one or more training datasets 122.

[0039] For example, the simplification component 114 may simplify source code samples 123 contained within one or more training datasets 122 into a single simplified source code sample 126. In another example, the simplification component 114 may simplify a single source code sample 123 contained within one or more training datasets 122 into multiple simplified source code samples 126. In one or more embodiments, the simplification component 114 may simplify source code samples 123 contained within one or more training datasets 122 into multiple simplified source code samples 126 based on size, where the number of simplified source code samples 126 generated from source code sample 123 may increase as the size of source code sample 123 increases. For example, as the number of source code samples 123 from one or more training datasets 122 increases, the simplification component 114 may perform additional reduction iterations through one or more simplification algorithms to result in more simplified source code samples 126.

[0040] Furthermore, the simplification component 114 may simplify each or a portion of the source code samples 123 contained within one or more training datasets 122. For example, in one or more embodiments, the simplification component 114 may simplify all, or substantially all, of the source code samples 123 contained within one or more training datasets 122. In another example, in one or more embodiments, the simplification component 114 may simplify the source code samples 123 contained within one or more training datasets 122 that are larger than the code complexity or a defined threshold (for example, defined by a metric for the complexity of one or more codes).

[0041] In various embodiments, the simplification component 114 may utilize any simplification algorithm that can reduce the input to the source code sample 123 while generating a reduced sample (e.g., a simplified source code sample 126) that retains task-relevant features of the source code comprehension task to be performed by one or more AI models 124. Exemplary simplification algorithms may include, but are not limited to, algorithms that can drop tokens and / or statements from the source code sample 123 in a naive manner, a randomized manner, a brute-force manner, or via an extensible reduction scheme (e.g., a delta debugging algorithm).

[0042] For example, the simplification component 114 may use a delta debugging algorithm to generate one or more simplified source code samples 126; however, the architecture of the simplification component 114 is not limited to the use of a delta debugging algorithm. The simplification component 114 may use a simplification algorithm (e.g., a delta debugging algorithm) to sequentially simplify a given source code sample 123 from one or more training datasets 122 while maintaining the task profile established by the given source code sample 123 (e.g., maintaining the vulnerability profile of healthy or defective code in a vulnerability detection setting). For example, the simplification component 114 may use one or more simplification algorithms to iteratively remove source code tokens from the source code sample 123 targeted for simplification, where each iteration of source code token removal may generate its own simplified source code sample 126. In various embodiments, the simplification component 114 may remove tokens based on, for example: smart reduction similar to binary search (e.g., via a delta debugging algorithm), linear methods (e.g., removing tokens one at a time over multiple iterations), randomization methods (e.g., where tokens are randomly selected from source code samples 123 for removal), multi-token level operations, statement level operations, multi-statement level operations, combinations thereof, and / or similar. At a minimum, since the simplified source code sample 126 is a reduction of source code sample 123 from training dataset 122, the simplified source code sample 126 may contain less noisy data than one or more training datasets 122.

[0043] Figure 2 shows an exemplary and non-limiting diagram of a system 100 further including a verification component 202 and / or a labeler component 204 according to one or more embodiments described herein. Repeated descriptions of similar elements used in other embodiments described herein are omitted for brevity. In various embodiments, a simplified source code sample 126 is generated by a simplification component 114, which can be analyzed by the verification component 202 and / or the labeler component 204, to identify a suitable candidate for augmenting one or more training datasets 122.

[0044] In one or more embodiments, the verification component 202 may determine whether the simplified source code sample 126 is correct (e.g., compileable and / or executable). For example, the verification component 202 may determine whether the simplified source code sample 126 can be compiled and / or executed. For example, the simplification component 114 may generate a simplified source code sample 126 regardless of whether reduction (e.g., source code token reduction) would cause the resulting simplified source code sample 126 to be incorrect (e.g., uncompileable and / or unexecutable). The verification component 202 may identify a simplified source code sample 126 that remains correct after reduction performed by the simplification component 114. For example, the validation component 202 could be a compiler, where a valid simplified source code sample 126 could be compiled (e.g., even if the compiled simplified source code sample 126 operates in a different manner than the parent source code sample 123 from which it was derived), and an invalid simplified source code sample 126 could not be compiled. In various embodiments, the training extension component 110 could discard an invalid simplified source code sample 126, so that as a result, an invalid simplified source code sample 126 is not present in the extension of one or more training datasets 122. In another example, the validation component 202 could be adjusted (e.g., via one or more settings defined by one or more input devices 106) to ensure that the main aspects of the parent source code sample 123 (e.g., API calls, code variables, and / or other constructions of the same) are preserved.

[0045] In one or more embodiments, a valid simplified source code sample 126 may further be analyzed by a labeler component 204 to determine whether the valid simplified source code sample 126 has the same task profile as the one or more parent source code samples 123 from which the valid simplified source code sample 126 was derived. In one or more embodiments, the labeler component 204 may include: labeler computer logic included in one or more training datasets 122; line-based code feature matching (e.g., comparing the simplified source code sample 126 with the parent source code samples 123 from one or more training datasets 122); a static analyzer, a fuzz test algorithm, a content domain expert; a combination thereof, and / or similar. Furthermore, the labeler component 204 may label valid simplified source code samples 126 that exhibit the same task profile as the parent source code samples 123. For example, a simplified source code sample 126 may exhibit the same task profile as the parent source code sample 123, where the generation of the simplified source code sample 126 avoids the generation of one or more new defects. For example, a simplified source code sample 126 may exhibit the same task profile as the parent source code sample 123, where the simplified source code sample 126 is: free from defects; or contains one or more defects or other task-related features, but the one or more defects or other task-related features are also present in the parent source code sample 123. Simplified source code samples 126 that exhibit a different task profile from the parent source code sample 123 from which they originate may contain one or more defects that are not present in the parent source code sample 123. In one or more embodiments, the labeler component 204 can store (e.g., and / or label) those simplified source code samples 126 that exhibit the same task profile as the parent source code sample 123.In one or more embodiments, the labeler component 204 can store (for example, and / or label) those simplified source code samples 126 that exhibit a different task profile from the parent source code sample 123.

[0046] In various embodiments, the labeling performed by the labeler component 204 may be based on a machine learning task to be performed by one or more AI models 124. For example, the labeler component 204 may label a simplified source code sample 126 (e.g., one that is valid and / or has been determined to maintain the task profile of its parent source code sample 123) according to one or more labeling schemes defined by the task of one or more AI models 124 (e.g., according to a task-specific oracle code). For example, with respect to a vulnerability task, the labeler component 204 may label a simplified source code sample 126 containing a defined vulnerability with "1" and a simplified source code sample 126 lacking the defined vulnerability with "0". In one or more embodiments, the label parameters utilized by the labeler component 204 may be customized (e.g., via one or more input devices 106) based on the task to be performed by one or more AI models 124. In addition, in one or more embodiments, the labeler component 204 may determine one or more confidence values ​​associated with the generated labels assigned to the simplified source code sample 126.

[0047] For example, the labeler component 204 may determine whether the simplified source code sample 126 contains the same defects, or contains other features of the one or more source code samples 123 from which the simplified source code sample 126 was derived that are relevant to a given source code comprehension task; whether it is free from defects; or whether it contains new defects (e.g., compared to the one or more source code samples 123 from which the simplified source code sample 126 was derived). If the simplified source code sample 126 is free from defects, the labeler component 204 may determine that the simplified source code sample 126 maintains the task profile and may label the simplified source code sample 126 according to a task-based labeling scheme. If the simplified source code sample 126 contains one or more defects, and those defects are also present in the source code sample 123 from which the simplified source code sample 126 originates, the labeler component 204 may determine that the simplified source code sample 126 maintains a task profile and may label the simplified source code sample 126 according to a task-based labeling scheme. If the simplified source code sample 126 contains one or more defects that are not present in the source code sample from which the simplified source code sample 126 originates, the labeler component 204 may determine that the simplified source code sample 126 has a different task profile from the parent source code sample 123, which may make the simplified source code sample 126 unsuitable for extending one or more training datasets 122. This ensures that the labeler component 204 can avoid introducing new defects (e.g., compared to the training dataset 122) into the training of the AI ​​model 124 by the training extension component 110, which is performed by the extension of one or more training datasets 122.

[0048] Figure 3 shows an exemplary and non-exclusive diagram of Table 300, which may include an exemplary source code sample 123 from one or more training datasets 122, and / or a simplified source code sample 126 that can be analyzed by the verification component 202 and / or labeler component 204, according to one or more embodiments described herein. Repeated descriptions of similar elements used in other embodiments described herein are omitted for brevity. Source code sample #1 may be an exemplary source code sample from one or more training datasets 122 for training one or more AI models 124 in a vulnerability detection task. Simplification, authentication, and / or labeling performed by the training extension component 110 according to various embodiments described herein are not limited to source code sample 123 or a particular type of source code understanding task.

[0049] As shown in Figure 3, exemplary source code samples #2 to #22 may be exemplary simplified source code samples 126 generated by the simplification component 114. For example, the simplification component 114 may reduce exemplary source code sample #1 by half to generate exemplary source code samples #2 to #3. In addition, the simplification component 114 may remove one or more tokens from exemplary source code sample #1 to generate exemplary simplified source code samples #3 to #22. For example, the simplification component 114 may iteratively reduce exemplary source code sample #1 at a finer granularity to achieve token-level reduction.

[0050] Verification component 202 may attempt to compile exemplary source code samples #1 through #22 (including, for example, the simplified source code sample 126). If the simplified source code sample 126 can be compiled, verification component 202 may determine that source code sample 126 is valid (for example, as indicated by the checkmark in Table 300). If the simplified source code sample 126 cannot be compiled, verification component 202 may determine that source code sample 126 is invalid (for example, as indicated by "X" in Table 300). As shown in Figure 3, verification component 202 may determine that exemplary source code samples #13 and #22 are valid simplified source code samples 126, while exemplary source code samples #2 through #5 and #21 are invalid simplified source code samples 126.

[0051] Furthermore, the labeler component 204 may determine, according to various embodiments described herein, whether exemplary source code samples #2 to #22 (e.g., simplified source code sample 126) can exhibit the same task profile as exemplary source code sample #1 (e.g., parent source code sample 123). For example, the labeler component 204 may determine that valid simplified source code samples #13 and #22 maintain the task profile of source code sample #1 because, at the very least, exemplary simplified source code samples #13 and #22 lack defects and / or contain defects that are also present in exemplary source code sample #1. Furthermore, the labeler component 204 may label exemplary source code samples #13 and #22 based on whether simplified source code sample 126 contains one or more defined vulnerabilities.

[0052] Figure 4 shows an exemplary and non-limiting diagram of System 100 further including a dataset component 402 according to one or more embodiments described herein. Repeated descriptions of similar elements used in other embodiments described herein are omitted for brevity. In various embodiments, the dataset component 402 may generate one or more extended training datasets 404 by utilizing a simplified source code sample 126 which has been determined to be valid and / or preferred by the validation component 202 and / or the labeler component 204.

[0053] For example, the extended training dataset 404 may include: all or substantially all simplified source code samples 126 that are deemed valid in combination with all or substantially all source code samples 123 of one or more training datasets 122, and / or labeled; all or substantially all simplified source code samples 126 that are deemed valid in combination with a portion of source code samples 123 of one or more training datasets 122, and / or labeled; a portion of a simplified source code sample 126 that is deemed valid in combination with all or substantially all source code samples 123 of one or more training datasets 122, and / or labeled; a portion of a simplified source code sample 126 that is deemed valid in combination with a portion of source code samples 123 of one or more training datasets 122, and / or labeled; and / or simply a simplified source code sample 126 that is deemed valid and / or labeled.

[0054] For example, the dataset component 402 can reduce overfitting while training one or more AI models 124 by combining the source code samples 123 of the training dataset 122 with simplified source code samples 126 that have been determined to be valid (e.g., via the validation component 202 and / or the labeler component 204) and / or labeled to generate one or more augmented training datasets 404. For example, adding labeled and / or validated simplified source code samples 126 to one or more training datasets 122 can generate one or more augmented training datasets 404 that contain more training samples than those originally provided by the training dataset 122. The amount of additional training samples can reduce the percentage of noise data included in the training of one or more AI models 124 and / or facilitate the learning of task-related aspects of the training dataset 122 for one or more AI models 124.

[0055] In another example, the dataset component 402 may reduce class imbalance during training of one or more AI models 124 by generating one or more augmented training datasets 404 by adding a small number of classified simplified source code samples 126 (e.g., those determined to be valid and / or preferred) to one or more training datasets 122. For example, source code samples 123 included in one or more training datasets 122 may be classified into two or more classes, where the distribution of source code samples 123 may be class-imbalanced. To reduce this imbalance, the dataset component 402 may generate one or more augmented training datasets 404 by adding valid simplified source code samples 126 labeled to belong to sparse classes to one or more training datasets 122, where valid simplified source code samples 126 labeled to belong to other classes may be excluded from the one or more augmented training datasets 404.

[0056] Figure 5 shows an exemplary and non-limiting diagram of a system 100 further including a model training component 502 according to one or more embodiments described herein. Repeated descriptions of similar elements used in other embodiments described herein are omitted for brevity. In various embodiments, the model training component 502 may train one or more AI models 124 on one or more extended training datasets 404 and / or training datasets 122 for one or more source code comprehension tasks (defined, for example, via one or more input devices 106).

[0057] In one or more embodiments, one or more extended training datasets 404 may include source code samples 123 from one or more training datasets 122, and verified and / or labeled simplified source code samples 126; where the model training component 502 may randomly sample from one or more extended training datasets 404 to train one or more AI models 124. Alternatively, the model training component 502 may sample from the extended training datasets 404 based on one or more parameters of the source code samples 123 and / or simplified source code samples 126. For example, the model training component 502 may sample from one or more extended training datasets 404 based on noise data content, code complexity, size, class labels, confidence values, combinations thereof, and / or similar. For example, the model training component 502 samples one or more augmented training datasets 404, and as a result, the samples may be supplied to one or more AI models 124 in a progressive order from smallest to largest. In another example, the model training component 502 samples one or more augmented training datasets 404, and as a result, the samples may be supplied to one or more AI models 124 in a progressive order from minimum code complexity to maximum code complexity.

[0058] In one or more embodiments, one or more extended training datasets 404 may simply include verified and / or labeled simplified source code samples 126; where the model training component 502 may sample from one or more training datasets 122 in addition to randomly sampling from one or more extended training datasets 404. In one or more embodiments, one or more extended training datasets 404 may simply include verified and / or labeled simplified source code samples 126; where the model training component 502 may simply sample from one or more extended training datasets 404 (for example, randomly or in a progressive order).

[0059] Figure 6 shows an exemplary and non-limiting training scheme 600 that may be performed by the training extension component 110 according to one or more embodiments described herein. Repeated descriptions of similar elements used in other embodiments described herein are omitted for brevity. Training scheme 600 illustrates the operation of the training extension component 110 by showing a combination of features described herein; however, the architecture of the training extension component 110 is not limited in this way. For example, training scheme 600 shows the iterative reduction of source code sample 123; however, source code sample 123 may be reduced via a single reduction or via multiple reductions independent of each other according to one or more embodiments described herein. In another example, training scheme 600 shows the expansion of training dataset 122 by combining training dataset 122 and simplified source code sample 126; however, training dataset 122 and simplified source code sample 126 may also be sampled independently of each other according to one or more embodiments described herein.

[0060] In 602, the training extension component 110 may obtain one or more source code samples 123 from one or more training datasets 122 and serve them as the initial source code sample input 604 for one or more subsequent simplification, authentication, and / or labeling processes. In 606, the training extension component 110 may share the source code sample input 604 with the simplification component 114. According to various embodiments described herein, the simplification component 114 may perform one or more simplification algorithms to reduce the source code sample input 604 to a simplified source code sample 126. In 608, the training extension component 110 may share the simplified source code sample 126 with the verification component 202. According to various embodiments described herein, the verification component 202 may verify the accuracy of the simplified source code sample 126. For example, verification component 202 may attempt to compile simplified source code sample 126; where simplified source code sample 126 that can be compiled is valid, and simplified source code sample 126 that does not compile is invalid.

[0061] In 610, the simplified source code sample 126 is determined to be valid, and the training extension component 110 may share the valid simplified source code sample 126 with the labeler component 204. According to various embodiments described herein, the labeler component 204 may label the valid simplified source code sample 126 via one or more labeling schemes, based on the determination that the valid simplified source code sample 126 maintains the task profile of the source code sample input 604. In 612, the training extension component 110 may share the labeled and validated simplified source code sample 126 with the dataset component 402. Also in 614, the training extension component 110 may use a copy of the validated and labeled simplified source code sample 126 as the source code sample input 604 for the next reduction iteration and / or generation of the simplified source code sample 126. In 616, the model training component 502 may sample from one or more training datasets 122 and / or one or more labeled and / or valid simplified source code samples 126 in order to train one or more AI models 124.

[0062] To demonstrate the effectiveness of the various embodiments described herein, System 100 is used in relation to vulnerability detection use cases for multiple datasets; however, the various embodiments are applicable to other source code comprehension tasks. The exemplary training dataset 122 used to demonstrate the effectiveness of System 100 includes the s-bAbI synthetic dataset ("s-bAbI"), the Juliet test suite ("Juliet"), and the D2A dataset. While the s-bAbI, Juliet, and D2A datasets are used to demonstrate the effectiveness of the various embodiments described herein, the features of the training extension component 110 are not limited to the exemplary s-bAbI, Juliet, and D2A datasets; rather, other source code datasets can be analyzed. s-bAbI includes a syntactically valid C program with a non-trivial control flow focused on buffer overflow vulnerabilities. For the purpose of vulnerability detection tasks, the s-bAbI generator can be used to create a dataset where samples with the "UNSAFE" tag may be labeled with 1, and samples with the "SAFE" tag may be labeled with 0. The Juliet dataset contains synthetic examples with various vulnerability types designed to test the static analyzer. From the Juliet test cases, samples tagged as "bad" and with clear bug information from Juliet's manifest.xml may be labeled with 1, while samples tagged as "good" may be labeled with 0. The D2A dataset is a real-world vulnerability detection dataset built through multiple GitHub projects—OpenSSL, FFMpeg, HTTPD, Nginx, and libtiff. The D2A dataset contains detailed trace-level bug information derived using diff analysis on the output of the infer static analyzer of consecutive repository versions before and after bug fix commits.Function-level sample extraction from D2A traces may yield a training function labeled as 1 (e.g., from a pre-correction commit) and a training function labeled as 0 (e.g., from a post-correction commit). Furthermore, in one or more embodiments, a convolutional neural network ("CNN"), a recurrent neural network ("RNN"), and / or a graph neural network ("GNN") AI model 124 may be used to demonstrate the effectiveness of system 100.

[0063] Figures 7-8 show illustrative and non-limiting graphs 704, 802, and / or 804 that may demonstrate the effectiveness of the training augmentation component 110 according to one or more embodiments described herein. Repeated descriptions of similar elements used in other embodiments described herein are omitted for brevity. The training augmentation component 110 may generate an augmented training dataset 404 containing, as a factor of the training dataset 122, 9x samples of s-bAbI, 9.6x samples of Juliet, and 53x samples of D2A. Variations in the augmentation level may be due to differences in the size of the source code samples 123 included in the training dataset 122. For example, the larger the source code sample input 604, the more reduction iterations the simplification component 114 may perform; thereby potentially resulting in more effective and / or labeled simplified source code samples 126.

[0064] As shown in Figure 7, training one or more AI models 124 with more loaded, augmented training datasets 404 can enhance model performance. For example, Graph 704 relates to a GNN model 124 trained with random sampling from an augmented Juliet dataset. A signal-aware recall metric can be used to indicate improvements in the performance of the AI ​​model 124, which can be achieved via the training augmentation component 110. The signal-aware recall metric can be used to measure the signal awareness of an AI-for-code model. For example, in the context of a vulnerability detection task, the recall metric can measure the percentage of vulnerable samples that the model correctly predicts, while the signal-aware recall metric can measure how often in those cases the model captures the correct signals to arrive at a prediction. For example, in the context of vulnerability detection, the signal-aware recall metric can measure the percentage of vulnerable samples that the AI ​​model 124 correctly predicts (e.g., using task-relevant signals). The x-axis of Graph 704 shows the percentage (e.g., as a percentage of the base dataset size) of samples added to the training dataset 122, which is randomly selected from the generated set and has a left endpoint (e.g., x=0) that refers to baseline model performance. In addition, the training augmentation component 110 can improve the performance of the AI ​​model 124 with respect to other evaluation metrics such as accuracy, F1 score, mean accuracy (e.g., PR-AUC), combinations thereof, and / or similar. As shown in Figure 7, the gains in model performance can continue with more augmentation. By presenting the AI ​​model 124 with smaller samples while still containing characteristics relevant to the task at hand, the AI ​​model 124 can be trained to focus more on task-relevant aspects of the source code sample 123 and less on noise data or dataset nuances.

[0065] Graphs 802 and / or 804 illustrate that the performance of the enhanced AI model 124 achievable by the training augmentation component 110 can be attributed to more than simply increasing the sample population. For example, Figure 8 shows the signal recognition recall values ​​obtained with a general augmentation compared to the augmentation performed by the training augmentation component 110 for several representative augmentation levels of the s-bAbI dataset. As shown in Figure 8, the gain in model performance can continue with increasing augmentation, with signal recognition recall reaching its achievable maximum of nearly 96% at a 50% augmentation, representing a 113% improvement over simply training the AI ​​model 124 with the training dataset 122. As also shown in Figure 8, a general augmentation to increase the sample population may have little effect on the performance of the AI ​​model 124 (e.g., with respect to the signal recognition recall metric). In contrast, further sampling of effective and / or labeled simplified source code samples 126 by augmenting the training dataset 122 can significantly improve the performance of the AI ​​model 124. In addition, as shown in Graph 804, the improved performance can be achieved across a wide variety of AI model types (124 types).

[0066] Figure 9 shows a flowchart of an exemplary and non-limiting computer implementation method 900 that facilitates generating one or more extended training datasets 404 and / or training one or more AI models 124 on one or more extended training datasets 404 according to one or more embodiments described herein. Repeated descriptions of similar elements used in other embodiments described herein are omitted for brevity.

[0067] In 902, the computer implementation method 900 includes the step of receiving one or more training datasets 122, which may contain one or more source code samples 123, by a system 100 operably coupled to a processor 120 (for example, via a communication component 112). For example, one or more training datasets 122 may be fed into the system 100 via one or more input devices 106. In 904, the computer implementation method 900 may have one or more simplification algorithms executed by the system 100 (for example, via a simplification component 114) that can generate one or more simplified source code samples 126 by reducing one or more source code samples 123 of one or more training datasets 122. For example, one or more simplified source code samples 126 may be generated by extracting one or more portions from one or more source code samples 123 of one or more training datasets 122. In another example, one or more simplified source code samples 126 may be generated by removing one or more tokens from one or more source code samples 123 of one or more training datasets 122. In yet another example, one or more simplified source code samples 126 may be generated through one or more iterative reductions of source code samples 123 of one or more training datasets 122, according to the various embodiments described herein.

[0068] In 906, the computer implementation method 900 may include a step of verifying the correctness of one or more simplified source code samples 126 by the system 100 (e.g., via a verification component 202). For example, the verification step in 906 may include an attempt to compile one or more simplified source code samples 126. Simplified source code samples 126 that can be compiled may be determined to be correct and / or valid, while simplified source code samples 126 that cannot be compiled may be determined to be incorrect and / or invalid. In 908, the computer implementation method 900 may include a step of labeling one or more simplified source code samples 126 by the system 100 (e.g., via a labeler component 204) via a task-based labeling scheme. For example, a simplified source code sample 126 that maintains the task profile of a parent source code sample 123 may be labeled into one or more classes, groups and / or clusters based on a defined machine learning task.

[0069] In 910, the computer implementation method 900 may include a step of generating one or more augmented training datasets 404, which may contain one or more simplified source code samples 126 for training one or more AI models 124, by the system 100 (for example, via a dataset component 402). For example, the generation step in 910 may include a step of adding one or more simplified source code samples 126 to one or more training datasets in order to generate one or more augmented training datasets 404. In 912, the computer implementation method 900 may include a step of training one or more AI models 124 on one or more augmented training datasets 404 by the system 100 (for example, via a model training component 502).

[0070] While this disclosure includes a detailed description of cloud computing, it should be understood that the implementation of the teachings described herein is not limited to cloud computing environments. Rather, embodiments of the present invention can be implemented in conjunction with any other type of computing environment that is currently known or may be developed in the future.

[0071] Cloud computing is a service delivery model that enables convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be rapidly provisioned and deployed with minimal management effort or interaction with service providers. This cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0072] The characteristics are as follows:

[0073] On-demand self-service: Cloud consumers can unilaterally provision computing power, such as server time and network storage, automatically as needed, without requiring human interaction with service providers.

[0074] Broad network access: This capability is available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs®).

[0075] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, with various physical and virtual resources dynamically allocated and reallocated according to demand. Consumers generally have no control or knowledge of the exact location of the resources provided, although they may be able to specify the location at a higher level of abstraction (e.g., country, state, or data center), thus exhibiting a degree of location independence.

[0076] Rapid resilience: This capability allows for rapid and elastic provisioning, sometimes automatically, enabling quick scaling out and rapid release and rapid scaling in. To consumers, the capacity available for provisioning often appears unlimited and can be purchased in any quantity at any time.

[0077] Measurement Services: Cloud systems automatically control and optimize resource usage by leveraging measurement capabilities appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts) at a certain level of abstraction. Resource utilization is monitored, controlled, and reported, thereby providing transparency to both service providers and consumers.

[0078] The service model is as follows:

[0079] Software as a Service (SaaS): The ability offered to consumers is the use of a provider's applications running on cloud infrastructure. These applications are accessible from various client devices via thin client interfaces such as web browsers (e.g., web-based email). Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage, or even individual application capabilities, with the exception of limited user-specific application configuration settings.

[0080] Platform as a Service (PaaS): The ability offered to consumers is the ability to deploy applications they have created or acquired, written using programming languages ​​and tools supported by the provider, onto a cloud infrastructure. Consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but they do control the configuration of the deployed applications and, in some cases, the application hosting environment.

[0081] Infrastructure as a Service (IaaS): The ability provided to consumers is to provision processing, storage, networking, and other basic computing resources, allowing consumers to deploy and run any software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but they do control the operating system, storage, and deployed applications, and, in some cases, have limited control over selected networking components (e.g., host firewalls).

[0082] The deployment model is as follows:

[0083] Private Cloud: The cloud infrastructure is operated solely for the organization. It may be managed by the organization or a third party and may reside on-premises or off-premises.

[0084] Community Cloud: This cloud infrastructure is shared by several organizations and supports a specific community that shares common interests (e.g., mission, security requirements, policies, and compliance considerations). The community cloud may be managed by those organizations or third parties and may reside on-premises or off-premises.

[0085] Public cloud: Cloud infrastructure is made available to the general public or large industry groups and is owned by organizations that sell cloud services.

[0086] Hybrid Cloud: This cloud infrastructure is a complex of two or more clouds (private, community, or public) that remain unique entities but are joined together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing across clouds).

[0087] Cloud computing environments are service-oriented, focusing on statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing lies an infrastructure that includes a network of interconnected nodes.

[0088] Referring now to Figure 10, an exemplary cloud computing environment 1000 is shown. As shown, the cloud computing environment 1000 comprises one or more cloud computing nodes 1002 that can communicate with local computing devices used by cloud consumers, such as a personal digital assistant (PDA) or mobile phone 1004, a desktop computer 1006, a laptop computer 1008, and / or an automotive computer system 1010. The nodes 1002 can communicate with each other. They may be physically or virtually grouped within one or more networks, such as a private cloud, community cloud, public cloud, or hybrid cloud, or a combination thereof, as described above in this specification (not shown). This makes it possible for the cloud computing environment 1000 to provide infrastructure, platform, and / or software as a service that does not require cloud consumers to maintain resources on their local computing devices for that purpose. The types of computing devices 1004-1010 shown in Figure 10 are for illustrative purposes only, and it should be understood that computing node 1002 and cloud computing environment 1000 can communicate with any type of computerized device via any type of network and / or network addressable connection (e.g., using a web browser).

[0089] Referring now to Figure 11, a set of functional abstraction layers provided by the cloud computing environment 1000 (Figure 10) is shown. Repeated descriptions of similar elements used in other embodiments described herein are omitted for brevity. It should be understood in advance that the components, layers, and functionalities shown in Figure 11 are for illustrative purposes only and that embodiments of the present invention are not limited thereto. As shown, the following layers and corresponding functionalities are provided:

[0090] The hardware and software layer 1102 comprises hardware components and software components. Examples of hardware components include a mainframe 1104; a RISC (Reduced Instruction Set Computer) architecture-based server 1106; a server 1108; a blade server 1110; a storage device 1112; and network and networking components 1114. In some embodiments, the software components include network application server software 1116 and database software 1118.

[0091] The virtualization layer 1120 provides an abstraction layer from which the following examples of virtual entities may be provided: a virtual server 1122, virtual storage 1124, a virtual network 1126 including a virtual private network, a virtual application and operating system 1128, and a virtual client 1130.

[0092] In one example, the management layer 1132 may provide the functions described below. Resource provisioning 1134 provides dynamic procurement of computing resources and other resources used to perform tasks within the cloud computing environment. Measurement and pricing 1136 provides cost tracking as resources are used within the cloud computing environment and billing or invoicing for the consumption of these resources. In one example, these resources may include application software licenses. Security provides identity verification for cloud consumers and tasks, as well as protection of data and other resources. User portal 1138 provides access to the cloud computing environment for consumers and system administrators. Service level management 1140 provides cloud computing resource allocation and management to ensure that required service levels are met. Service Level Agreement (SLA) planning and execution 1142 provides pre-arrangement and procurement of cloud computing resources where future requirements are anticipated in accordance with SLAs.

[0093] The workload layer 1144 provides examples of functions for which a cloud computing environment may be utilized. Examples of workloads and functions that may be provided from this layer include mapping and navigation 1146, software development and lifecycle management 1148, provision of virtual classroom education 1150, data analysis processing 1152, transaction processing 1154, and AI model training 1156. Various embodiments of the present invention can utilize the cloud computing environment described with reference to Figures 10 and 11 to extend one or more training datasets and / or train one or more AI models.

[0094] The present invention may also be an integrated system, method, and / or computer program product at any possible level of technical detail. The computer program product may include a computer-readable storage medium (or a plurality of computer-readable storage media) having computer-readable program instructions for causing a processor to perform aspects of the present invention. A computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer-readable storage medium may, but is not limited to, electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any preferred combination thereof. A non-exhaustive list of further specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM, or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices such as punched cards or grooved structures on which instructions are recorded, and any preferred combination thereof. When used herein, computer-readable storage media should not be interpreted as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through optical fiber cables), or electrical signals transmitted through wires.

[0095] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to each computing / processing device, or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing / processing device receives computer-readable program instructions from the network and transfers such instructions for storage in a computer-readable storage medium within the respective computing / processing device.

[0096] The computer-readable program instructions for performing the operation of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk® or C++, and procedural programming languages ​​such as the "C" programming language or similar programming languages. The computer-readable program instructions may run as a standalone software package entirely on the user's computer, partially on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or wide area network (WAN), or the connection may be made to an external computer (for example, via the Internet using an Internet service provider). In some embodiments, to carry out aspects of the present invention, an electronic circuit including, for example, a programmable logic circuit, a field-programmable gate array (FPGA), or a programmable logic array (PLA) can execute a computer-readable program instruction by personalizing the electronic circuit using state information of the computer-readable program instruction.

[0097] Aspects of the present invention are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present invention. It will be understood that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0098] These computer-readable program instructions may be provided to the processor of a general-purpose computer, a dedicated computer, or other programmable data processing device to generate a machine, and the instructions executed via the processor of the computer or other programmable data processing device will create means for implementing functions / operations specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions may also be stored in a computer-readable storage medium that can instruct a computer, a programmable data processing device, and / or other device to function in a particular manner, and the computer-readable storage medium having instructions stored therein will comprise a product containing instructions that implement modes of functions / operations specified in one or more blocks of a flowchart and / or block diagram.

[0099] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing device, or other device to execute a series of operational steps on the computer, other programmable device, or other device, thereby generating a computer implementation process, the instructions executed on the computer, other programmable device, or other device, which implement the functions / operations specified in one or more blocks of a flowchart and / or block diagram.

[0100] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of the system, method, and computer program product according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of instructions containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions described within a block may occur in an order different from the order shown in the drawings. For example, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may be executed in reverse order depending on the related functions. It should also be noted that each block in the block diagram and / or flowchart diagram, and combinations of blocks in the block diagram and / or flowchart diagram, can be implemented by a dedicated hardware-based system that performs a specified function or operation, or a combination of dedicated hardware and computer instructions.

[0101] To provide additional context for the various embodiments described herein, Figure 12 and the following description are intended to provide a general description of a preferred computing environment 1200 in which various embodiments of the embodiments described herein may be implemented. Although the embodiments have been described above in the general context of computer executable instructions that may be run on one or more computers, those skilled in the art will recognize that the embodiments may also be implemented in combination with other program modules and / or as a combination of hardware and software.

[0102] Generally, a program module includes routines, programs, components, data structures, and / or similar items that perform a specific task or implement a specific abstract data type. Furthermore, those skilled in the art will understand that the methods of the present invention can be implemented using single-processor or multi-processor computer systems, minicomputers, mainframe computers, Internet of Things ("IoT") devices, distributed computing systems, and other computer system configurations including personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, each of which can be operably coupled to one or more associated devices.

[0103] The embodiments described herein may also be implemented in a distributed computing environment in which specific tasks are performed by remote processing devices linked through a communication network. In a distributed computing environment, program modules may reside in both local and remote memory storage devices. For example, in one or more embodiments, a computer executable component may include one or more distributed memory units, or may run from memory that can consist of one or more distributed memory units. As used herein, the terms “memory” and “memory unit” are interchangeable. Furthermore, one or more embodiments described herein may execute the code of a computer executable component in a distributed manner, for example, by having multiple processors working together or in conjunction to execute the code from one or more distributed memory units. As used herein, the term “memory” may encompass one or more locations of memory, or a single memory or memory unit in memory units at one or more locations.

[0104] Computing devices typically include a variety of media, which may include computer-readable storage media, machine-readable storage media, and / or communication media, as used herein, with the two terms being distinct from each other as follows: Computer-readable storage media or machine-readable storage media may be any available storage media that can be accessed by a computer and include both volatile and non-volatile media, and both removable and non-removable media. For example, but not limited to, computer-readable storage media or machine-readable storage media may be implemented in relation to any method or technique for storing information such as computer-readable or machine-readable instructions, program modules, structured data, or unstructured data.

[0105] Computer-readable storage media may include, but are not limited to, random access memory ("RAM"), read-only memory ("ROM"), electrically erasable programmable read-only memory ("EEPROM"), flash memory or other memory technologies, compact disk read-only memory ("CD-ROM"), digital versatile disk ("DVD"), Blu-ray disc ("BD") or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, solid-state drives or other solid-state storage devices, or other tangible and / or non-temporary media that can be used to store desired information. In this regard, the terms "tangible" or "non-temporary" as used herein to apply to storage, memory, or computer-readable media should be understood to exclude only temporary signals that propagate on their own as modifiers, and not to waive any rights to all standard storage, memory, or computer-readable media that are not merely temporary signals that propagate on their own.

[0106] Computer-readable storage media can be accessed by one or more local or remote computing devices for various operations relating to the information stored in the media, for example, via access requests, queries, or other data retrieval protocols.

[0107] Communication media typically include any information delivery or transport medium that embodies computer-readable instructions, data structures, program modules, or other structured or unstructured data in modulated data signals, such as carrier waves or other transport mechanisms. The term “modulated data signal” or “signal” refers to a signal having one or more of its characteristic sets, or a signal that has been modified in such a way as to encode information within one or more signals. Examples, but not limited to, include wired media such as wired networks or direct wired connections, and wireless media such as sound waves, RF, infrared, and other wireless media.

[0108] Referring again to Figure 12, an exemplary environment 1200 for implementing various embodiments of the embodiments described herein includes a computer 1202, a computer 1202 including a processing unit 1204, system memory 1206, and a system bus 1208. The system bus 1208 connects system components, including but not limited to the system memory 1206, to the processing unit 1204. The processing unit 1204 may be any of various commercially available processors. Dual microprocessor and other multiprocessor architectures may also be used as the processing unit 1204.

[0109] The system bus 1208 can be one of several types of bus structures that can further interconnect to a memory bus, peripheral bus, and local bus (with or without a memory controller) using any of a variety of commercially available bus architectures. The system memory 1206 includes ROM 1210 and RAM 1212. The basic input / output system ("BIOS") may be stored in non-volatile memory such as ROM, erasable programmable read-only memory ("EPROM"), or EEPROM, which contains basic routines that help transfer information between elements within the computer 1202 during startup, etc. RAM 1212 may also include high-speed RAM such as static RAM for caching data.

[0110] Computer 1202 further comprises an internal hard disk drive ("HDD") 1214 (e.g., EIDE, SATA), one or more external storage devices 1216 (e.g., magnetic floppy disk drive ("FDD") 1216, memory stick or flash drive reader, memory card reader, combination thereof, and / or similar), and an optical disk drive 1220 (e.g., capable of reading from or writing to CD-ROM disks, DVDs, BDs, and / or similar). Although the internal HDD 1214 is shown as being located within computer 1202, the internal HDD 1214 may also be configured for external use in a suitable chassis (not shown). In addition, although not shown in environment 1200, a solid-state drive ("SSD") may be used in addition to or instead of the HDD 1214. The HDD 1214, external storage device 1216, and optical disc drive 1220 may be connected to the system bus 1208 by the HDD interface 1224, external storage interface 1226, and optical drive interface 1228, respectively. The interface 1224 for external drive implementation may include at least one or both of the Universal Serial Bus ("USB") and the Institute of Electrical and Electronics Engineers ("IEEE") 1394 interface technologies. Other external drive connection technologies are included in the intent of the embodiments described herein.

[0111] The drive and its associated computer-readable storage medium provide non-volatile storage of data, data structures, computer-executable instructions, etc. For computer 1202, the drive and storage medium adapt the storage of any data in a suitable digital format. While the above description of computer-readable storage medium refers to each type of storage device, it should be understood by those skilled in the art that other types of computer-readable storage mediums, whether currently existing or to be developed in the future, may also be used in the exemplary operating environment, and furthermore, any such storage medium may contain computer-executable instructions for performing the methods described herein.

[0112] An operating system 1230, one or more application programs 1232, other program modules 1234, and several program modules, including program data 1236, may be stored in the drive and RAM 1212. All or part of the operating system, applications, modules, and / or data may also be cached in RAM 1212. The systems and methods described herein may be implemented using various commercially available operating systems or combinations of operating systems.

[0113] Computer 1202 may optionally be equipped with emulation technology. For example, a hypervisor (not shown) or other intermediary may emulate a hardware environment for operating system 1230, and the emulated hardware may optionally differ from the hardware shown in Figure 12. In such an embodiment, operating system 1230 may include one VM from a plurality of virtual machines ("VMs") hosted on computer 1202. Furthermore, operating system 1230 may provide a runtime environment for application 1232, such as a Java® runtime environment or a .NET framework. The runtime environment is a consistent execution environment that enables application 1232 to run on any operating system that includes the runtime environment. Similarly, operating system 1230 may support containers, for example, lightweight, standalone, and executable packages of software including code, runtime, system tools, system libraries, and configuration for the application; application 1232 may be in the form of a container.

[0114] Furthermore, computer 1202 can be enabled using security modules such as a Trusted Processing Module ("TPM"). For example, with a TPM, the boot component hashs the next boot component in time before loading the next boot component and waits for the result to match a protected value. This process can occur at any layer in the code execution stack of computer 1202, for example, at the application execution level or the operating system ("OS") kernel level, thereby enabling security at any level of code execution.

[0115] The user may input commands and information to the computer 1202 through one or more wired / wireless input devices, such as a keyboard 1238, a touchscreen 1240, and a pointing device such as a mouse 1242. Other input devices (not shown) may include microphones, infrared ("IR") remote controls, radio frequency ("RF") remote controls, or other remote controls, joysticks, virtual reality controllers and / or virtual reality headsets, gamepads, stylus pens, image input devices such as cameras, gesture sensor input devices, vision movement sensor input devices, facial expression or face detection devices, biometric input devices such as fingerprint or iris scanners, etc. These and other input devices are often connected to the processing unit 1204 through an input device interface 1244 which can be coupled to the system bus 1208, but may also be connected by other interfaces such as parallel ports, IEEE 1394 serial ports, game ports, USB ports, IR interfaces, BLUETOOTH® interfaces, and / or similar.

[0116] The monitor 1246 or other types of display devices may also be connected to the system bus 1208 via an interface such as the video adapter 1248. In addition to the monitor 1246, the computer typically includes other peripheral output devices (not shown) such as speakers, printers, combinations thereof, and / or similar.

[0117] Computer 1202 may operate in a networked environment using logical connections via wired and / or wireless communication to one or more remote computers, such as remote computers 1250. Remote computers 1250 can be workstations, server computers, routers, personal computers, portable computers, microprocessor-based entertainment equipment, peer devices, or other common network nodes, typically including many or all of the elements described with respect to computer 1202, although typically only a memory / storage device 1252 is shown for brevity. The shown logical connections include wired / wireless connectivity to a local area network ("LAN") 1254 and / or a larger network, such as a wide area network ("WAN") 1256. Such LAN and WAN networking environments are common in offices and enterprises, facilitating enterprise-scale computer networks such as intranets, all of which may connect to global communication networks, such as the Internet.

[0118] When used in a LAN networking environment, computer 1202 can connect to local network 1254 via a wired and / or wireless network interface or adapter 1258. Adapter 1258 can facilitate wired or wireless communication to LAN 1254, which may also include a wireless access point ("AP") located therein to communicate with adapter 1258 in wireless mode.

[0119] When used in a WAN networking environment, computer 1202 may include a modem 1260 or may connect to a communication server on WAN 1256 via other means for establishing communication on WAN 1256, such as the Internet. The modem 1260, which may be internal or external and may be a wired or wireless device, may connect to the system bus 1208 via an input device interface 1244. In a networked environment, program modules or parts thereof shown to computer 1202 may be stored in a remote memory / storage device 1252. The shown network connections are illustrative, and it will be understood that other means for establishing communication links between computers may be used.

[0120] When used in either a LAN or WAN networking environment, computer 1202 may access a cloud storage system or other network-based storage system in addition to, or instead of, the external storage device 1216 as described above. Generally, the connection between computer 1202 and the cloud storage system may be established over LAN 1254 or WAN 1256, for example, by adapter 1258 or modem 1260, respectively. When computer 1202 is connected to the associated cloud storage system, the external storage interface 1226 may manage the storage provided by the cloud storage system, with the assistance of adapter 1258 and / or modem 1260, as to manage other types of external storage. For example, the external storage interface 1226 may be configured to provide access to cloud storage sources as if those sources were physically connected to computer 1202.

[0121] Computer 1202 may be capable of communicating with any wireless device or entity configured to operate in wireless communication, such as printers, scanners, desktop and / or portable computers, personal digital assistants, communication satellites, any equipment or location associated with wirelessly discoverable tags (e.g., kiosks, newsstands, store shelves, and / or similar), and telephones. This may include Wireless Fidelity ("Wi-Fi" registered trademark) and Bluetooth® wireless technologies. Thus, communication may be a predefined structure, similar to conventional networks, or simply ad-hoc communication between at least two devices.

[0122] The above descriptions include only examples of systems, computer program products, and computer implementation methods. Naturally, it is not possible to describe every conceivable combination of components, products, and / or computer implementation methods for the purposes of illustrating this disclosure, and those skilled in the art will recognize that many further combinations and substitutions of the disclosure are possible. Furthermore, to the extent that terms such as “includes,” “has,” and “possesse” are used in the detailed description, claims, appendices, and drawings, such terms are intended to be comprehensive in the same manner as the term “comprising” is interpreted when “comprising” is used as a transitional term in the claims. The descriptions of various embodiments have been presented for illustrative purposes, but are not intended to be comprehensive or limitless to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the embodiments described. The terminology used herein has been selected to best describe the principles of the embodiments, their practical applications, or technical improvements over technologies found in the market, or to enable other persons skilled in the art to understand the embodiments disclosed herein. 。 [Item 1] Memory for storing computer executable components; and A processor operably coupled to the memory and capable of executing the computer executable component stored in the memory, wherein the computer executable component is: A training extension component that generates an extended training dataset for training an artificial intelligence model by extracting simplified source code samples from source code samples included in the training dataset. A processor having A system that includes these features. [Item 2] A simplification component that generates a plurality of simplified source code samples from the source code sample through a series of iterative reductions of the source code sample, wherein each iteration of the series of iterative reductions generates a new simplified source code sample that is simpler than the previous simplified source code sample generated by the series of iterative reductions. The system described in item 1, further comprising the features described above. [Item 3] A verification component for verifying the accuracy of the simplified source code sample; and A labeler component that labels the simplified source code sample via a task-specific labeling scheme. The system described in item 1, further comprising the features described above. [Item 4] Following authentication by the verification component and labeling by the labeler component, the dataset component adds the simplified source code sample to the training dataset. The system described in item 3 further includes the features described therein. [Item 5] A model training component that trains the artificial intelligence model from the aforementioned expanded training dataset. The system described in item 4, which further includes the features described therein. [Item 6] Memory for storing computer executable components; and A processor operably coupled to the memory and capable of executing the computer executable component stored in the memory, wherein the computer executable component is: A model training component that trains an artificial intelligence model on an extended training dataset that includes simplified source code samples from the training dataset. A processor having A system that includes these features. [Item 7] A simplification component that executes a simplification algorithm to generate the simplified source code sample by reducing the source code sample. The system described in item 6, which further includes the features described therein. [Item 8] The simplification component generates a number of simplified source code samples from the source code sample through a series of iterative reductions, as described in item 7. [Item 9] A verification component for verifying the accuracy of the simplified source code sample; A labeler component that labels the simplified source code samples via a task-based labeling scheme; and Following authentication by the verification component and labeling by the labeler component, the dataset component generates the expanded training dataset by adding the simplified source code sample to the training dataset. The system described in item 6, which further includes the features described therein. [Item 10] A computer implementation method comprising a step in which a system operablely coupled to a processor generates an augmented training dataset for training an artificial intelligence model by extracting simplified source code samples from source code samples contained within a training dataset. [Item 11] The system comprises the step of generating a plurality of simplified source code samples from the source code sample through a series of iterative reductions of the source code sample, wherein the simplified source code samples are derived from the plurality of simplified source code samples. The computer implementation method described in item 10, further comprising the features described therein. [Item 12] The system utilizes a delta debugging algorithm to perform the series of iterative reductions. A computer implementation method as described in item 11, further comprising the features described therein. [Item 13] The computer implementation method according to item 11, wherein the iteration of the series of iterative reductions generates a new simplified source code sample that is simpler than the previous simplified source code sample produced by the series of iterative reductions. [Item 14] The system includes the step of verifying the accuracy of the simplified source code sample; and The system labels the simplified source code sample via a task-specific labeling scheme. The computer implementation method described in item 10, further comprising the features described therein. [Item 15] The system adds the simplified source code sample to the training dataset, following the verification and labeling; and The system trains the artificial intelligence model from the expanded training dataset. A computer implementation method as described in item 14, further comprising the features described therein. [Item 16] A computer implementation method comprising a step in which a system operablely coupled to a processor trains an artificial intelligence model on an augmented training dataset containing simplified source code samples of source code samples from the training dataset. [Item 17] The system executes a simplification algorithm to generate the simplified source code samples by reducing the source code samples from the training dataset. A computer implementation method as described in item 16, further comprising the features described therein. [Item 18] The system generates multiple source code samples by performing a series of iterative reductions, where each iteration of the series of iterative reductions generates a new simplified source code sample that is simpler than the previous simplified source code sample generated by the series of iterative reductions. A computer implementation method as described in item 16, further comprising the features described therein. [Item 19] The system then verifies the accuracy of the simplified source code sample; The system labels the simplified source code sample via a task-based labeling scheme; and The system adds the simplified source code sample to the training dataset following the verification and labeling. A computer implementation method as described in item 16, further comprising the features described therein. [Item 20] A computer program product for training an artificial intelligence model, wherein the computer program product comprises a computer-readable storage medium having program instructions embodied therein, and the program instructions are transmitted to a processor: The processor generates an expanded training dataset for training the artificial intelligence model by extracting simplified source code samples from the source code samples included in the training dataset. A computer program product that is executable by the processor to perform the following actions. [Item 21] The aforementioned program instruction is, The processor generates a plurality of simplified source code samples from the source code sample through a series of iterative reductions of the source code sample, wherein the simplified source code samples are derived from the plurality of simplified source code samples. A computer program product as described in item 20, which further causes the aforementioned processor to execute the aforementioned program. [Item 22] The computer program product described in item 21, wherein the iteration of the series of iterative reductions generates a new simplified source code sample that is simpler than the previous simplified source code sample produced by the series of iterative reductions. [Item 23] The aforementioned program instruction is, The processor utilizes a delta debugging algorithm to perform the series of iterative reductions. A computer program product as described in item 22, which causes the processor to execute the aforementioned. [Item 24] The aforementioned program instruction is, The processor verifies the accuracy of the simplified source code sample; and The processor labels the simplified source code sample via a task-based labeling scheme. A computer program product as described in item 20, which causes the processor to further execute the aforementioned. [Item 25] The aforementioned program instruction is, The processor adds the source code samples to the training dataset after validation and labeling in order to generate the expanded training dataset; and The processor trains the artificial intelligence model from the expanded training dataset. A computer program product as described in item 23, which causes the processor to further execute the aforementioned.

Claims

1. Memory for storing computer executable components; and A processor operably coupled to the memory and capable of executing the computer executable component stored in the memory, wherein the computer executable component is: From multiple source code samples within the training dataset, source code samples with a threshold level of complexity are identified according to the code complexity metric. A simplification component that generates a plurality of simplified source code samples from the source code sample through a series of iterative code reductions of the source code sample, wherein the plurality of simplified source code samples have different amounts of complexity according to the code complexity metric, and A training extension component that generates an extended training dataset for training an artificial intelligence model to incorporate code complexity into the artificial intelligence model by selecting a simplified source code sample from a subset of the multiple simplified source code samples that are determined to be correct by one or more accuracy criteria and have the same task profile as the source code sample by one or more verification criteria, and adding the simplified source code sample to the training dataset to generate an extended training dataset. A processor having A system equipped with these features.

2. A model training component for training the artificial intelligence model using the expanded training dataset, wherein the training comprises iteratively training the artificial intelligence model by selecting simplified source code samples from the subset of a plurality of simplified source code samples in an order progressing from minimum code complexity to maximum code complexity. The system according to claim 1.

3. Herein, the iteration of a series of iterative code reductions generates a new simplified source code sample that is simpler than the previous simplified source code sample generated by the series of iterative code reductions. The system according to claim 1.

4. A verification component for verifying the accuracy of the simplified source code sample; and A labeler component that labels the simplified source code sample via a task-specific labeling scheme. The system according to any one of claims 1 to 3, further comprising the above.

5. Following authentication by the verification component and labeling by the labeler component, the dataset component adds the simplified source code sample to the training dataset. The system according to claim 4, further comprising:

6. A model training component that trains the artificial intelligence model from the aforementioned expanded training dataset. The system according to claim 5, further comprising:

7. Memory for storing computer executable components; and A processor operably coupled to the memory and capable of executing the computer executable component stored in the memory, wherein the computer executable component is: A simplification component that generates a number of simplified source code samples from source code samples in a training dataset through a series of iterative code reductions of said source code samples, wherein the simplification component utilizes a delta debugging algorithm for the series of iterative code reductions, A training extension component that generates an extended training dataset for training an artificial intelligence model by selecting a simplified source code sample from the aforementioned multiple simplified source code samples and adding the simplified source code sample to the training dataset. A processor having A system equipped with these features.

8. Memory for storing computer executable components; and A processor operably coupled to the memory and capable of executing the computer executable component stored in the memory, wherein the computer executable component is: From multiple source code samples within the training dataset, source code samples with a threshold level of complexity are identified according to the code complexity metric. A simplification component that generates a plurality of simplified source code samples from the source code sample using a simplification algorithm, wherein the simplification algorithm removes one or more portions of the source code sample, and the plurality of simplified source code samples have less complexity according to the code complexity metric. A model training component that trains an artificial intelligence model to incorporate code complexity into the artificial intelligence model, based on an extended training dataset containing a subset of the aforementioned simplified source code samples. A processor having A system comprising, wherein the subset of the plurality of simplified source code samples is determined to be correct by one or more accuracy criteria, and the training includes iteratively training the artificial intelligence model by selecting simplified source code samples from the subset of the plurality of simplified source code samples in an order progressing from the minimum code complexity to the maximum code complexity.

9. The system according to claim 8, wherein the simplified source code sample is selected from a subset of the plurality of simplified source code samples such that it has the same task profile as the source code sample with respect to one or more verification criteria.

10. A verification component for verifying the accuracy of the simplified source code sample; A labeler component that labels the simplified source code samples via a task-based labeling scheme; and Following authentication by the verification component and labeling by the labeler component, the dataset component generates the expanded training dataset by adding the simplified source code sample to the training dataset. The system according to claim 8 or 9, further comprising:

11. Memory for storing computer executable components; and A processor operably coupled to the memory and capable of executing the computer executable component stored in the memory, wherein the computer executable component is: A simplification component that generates a plurality of simplified source code samples from a source code sample in a training dataset through a series of iterative code reductions of said source code samples, wherein the simplification component utilizes a delta debugging algorithm for the series of iterative code reductions, and A model training component that trains an artificial intelligence model on an extended training dataset containing a subset of the aforementioned simplified source code samples. A processor having A system equipped with these features.

12. A system operablely coupled to a processor identifies source code samples from a training dataset that have a threshold level of complexity according to a code complexity metric. The system comprises the steps of generating a plurality of simplified source code samples from the source code sample through a series of iterative code reductions of the source code sample, wherein the plurality of simplified source code samples have different amounts of complexity according to the code complexity metric, The system generates an expanded training dataset for training the artificial intelligence model to incorporate code complexity into the artificial intelligence model by selecting a simplified source code sample from a plurality of simplified source code samples that are determined to be correct by one or more accuracy criteria and have the same task profile as the source code sample by one or more verification criteria, and adding the simplified source code sample to the training dataset to generate an expanded training dataset. The system includes the step of training the artificial intelligence model using the expanded training dataset. A computer implementation method comprising the following features.

13. The system utilizes a delta debugging algorithm to perform the series of iterative code reductions. The computer implementation method according to claim 12, further comprising:

14. The computer implementation method according to claim 12, wherein the series of iterative code reduction iterations generates a new simplified source code sample that is simpler than the previous simplified source code sample generated by the series of iterative code reductions.

15. The system performs the step of verifying the accuracy of the simplified source code sample; and The system labels the simplified source code sample via a task-specific labeling scheme. A computer implementation method according to any one of claims 12 to 14, further comprising the above.

16. The system adds the simplified source code sample to the training dataset, following the verification and labeling; and The system trains the artificial intelligence model from the expanded training dataset. The computer implementation method according to claim 15, further comprising:

17. A system operably coupled to a processor generates a plurality of simplified source code samples from a source code sample through a series of iterative code reductions of the source code sample contained in a training dataset, The system utilizes a delta debugging algorithm to perform the series of iterative code reductions, The system generates an expanded training dataset for training an artificial intelligence model by selecting a simplified source code sample from the plurality of simplified source code samples and adding the simplified source code sample to the training dataset. A computer implementation method comprising the following features.

18. A system operablely coupled to a processor identifies source code samples from multiple source code samples in a training dataset that have a threshold level of complexity according to a code complexity metric. A system operablely coupled to a processor generates a plurality of simplified source code samples from the source code sample using a simplification algorithm, wherein the simplification algorithm removes one or more portions of the source code sample, and the plurality of simplified source code samples have less complexity than the source code sample according to the code complexity metric. The system includes the step of training the artificial intelligence model to incorporate code complexity into the artificial intelligence model based on an expanded training dataset which includes a subset of the plurality of simplified source code samples. The system comprises a subset of the plurality of simplified source code samples being determined to be correct according to one or more accuracy criteria, and the training includes iteratively training the artificial intelligence model by selecting simplified source code samples from the subset of the plurality of simplified source code samples in an order progressing from the minimum code complexity to the maximum code complexity. Computer implementation method.

19. The system then verifies the accuracy of the simplified source code sample; The system labels the simplified source code sample via a task-based labeling scheme; and The system adds the simplified source code sample to the training dataset following the verification and labeling. The computer implementation method according to claim 18, further comprising:

20. A step in which a system operably coupled to a processor executes a simplification algorithm to generate a plurality of simplified source code samples by reducing source code samples from a training dataset, wherein a delta debugging algorithm is used to perform a series of iterative code reductions. The system includes the step of training an artificial intelligence model on an expanded training dataset which includes a subset of the plurality of simplified source code samples, A computer implementation method comprising the following features.

21. A computer program for training an artificial intelligence model, with a processor: The processor includes a procedure for identifying source code samples from multiple source code samples in a training dataset that have a threshold level of complexity according to a code complexity metric, A procedure comprising the processor generating a plurality of simplified source code samples from the source code sample through a series of iterative code reductions of the source code sample, wherein the plurality of simplified source code samples have different amounts of complexity according to the code complexity metric, A procedure by which the processor generates an expanded training dataset for training the artificial intelligence model to incorporate code complexity into the artificial intelligence model, by selecting a simplified source code sample from a subset of a plurality of simplified source code samples that are determined to be correct by one or more accuracy criteria and have the same task profile as the source code sample by one or more verification criteria, and adding the simplified source code sample to the training dataset to generate an expanded training dataset. The processor performs the procedure of training the artificial intelligence model using the expanded training dataset. A computer program designed to execute something.

22. To the aforementioned processor: The procedure by which the processor utilizes a delta debugging algorithm to perform the series of iterative code reductions. A computer program according to claim 21 for causing execution.

23. To the aforementioned processor: The processor provides a procedure for verifying the accuracy of the simplified source code sample; and The processor performs a procedure to label the simplified source code sample via a task-based labeling scheme. A computer program according to claim 21 or 22 for further execution.

24. To the aforementioned processor: The processor performs the following steps to generate the expanded training dataset: validation and labeling, followed by adding the source code samples to the training dataset. A computer program according to claim 22 for further execution.

25. A computer program for training an artificial intelligence model, with a processor: The process involves the processor generating a number of simplified source code samples from the source code samples through a series of iterative code reductions of the source code samples included in the training dataset, The process involves the processor utilizing a delta debugging algorithm to perform the series of iterative code reductions, The process involves the processor selecting a simplified source code sample from the plurality of simplified source code samples and adding the simplified source code sample to the training dataset to generate an expanded training dataset for training the artificial intelligence model. A computer program designed to execute something.

Images