Bootstrap program synthesis language model to perform repair

Abstract

Certain aspects of the present disclosure provide techniques and apparatus for improved program synthesis using machine learning. Input indicative of a programming task is accessed. A generated program is generated based on processing the input using the trained machine learning model. In response to determining that the generated program fails to satisfy the programming task, feedback is generated, and a revised program is generated based on processing the feedback using the trained machine learning model. In response to determining that the revised program satisfies the programming task, one or more parameters of the trained machine learning model are updated based on the revised program.

Description

[0001] Cross-references to related applications

[0002] This application claims priority to U.S. Patent Application No. 18 / 455,992, filed August 25, 2023, which has been assigned to the assignee of this application and is expressly incorporated herein by reference in its entirety, as fully set forth below and for all applicable purposes. Background Technology

[0003] This disclosure relates to various aspects of machine learning.

[0004] A wide variety of machine learning model architectures have proliferated and have been used to provide solutions for a wide range of prediction problems. Language models (also known as large language models (LLMs)) are a class of models that learn to generate text (such as natural language). Language models have been used in a variety of solutions, including chatbots, program synthesis, and more. In program synthesis tasks, the model attempts to learn to generate computer program code that can be executed (by a computer) to perform a desired task. In some conventional systems, neurally guided program synthesis methods rely on using domain-specific languages ​​(DSLs) due to the extremely large search space of popular programming languages.

[0005] In some conventional approaches, language models for program synthesis have been developed to improve upon DSL-based methods. These language models can be trained and evaluated on programming competition datasets, such as the most basic Python Programming (MBPP) dataset or the Automatic Programming Progress Standard (APPS) dataset. Such datasets typically contain hints describing the computational task or problem that the generated program should solve or satisfy, along with one or more samples of program code that solves or performs the task (e.g., written by human programmers). However, such datasets are largely limited in size (with relatively few examples) and quality (some of which claim to provide solutions that don't work or solve the corresponding task). The problem is that many machine learning models, especially large language models, rely on large amounts of highly accurate training data to learn to produce accurate outputs. Summary of the Invention

[0006] One aspect provides a processing system comprising: one or more memories including processor-executable instructions; and one or more processors configured to execute the processor-executable instructions and cause the processing system to: access a first input indicating a first programming task; generate a first generated program, wherein, in order to generate the first generated program, the one or more processors are configured to execute the processor-executable instructions to cause the processing system to process the first input using a trained machine learning model; determine that the first generated program fails to satisfy the first programming task; generate first feedback based on the determination that the first generated program fails to satisfy the first programming task; generate a first revised program, wherein, in order to generate the first revised program, the one or more processors are configured to execute the processor-executable instructions to cause the processing system to process the first feedback using the trained machine learning model; determine that the first revised program satisfies the first programming task; and, in response to the determination that the first revised program satisfies the first programming task, update one or more parameters of the trained machine learning model based on the first revised program.

[0007] Some aspects provide a method comprising: accessing a first input indicative of a first programming task; generating a first generated program based on processing the first input using a trained machine learning model; generating first feedback based on the determination that the first generated program fails to satisfy the first programming task in response to determining that the first generated program fails to satisfy the first programming task; generating a first revised program based on processing the first feedback using the trained machine learning model; and updating one or more parameters of the trained machine learning model based on the first revised program in response to determining that the first revised program satisfies the first programming task.

[0008] Some aspects provide one or more non-transitory computer-readable media, the one or more non-transitory computer-readable media including computer-executable instructions, which, when executed by one or more processors of one or more processing systems, cause the one or more processing systems to: access a first input indicating a first programming task; generate a first generated program, wherein, in order to generate the first generated program, the one or more non-transitory computer-readable media further includes computer-executable instructions, which, when executed by the one or more processors, cause the one or more processing systems to process the first input using a trained machine learning model; determine that the first generated program fails to satisfy... The first programming task; generating first feedback based on the determination that the first generated program fails to satisfy the first programming task; generating a first revised program, wherein, in order to generate the first revised program, the one or more non-transitory computer-readable media further includes computer-executable instructions that, when executed by the one or more processors, cause the one or more processing systems to use a trained machine learning model to process the first feedback; determining that the first revised program satisfies the first programming task; and updating one or more parameters of the trained machine learning model based on the first revised program in response to the determination that the first revised program satisfies the first programming task.

[0009] Other aspects include: processing systems configured to perform the foregoing methods and those methods described herein; a non-transitory computer-readable medium including instructions that, when executed by one or more processors of the processing system, cause the processing system to perform the foregoing methods and those methods described herein; a computer program product embodied on a computer-readable storage medium including code for performing the foregoing methods and those methods further described herein; and a processing system including components for performing the foregoing methods and those methods further described herein.

[0010] The following description and related figures illustrate certain exemplary features of one or more aspects. Attached Figure Description

[0011] The accompanying drawings depict certain features of one or more aspects of this disclosure and are therefore not intended to limit the scope of this disclosure.

[0012] Figure 1 An updated example environment for improving procedural synthetic machine learning models is illustrated according to various aspects of this disclosure.

[0013] Figure 2An updated example workflow for improving procedural synthetic machine learning models is described according to various aspects of this disclosure.

[0014] Figure 3 This is a flowchart depicting an updated example method for improving procedural synthetic machine learning models using bootstrapping and procedural repair, according to various aspects of this disclosure.

[0015] Figure 4 This is a flowchart depicting an example method for generating fine-tuning data for procedurally synthesized machine learning models, according to various aspects of this disclosure.

[0016] Figure 5 This is a flowchart depicting example methods for updating machine learning models for use in procedural synthesis, according to various aspects of this disclosure.

[0017] Figure 6 Example processing systems configured to perform various aspects of this disclosure are described.

[0018] For ease of understanding, the same reference numerals are used where possible to designate common elements shared by all figures. Elements and features conceived in one aspect can be beneficially incorporated into other aspects without further elaboration. Detailed Implementation

[0019] Various aspects of this disclosure provide apparatus, methods, processing systems, and computer-readable media for improved training and fine-tuning of machine learning models using bootstrapping and procedural repair.

[0020] In various aspects of this disclosure, a combination of bootstrapping and procedural refinement can be used to train, update, and / or fine-tune machine learning models (e.g., large language models). As used herein, “fine-tuning” refers to updating the parameters of a machine learning model (often referred to as the base model) that has already been at least partially trained. For example, a model can be trained using a set of data and fine-tuned for a specific task (e.g., procedural synthesis) based on a set of fine-tuning data. As used herein, “bootstrapping” generally refers to refining the model itself using the model’s output. For example, a system can use correctly generated output (e.g., a synthesized program that accurately performs the desired task) as training data during the refinement or fine-tuning phase.

[0021] Additionally, as used herein, "repair" refers to the process of iteratively attempting to generate improved program code. For example, if the initial synthesized program fails to solve or satisfy the task (e.g., due to compilation errors, runtime errors, or errors in the program's logic), the system can use these errors to prompt the model to generate an updated program (e.g., attempting to repair the first generated program). This use of bootstrapping with an iterative repair process enables fine-tuning of the model's improvements, leading to significantly improved prediction accuracy (e.g., the improved ability to generate program code that satisfies the desired task).

[0022] In some respects, using a decoded language model (also known as a program synthesis machine learning model), a machine learning system generates a code solution (also known as a generated program) for a programming problem or task. The machine learning system can then attempt to validate or verify the generated program (e.g., using a compiler, using test cases, etc.) to determine whether the generated code is accurate (e.g., whether the generated code can be compiled into an effective program that accurately solves or meets the expected task).

[0023] In some respects, if a generated program fails validation, feedback indicating the failure (e.g., compiler / interpreter information) can be provided as input to the machine learning model, prompting it to generate a new program (referred to in some respects as a revised program). In some respects, if the model produces the correct answer (e.g., code that compiles and solves the problem efficiently), the model output can be used during the fine-tuning phase. In some respects, if the model fails to produce the correct solution (e.g., the revised code still fails), the system can use a ground-valued solution for the task during the fine-tuning phase.

[0024] In this way, during fine-tuning, the model can be significantly improved compared to some conventional training and fine-tuning methods that rely on the programming task dataset itself. For example, using aspects of this disclosure, the model is exposed to more diverse training data (potentially including information from the compiler / interpreter), which can have higher quality than the true values ​​in the available dataset. Additionally, allowing the model to attempt to fix faulty procedures can lead to significantly improved output, and this iterative improvement process yields more useful fine-tuning data.

[0025] An updated example environment for improving procedural synthetic machine learning models.

[0026] Figure 1 An updated example environment 100 for procedural synthesis of machine learning models is illustrated, according to various aspects of this disclosure.

[0027] In the illustrated environment 100, a set of task descriptions 110 is accessed by a fine-tuning system 125. In some aspects, the fine-tuning system 125 corresponds to a computational system that fine-tunes (e.g., updates) a trained machine learning model to improve (or attempts to improve) the model's inference accuracy. In some aspects, the fine-tuning system 125 may additionally serve as a training system (e.g., a computational system for initially training a model) and / or as an inference system (e.g., a computational system for generating inference during runtime using a trained and / or fine-tuned model).

[0028] In the illustrated example, the fine-tuning system 125 further accesses the machine learning model 105, using the task description 110 to update or revise the machine learning model 105 to generate a revised machine learning model 130. As used herein, "accessing" data (e.g., model parameters) can generally include receiving, retrieving, requesting, collecting, obtaining, generating, or otherwise acquiring access to data (e.g., model parameters).

[0029] In some respects, machine learning model 105 is a pre-trained machine learning model, such as a large language model. In other respects, machine learning model 105 has been trained for program synthesis and is revised or fine-tuned (e.g., by fine-tuning system 125) to generate more accurate programs. As discussed above, program synthesis generally corresponds to the generation of computer program code that can be compiled and executed to perform a desired task. In some respects, machine learning model 105 is initially trained on a relatively large code corpus, allowing machine learning model 105 to learn to synthesize new code. During the fine-tuning phase (also known as the refinement phase, update phase, or simply the training phase), fine-tuning system 125 uses task description 110 to teach the machine learning model (revised machine learning model 130) how to generate program code that solves the desired task.

[0030] In some aspects, each task description 110 typically includes a description of the programming task (also known as a hint), one or more unit tests 115 (also known in some aspects as test cases and / or assertions) and one or more solutions 120 (also known as ground truths or labels). For example, in some aspects, the task description 110 may correspond to samples from programming competition datasets (e.g., the MBPP dataset and / or the APPS dataset).

[0031] In some aspects, the hint portion of task description 110 may include a natural language description (e.g., describing the task in natural language) and / or a logical description (e.g., describing the task according to Boolean logic or other operators). For example, task description 110 may include a hint such as “Write a function to find the first repeating character in a given string.” Additional examples may include, but are not limited to, hints such as “Write a function to find the longest possible chain from a given set of pairs of numbers” or “Write a function to obtain a clear number less than or equal to n.” In general, hints can be of any length and may include unstructured natural language, as well as other information such as numerical data, structured data, etc.

[0032] In some aspects, each task description 110 may include zero or more unit tests 115. Generally, each unit test 115 specifies a test that can be used to determine the validity or accuracy of the generated code. For example, continuing the programming task described above to find repeating characters in a string, a unit test 115 might specify that when given the string "ABCABC", the program should output the character "A". In some aspects, some or all of these unit tests 115 are provided as part of the input (along with prompts) to the machine learning model 105, as discussed in more detail below. In some aspects, some or all of these unit tests 115 are hidden (e.g., not provided to the model) and used to validate the resulting code.

[0033] In some respects, solution 120 typically refers to one or more ground truth solutions for a corresponding programming task. That is, solution 120 may typically include computer program code (purely) solving the task, such as code that can be compiled and executed to find and output the first repeating character in a string. In some respects, some or all of the solutions in solution 120 are manually created (e.g., by a programmer) and / or generated by other procedural synthetic models. Although referred to as “solution” and “ground truth” answers in some examples described herein, in some respects, one or more solutions in solution 120 may actually contain incorrect answers. That is, solution 120 itself may contain errors that lead to invalid code (e.g., code that does not compile, code that returns an incorrect answer, etc.). Unfortunately, such errors are common in many available datasets. However, by using aspects of this disclosure to provide bootstrapping and procedural fixes, the resulting revised machine learning model 130 can still be significantly improved. For example, because the validity of the model output is determined based on criteria other than solution 120 itself, and the fine-tuning system 125 can use labels other than solution 120 itself, the model can be fine-tuned based on more accurate data.

[0034] In some aspects, the fine-tuning system 125 may use the machine learning model 105 to process each task description 110 (e.g., natural language text describing a programming task) to generate a corresponding program. The fine-tuning system 125 may then verify (or attempt to verify) the generated program, such as by determining whether the program compiled successfully (as opposed to generating a compile or interpreter error), whether the program ran successfully (as opposed to generating a runtime error), and / or whether the program correctly handled the unit tests 115 (e.g., whether the program returned the correct result based on the tests). In some aspects, if the generated program is verified (e.g., the program successfully performed the programming task), the fine-tuning system 125 may use the generated program as a ground truth label to fine-tune the machine learning model 105 (immediately, or after multiple task descriptions 110 have been processed), as discussed in more detail below.

[0035] In some respects, if the generated program fails validation (e.g., the program causes generation errors and / or returns inaccurate results), the fine-tuning system 125 can prompt the machine learning model 105 to attempt to fix the code and / or generate new code. For example, the fine-tuning system 125 can provide feedback (such as generated errors) as input, allowing the machine learning model 105 to generate revised code. As discussed in more detail below, the revised code may include the originally generated code with revision or fix attempts (e.g., modifications to the program) and / or may include entirely new code (e.g., the model may attempt to write entirely new code).

[0036] In some respects, the fine-tuning system 125 can similarly validate (or attempt to validate) the revised procedure. If the revised procedure is successfully validated, in some respects, the fine-tuning system 125 can use the revised procedure as a ground truth label to fine-tune the machine learning model 105 (immediately, or after multiple task descriptions 110 have been processed), as discussed in more detail below. In some respects, if the revised procedure still fails validation, the fine-tuning system 125 can use the solution 120 of task description 110 as a ground truth to fine-tune the model, as discussed in more detail below.

[0037] In some respects, the fine-tuning system 125 (or another system) may deploy the revised machine learning model 130 for inference after fine-tuning. Generally, deploying the revised machine learning model 130 may include any operation or process of providing the revised machine learning model 130 for inference, including instructing or sending updated parameters to the inference system, instantiating or storing the updated model for local inference, etc.

[0038] In some respects, the fine-tuning system 125 can perform multiple iterations of fine-tuning. For example, after processing each task description 110 to generate fine-tuning paradigms and updating the machine learning model 105 using these paradigms (to generate a revised machine learning model 130), the fine-tuning system 125 can then repeat the process to further fine-tune the revised machine learning model 130. That is, the fine-tuning system 125 can use the revised machine learning model 130 to reprocess the task description 110 to generate new fine-tuning paradigms, and use these new paradigms to further fine-tune the revised machine learning model 130. In this way, the fine-tuning system 125 can use bootstrapping and procedural repair to significantly improve the predictive performance of the machine learning model, resulting in more accurate and reliable procedural synthesis.

[0039] An updated example workflow for improving procedural synthetic machine learning models.

[0040] Figure 2 An example workflow 200 for improving procedural synthetic machine learning models is described according to various aspects of this disclosure. In some aspects, workflow 200 is comprised of a fine-tuning system such as... Figure 1 The fine-tuning system 125 is used to perform this. That is, the depicted components (e.g., verification component 215 and training component 220) can be components of the fine-tuning system. Although depicted as discrete components for clarity of concept, the depicted components (and other components not illustrated) can be combined or distributed across any number of devices and components, and are generally implemented using hardware, software, or a combination of hardware and software.

[0041] In the illustrated workflow 200, the fine-tuning system accesses task description 110 and processes task description 110 using machine learning model 205 to generate program 210. In some aspects, as discussed above, task description 110 includes a natural language description of the programming task, such as: “Write a program that finds the first repeating character in the input string.” In some aspects, as discussed above, task description 110 further includes unit tests or assertions, such as specifying the input and expected output (e.g., indicating that the program should output “B” when the input is “ABCDBA”).

[0042] As discussed above, in some aspects, task description 110 may include one or more unit tests (e.g., Figure 1 The unit test 115 may be associated with one or more unit tests. In some respects, the fine-tuning system may retain at least one unit test as a hidden test. That is, the fine-tuning system may optionally provide one or more unit tests in the hints to the machine learning model 205, but may ensure that at least one unit test is kept hidden (e.g., for verifying the generated program).

[0043] In some respects, machine learning model 205 corresponds to a pre-trained model, such as Figure 1 The machine learning model 105. Program 210 typically includes computer program code generated by machine learning model 205 based on prompts (e.g., based on task description 110 and / or unit tests) to (attempt) solve the described programming task. Generally, program 210 may include code in any programming language and using any suitable format (e.g., any format that can be compiled and executed according to the specific language).

[0044] In the illustrated example, the generated program 210 is accessed by the verification component 215 for evaluation (also referred to as verification and / or validation). In some respects, as discussed above, the verification component 215 may validate the program 210 at least in part by determining whether the program 210 results in any errors. For example, the verification component 215 may determine whether any compilation errors occur when the verification component 215 (or another component) attempts to compile or interpret the program 210, and / or whether any runtime errors occur when the verification component 215 (or another component) attempts to process input data using the program 210. In some respects, if any errors are generated, the verification component 215 may determine that the program 210 failed to solve (also referred to as failing to satisfy) the programming task.

[0045] In the illustrated workflow 200, the verification component 215 may also access one or more unit tests 115 associated with the task description 110. The verification component 215 can use the unit tests to verify the program 210. For example, the verification component 215 can use the program 210 to process the input portion of a given unit test 115 to generate output, and the verification component 215 can compare the program output with the expected output portion of the unit test 115. In some aspects, if the verification component 215 determines that any unit test 115 fails (due to inaccurate output), the verification component 215 can determine that the program 210 has failed to solve the programming task.

[0046] In the illustrated workflow 200, the verification component 215 indicates the result of the verification to the training component 220. In some aspects, the verification component 215 may indicate that program 210 has failed without providing additional details or explanations. In some aspects, this may be referred to as a “simple feedback” implementation. In some aspects, the verification component 215 may provide additional details, such as an explanation of the error indicating the cause of the failure (e.g., indicating a specific error generated and / or indicating that one or more unit tests 115 failed). In some implementations, this may be referred to as “full feedback”. In some aspects, the training component 220 may additionally provide suggestions or more detailed input, if available. For example, if the interpreter and / or compiler is configured to provide more detailed suggestions (e.g., indicating a potential source of an error or suggesting a fix), the verification component 215 may include that detail in the feedback.

[0047] In some respects, training component 220 may take various actions based on feedback provided by validation component 215. For example, in the illustrated workflow 200, if training component 220 determines that program 210 has successfully solved the programming task (also known as fulfilling the task) (e.g., because no errors were generated and unit tests 115 were handled accurately), training component 220 may add program 210 as a fine-tuning paradigm to fine-tuning dataset 235. In some respects, to form a fine-tuning paradigm, training component 220 may use task description 110 (and in some respects, any unit tests 115 in the hints to machine learning model 205) as the input part of the paradigm, and training component 220 may use the generated program 210 as the target output. That is, training component 220 may use the output of machine learning model 205 itself, rather than using the real-valued solution (e.g., Figure 1 Solution 120) is used as a label.

[0048] In some respects, if training component 220 determines that program 210 has failed to successfully fulfill the programming task, training component 220 may determine whether to attempt to fix program 210. For example, in some respects, the fine-tuning system may use a defined or maximum number of fix iterations (e.g., allowing machine learning model 205 a maximum number of fix attempts). In some respects, the fine-tuning system may allow a single fix attempt. That is, if the first check of program 210 fails, training component 220 may prompt machine learning model 205 to fix the code once. If the revised program also fails, training component 220 may determine to use a ground truth solution, as discussed above and in more detail below.

[0049] In the illustrated workflow 200, if the training component 220 determines to attempt repair 210, the training component 220 provides feedback 225 as input to the machine learning model 205. In some aspects, as discussed above, the feedback may include simple feedback (e.g., text indicating failure, such as "This is incorrect. Please try again") and / or more complex feedback (e.g., text indicating a specific error that occurred). In some aspects, feedback 225 typically includes natural language text.

[0050] In some aspects, in order to use feedback 225 as input, training component 220 may additionally provide other data as input to machine learning model 205. For example, in some aspects, training component 220 may provide the original task description 110, feedback 225, and / or the original program 210 as input. In some aspects, training component 220 may cascade some or all of task description 110, program 210, and feedback 225, and use the cascaded text as input prompts for machine learning model 205. Using these prompts, machine learning model 205 may generate a revised program (which may be represented as program 210 in the illustrated example). As discussed above, the revised program may correspond to an updated version of the original program 210. Alternatively, the revised program may be a completely new program with newly drafted code.

[0051] In the illustrated workflow 200, the verification component 215 may similarly evaluate the revised program to attempt to validate or verify the revised code, as discussed above. The results of this verification may similarly be provided to the training component 220, as discussed above.

[0052] As shown in the figure, training component 220 can similarly evaluate the results of validation against the revised program to determine whether the revised program satisfies the programming task. If so, as discussed above, the revised program can be used as the true value for fine-tuning the model. For example, training component 220 can add the revised program as a fine-tuning paradigm to fine-tuning dataset 235. In some aspects, in order to form fine-tuning paradigms based on the revised program, training component 220 can use (revised) prompts (e.g., task description 110, unit tests, original program 210, and feedback 225) as input parts of the paradigm, and use the revised program as the target output. That is, training component 220 can use the same strings used to generate the revised program (e.g., cascaded task, original (failed) program, and feedback) as paradigm input, and use the revised (e.g., fixed) program as the target output.

[0053] In the illustrated example, if training component 220 determines that the revised program still fails validation, it may again determine whether to attempt to fix the code. In some respects, as discussed above, training component 220 may use a defined or maximum number of fix iterations (e.g., allowing one fix attempt). If this threshold has been reached, training component 220 may use solution 120 as the true value. In some respects, if training component 220 determines that the revised program still fails validation, but for one or more different reasons, the number of fix iterations may not be increased and / or may be reset.

[0054] In other words, training component 220 can add solution 120 as a fine-tuning paradigm to fine-tuning dataset 235. In some respects, to form a fine-tuning paradigm based on the failure of the fix, training component 220 can use (revised) hints (e.g., task description 110, unit tests, original program 210, and feedback 225) as input to the paradigm and use solution 120 as the target output. That is, training component 220 can use the same strings used to generate the revised program (e.g., the cascaded task, the original (failed) program, and the feedback) as paradigm input and use the actual ground truth solution as the target output.

[0055] In some respects, the illustrated workflow 200 can be executed for each task description 110 in a set of examples that can be used to fine-tune the machine learning model 205. Once each task description 110 has been processed to generate a corresponding fine-tuning example in the fine-tuning dataset 235, the fine-tuning system can use the fine-tuning dataset 235 to refine or update the parameters of the machine learning model 205.

[0056] Generally, the specific operations used to update the machine learning model 205 can vary depending on the specific architecture and implementation. For example, if the machine learning model 205 is implemented as a neural network, the fine-tuning system can use the machine learning model 205 to process the input portion of each paradigm in the fine-tuning dataset 235 to generate an output program (or, if a previously generated program is saved for fine-tuning, the program previously generated for the paradigm during workflow 200 can be accessed). The fine-tuning system can then compare the output portion of the paradigm (e.g., a correctly generated program 210, a correctly fixed program, or solution 120) with the generated program to generate a loss. This loss can then be used to update the model parameters, such as using backpropagation.

[0057] Generally speaking, the fine-tuning system can update the parameters of the machine learning model 205 using a single example from the fine-tuning dataset 235 (e.g., using stochastic gradient descent) or using batches of samples (e.g., using batch gradient descent).

[0058] As discussed above, using the fine-tuning dataset 235, the fine-tuning system can thereby generate a refined or updated version of the machine learning model 205 (e.g., Figure 1 The revised machine learning model 130. In some respects, the updated model can then be deployed for inference. In some respects, as discussed above, the fine-tuning system can optionally perform another round of fine-tuning. That is, the fine-tuning system can use the updated machine learning model instead of the original machine learning model 205 to repeat the workflow 200 for each task description 110.

[0059] Depending on the specific implementation, this fine-tuning workflow can be repeated any number of times. For example, a fine-tuning system can repeat workflow 200 a defined number of times (e.g., a fixed number of fine-tuning iterations) until the desired accuracy or performance of the updated machine learning model is achieved, until a defined amount of time and / or computational resources have been consumed during the fine-tuning, and so on.

[0060] Example methods for updating procedural synthetic machine learning models using bootstrapping and procedural repair.

[0061] Figure 3 This is a flowchart 300 depicting an example method for improving procedural synthetic machine learning models using bootstrapping and procedural repair, according to various aspects of this disclosure. In some aspects, method 300 is comprised of a fine-tuning system (such as...) Figure 1 The fine-tuning system 125) is executed. In some respects, method 300 is aimed at... Figure 2 Workflow 200 provides additional details.

[0062] At box 305, the system fine-tunes access to a set of task definitions (e.g., ...). Figure 1 and Figure 2 Task description 110). As discussed above, a task description typically describes or indicates a programming task, such as using natural language to describe the expected functionality of a program. In some aspects, as discussed above, each task definition may include one or more unit tests (e.g., Figure 1 and Figure 2 Unit tests 115), one or more of which can be used to verify the functionality of the code (e.g., verify that the code satisfies a programming task). In some aspects, each task definition may further include one or more solutions corresponding to program code that (allegedly) satisfies the programming task (e.g., Figure 1 and Figure 2 Solution 120). For example, the solution could be a manually written program to perform the task. Generally, the fine-tuning system can access the task definition from any suitable repository, including local storage, remote storage, publicly accessible paradigms (e.g., MBPP and / or APPS datasets), etc.

[0063] At box 310, the fine-tuning system selects one task definition from the task definitions. Generally, the fine-tuning system can use any suitable technique to select the task definition (including random or pseudo-random), since each task in the selection set will be evaluated during method 300.

[0064] At box 315, the fine-tuning system uses a machine learning model (e.g., Figure 2 Machine learning model 205) generation program (e.g., Figure 2 (Program 210). For example, as discussed above, the fine-tuning system can use a natural language task description as input to a machine learning model, prompting the model to generate a synthetic procedure that attempts to satisfy the task. In some aspects, as discussed above, the fine-tuning system can further use one or more unit tests of the task (if available) as further input (e.g., cascading task descriptions and unit tests, and using already cascaded text as input).

[0065] At box 320, the fine-tuning system determines whether the programming task is satisfied by the generated program. That is, the fine-tuning system can determine whether the generated program satisfies or performs the expected programming task. In some aspects, as discussed above, the fine-tuning system can determine whether the task is satisfied based on whether the generated program generates or causes any errors (e.g., compiler or interpreter errors, runtime errors, etc.). In other aspects, the fine-tuning system can additionally or alternatively determine whether the task is satisfied based on whether the generated program accurately or successfully executes any (hidden) unit tests associated with the task description but not provided as input to the model.

[0066] At box 320, if the fine-tuning system determines that the task is satisfied, method 300 continues to box 330, as discussed in more detail below. If the fine-tuning system determines that the task is not satisfied by the generated program, method 300 continues to box 325.

[0067] At box 325, the fine-tuning system determines whether to attempt to repair (failed) the program. Generally, depending on the specific implementation, the fine-tuning system may use various criteria to determine whether to repair the program. For example, in some aspects, the fine-tuning system may determine whether the maximum number of repair attempts has been performed (e.g., where the fine-tuning system attempts to repair the program once before moving to the next paradigm). As another (non-limiting) example, the fine-tuning system may determine whether computational resources are available for repairing the program. In some aspects, the fine-tuning system may evaluate the progress or improvement of the program repair. For example, the fine-tuning system may determine whether the most recently generated program is better than a previous version based on one or more metrics or key performance indicators (e.g., whether the revised version has fewer bugs than the original version, whether it accurately addresses more unit tests, etc.). In some such aspects, if the program has improved, the fine-tuning system may determine to attempt another round of repair. If the program has not improved (or has deteriorated), the fine-tuning system may determine not to attempt any further repairs. In some aspects, the fine-tuning system may undo the changes made in the last round of repair.

[0068] At box 325, if the system determines not to repair the program, then method 300 continues to box 330, as discussed in more detail below. If the fine-tuning system determines to repair the program (or attempts to repair the program), then method 300 continues to box 335.

[0069] At box 335, the fine-tuning system generates feedback based on the evaluation of the generated program. As discussed above, in some aspects, the feedback includes simple feedback, such as indicating that the program failed to meet the task (e.g., natural language text stating "Incorrect. Please try again") without providing additional context or explanation. In other aspects, as discussed above, the feedback may include more complex feedback (e.g., complete feedback), such as text indicating a specific error that occurred, text indicating that a unit test failed, etc.

[0070] At box 340, the fine-tuning system generates a revised procedure based on the feedback. For example, the fine-tuning system may use a machine learning model to process the feedback to generate the revised procedure. In some aspects, as discussed above, the fine-tuning system may cascade the feedback with other data to generate new hints for the model. For example, in some aspects, the fine-tuning system may cascade the original hint (e.g., a task description, which may include one or more unit tests), the original (or recently generated) procedure (e.g., generated at box 315 and / or at box 340 during a previous fix attempt), and the feedback. This cascaded string can then be used as input to a machine learning model to generate the revised procedure. Method 300 then returns to box 320 to verify the revised procedure.

[0071] Returning to box 330, after the fine-tuning system determines that the procedure is correct (at box 320) or that additional repair attempts should not be made (at box 325), the fine-tuning system adds the new paradigm to the fine-tuning dataset based on the selected task definition (e.g., Figure 2 The fine-tuned dataset 235). In some respects, as discussed above, the input portion of the new paradigm may correspond to the model input used to generate the most recent program (e.g., the task definition itself, or the task definition cascaded with incorrectly generated code and the resulting feedback). The output portion of the paradigm may similarly vary depending on whether the program satisfies the task. For example, if the program (or the revised program) performs the task accurately, the label or target output portion of the paradigm may include the generated program. If the program (or the revised program) fails to satisfy the task, the target output may correspond to the true value solution of the task (e.g., Figure 1 and Figure 2 Solution 120). See below for reference. Figure 4 A sample method for adding examples will be discussed in more detail.

[0072] The fine-tuning paradigm can then be stored in the fine-tuning dataset for later use when updating the model. Method 300 then proceeds to box 345, where the fine-tuning system determines whether at least one additional task remains in the set that has not yet been used to generate the fine-tuning paradigm during the current update iteration. If yes, method 300 returns to box 310 to select the next task definition. If no, method 300 proceeds to box 350. Although the illustrated example depicts a conceptually clear iterative process (e.g., selecting and processing each task sequentially), in some respects, the fine-tuning system can process some or all of the tasks in parallel.

[0073] At box 350, the fine-tuning system fine-tunes the machine learning model based on the fine-tuning dataset. For example, as discussed above, the fine-tuning system may determine or generate the model output based on the input portion of each paradigm (e.g., by processing strings using the model) and compare the generated output with the target output portion of the paradigm (e.g., accurately generated code or a ground truth procedure). The result of this comparison (e.g., the loss) can then be used to refine the model (e.g., using backpropagation) for each fine-tuning paradigm (e.g., using stochastic or batch gradient descent).

[0074] In the illustrated example, method 300 terminates after box 350. In some respects, as discussed above, the fine-tuning system can repeatedly perform the fine-tuning operation (e.g., execute method 300). For example, the fine-tuning system can use an updated version of the model (generated at box 350) as the (new) base model and execute method 300 again to generate another updated version. This fine-tuning can continue for any number of iterations, such as until one or more termination criteria are met (e.g., maximum number of iterations, desired model accuracy, etc.).

[0075] In some respects, after fine-tuning is complete (e.g., after processing all fine-tuning paradigms in the dataset during the final fine-tuning iteration), the fine-tuning system may deploy an updated machine learning model for inference. As discussed above, deploying the model may typically include deploying the model locally (e.g., for inference by the fine-tuning system) and / or deploying the model to one or more other systems (e.g., a dedicated inference system).

[0076] Example methods for generating fine-tuning data for procedural synthetic machine learning models

[0077] Figure 4 This is a flowchart depicting an example method 400 for generating fine-tuning data for a procedural synthetic machine learning model according to various aspects of this disclosure. In some aspects, method 400 is comprised of a fine-tuning system (such as...) Figure 1 The fine-tuning system 125) is executed. In some respects, method 400 is aimed at... Figure 3 Box 330 provides additional details.

[0078] At box 405, the fine-tuning system determines whether the first generated program for the task description is correct. That is, the fine-tuning system determines whether the machine learning model generated a correct program that satisfies the programming task during the first attempt (e.g., without using fixes). For example, the fine-tuning system may determine that... Figure 3 Does the program generated at box 315 satisfy the task? If so, method 400 continues to box 410.

[0079] At box 410, the fine-tuning system generates a fine-tuning record that includes prompts for generating the procedure and the procedure generated by the model. For example, as discussed above, each fine-tuning record may include an input portion (e.g., a text string that will be used as input to the model) and an output portion (e.g., labels of the true values ​​that will be used as input during fine-tuning). At box 410, the input portion may correspond to the original task description used to generate the procedure (and in some cases, unit tests). The output portion includes the generated procedure itself, allowing the fine-tuning system to use bootstrapping (instead of a manually written solution) during fine-tuning.

[0080] Returning to box 405, if the fine-tuning system determines that the first procedure is incorrect, method 400 continues to box 415, where the fine-tuning system cascades the original prompts (e.g., task description and / or unit tests), the first (failed) procedure, and feedback generated based on the failed procedure. Although the illustrated example depicts cascading the first procedure, in some respects, the fine-tuning system may cascade the most recent or ultimately failed procedure. That is, if multiple repair attempts are used, the fine-tuning system may cascade the prompts and feedback with the last or most recent attempt (rather than the first attempt) when generating the procedure.

[0081] At box 420, the fine-tuning system determines whether the (final) revised procedure is correct. That is, the fine-tuning system determines, based on feedback, whether the machine learning model can generate the correct procedure, as discussed above. If so, method 400 continues to box 425, where the fine-tuning system generates a fine-tuned record that includes the concatenated text and the revised procedure (generated at box 415). For example, as discussed above, the concatenated text may correspond to the input portion of the record, and the revised procedure may correspond to the target output.

[0082] Returning to box 420, if the fine-tuning system determines that the (finally) revised program also fails to meet the task, method 400 continues to box 430. At box 430, the fine-tuning system generates a fine-tuning record that includes (generated at box 415) cascaded text and a real-value program for the task. For example, as discussed above, the cascaded text may correspond to the input portion of the record, and the real-value program may correspond to the target output. In some respects, as discussed above, the real-value program may be a manually written solution for the programming task.

[0083] Although the illustrated examples depict the generation of a single fine-tuning record for each task description (e.g., at boxes 410, 425, or 430, depending on whether the program or revised program successfully solved the task), in some respects, the fine-tuning system may generate multiple records for some (or all) of the tasks. For example, in some respects, if the fine-tuning system determines that a first program or revised program for a given task has been successfully validated, the fine-tuning system may generate a first record that includes the generated program as the target (e.g., at boxes 410 or 425), and a second record that includes the original (e.g., manually written) true value as the target.

[0084] Using method 400, the fine-tuning system can dynamically generate fine-tuning records based on the model's performance itself. This allows the fine-tuning system to use bootstrapping and repair to generate improved and more diverse fine-tuning data, while retaining the use of the true value labels even when the model fails to generate accurate output. This can significantly improve the accuracy of updated models, as discussed above.

[0085] Example methods for updating machine learning models for procedural synthesis

[0086] Figure 5 This is a flowchart depicting an example method 500 for updating a machine learning model for use in procedural synthesis, according to various aspects of this disclosure. In some aspects, method 500 is comprised of a fine-tuning system (such as...) Figure 1 The fine-tuning system 125) performs this. In some aspects, method 500 is performed by the validation component 215, the training component 220, and / or using the machine learning model 205, as described above. Figure 2 As shown. In some aspects, method 500 is handled by a processing system (such as...) Figure 6 The processing system executes this. In some respects, method 500 is aimed at... Figure 3 Method 300 and / or Figure 4 Method 400 provides additional details.

[0087] At box 505, access indicates a first input for the first programming task. In some aspects, the first input includes a natural language description of the first programming task. In some aspects, the first input further includes a first unit test to verify the accuracy of the program for the first programming task.

[0088] At box 510, a first generated program is generated based on processing the first input using a trained machine learning model.

[0089] At box 515, a first feedback is generated in response to determining that the first generated program has failed to satisfy the first programming task.

[0090] In some respects, the first feedback includes an indication that the first generated program is incorrect. In other respects, the first feedback further includes one or more erroneous interpretations of the first generated program.

[0091] In some respects, determining that the first generated program fails to meet the first programming task includes at least one of the following: (i) determining that the first generated program causes one or more compilation errors, (ii) determining that the first generated program causes one or more runtime errors, or (iii) determining that the first generated program fails to accurately handle one or more unit tests associated with the first programming task.

[0092] At box 520, the first revised procedure is generated based on processing the first feedback using a trained machine learning model.

[0093] In some respects, generating the first revised procedure includes: generating the cascaded input based on the cascaded first input, the first generated procedure, and the first feedback, and processing the cascaded input using a trained machine learning model.

[0094] At box 525, in response to determining that the first revised procedure satisfies the first programming task, one or more parameters of the trained machine learning model are updated based on the first revised procedure.

[0095] In some aspects, method 500 further includes: accessing a second input indicating a second programming task, generating a second generated program based on processing the second input using a trained machine learning model, and, in response to determining that the second generated program satisfies the second programming task, further updating one or more parameters of the trained machine learning model based on the second generated program.

[0096] In some aspects, method 500 further includes: accessing a second input indicative of a second programming task, and generating a second generated program based on processing the second input using a trained machine learning model. In some aspects, method 500 further includes: in response to determining that the second generated program fails to satisfy the second programming task, generating second feedback based on the determination that the second generated program fails to satisfy the second programming task, and generating a second revised program based on processing the second feedback using a trained machine learning model. In some aspects, method 500 further includes: in response to determining that the second revised program fails to satisfy the second programming task, further updating one or more parameters of the trained machine learning model based on a true value program for the second programming task.

[0097] Example processing system for training machine learning models

[0098] In some respects, refer to Figures 1 to 5 The described workflows, techniques, and methods can be implemented on one or more devices or systems. Figure 6 An example processing system 600 is depicted, which is configured to perform various aspects of this disclosure, including, for example, regarding... Figures 1 to 5 The techniques and methods described. In some aspects, the processing system 600 may correspond to a fine-tuning system, such as... Figure 1 The fine-tuning system 125. In general, the processing system 600 can correspond to any computing system for updating or fine-tuning machine learning models. Although depicted as a single system for clarity of concept, in some respects, as discussed above, the operations described below with respect to the processing system 600 can be distributed across any number of devices or systems.

[0099] Processing system 600 includes a central processing unit (CPU) 602, which in some examples may be a multi-core CPU. Instructions executed at CPU 602 may be loaded, for example, from program memory associated with CPU 602 or from a memory partition (e.g., a partition of memory 624).

[0100] The processing system 600 also includes additional processing components tailored for specific functions, such as a graphics processing unit (GPU) 604, a digital signal processor (DSP) 606, a neural processing unit (NPU) 608, a multimedia component 610 (e.g., a multimedia processing unit), and a wireless connectivity component 612.

[0101] NPUs (such as the NPU 608) are typically configured to implement dedicated circuitry for implementing control and arithmetic logic for executing machine learning algorithms, such as those for processing artificial neural networks (ANNs), deep neural networks (DNNs), random forests (RFs), etc. NPUs are sometimes alternatively referred to as neural signal processors (NSPs), tensor processing units (TPUs), neural network processors (NNPs), intelligent processing units (IPUs), vision processing units (VPUs), or graphics processing units.

[0102] NPUs such as the NPU 608 are configured to accelerate the execution of common machine learning tasks, such as image classification, machine translation, object detection, and various other predictive models. In some examples, multiple NPUs may be instantiated on a single chip, such as a system-on-a-chip (SoC), while in other examples, an NPU may be part of a dedicated neural network accelerator.

[0103] An NPU can be optimized for either training or inference, or in some cases configured to balance performance between the two. For an NPU capable of performing both training and inference, these two tasks can generally still be performed independently.

[0104] NPUs designed to accelerate training are typically configured to speed up the optimization of new models. This involves taking an existing dataset (usually labeled or sublabeled), iterating over the dataset, and subsequently tuning model parameters (such as weights and biases) to improve model performance—a highly computationally intensive operation. Generally, optimization based on incorrect predictions involves backtracking through the layers of the model and determining gradients to reduce prediction errors.

[0105] NPUs designed to accelerate inference are typically configured to operate on the full model. Therefore, such NPUs can be configured to take new data segments as input and rapidly process those segments through a trained model to generate model outputs (e.g., inference).

[0106] In some implementations, the NPU 608 is part of one or more of the CPU 602, GPU 604, and / or DSP 606.

[0107] In some examples, the wireless connectivity component 612 may include sub-components for, for example, third-generation (3G) connectivity, fourth-generation (4G) connectivity (e.g., 4G Long Term Evolution (LTE)), fifth-generation connectivity (e.g., 5G or New Radio (NR)), Wi-Fi connectivity, Bluetooth connectivity, and / or other wireless data transmission standards. The wireless connectivity component 612 is further coupled to one or more antennas 614.

[0108] The processing system 600 may also include one or more sensor processing units 616 associated with any type of sensor, one or more image signal processors (ISPs) 618 associated with any type of image sensor, and / or a navigation processor 620, which may include satellite-based positioning system components (e.g., GPS or GLONASS) and inertial positioning system components.

[0109] The processing system 600 may also include one or more input and / or output devices 622, such as screens, touch-sensitive surfaces (including touch-sensitive displays), physical buttons, speakers, microphones, etc.

[0110] In some examples, one or more processors in the processing system 600 may be based on the ARM or RISC-V instruction set.

[0111] The processing system 600 also includes a memory 624, which represents one or more static and / or dynamic memories, such as dynamic random access memory, flash-based static memory, etc. In this example, the memory 624 includes computer-executable components that can be executed by one or more of the aforementioned processors of the processing system 600.

[0112] Specifically, in this example, memory 624 includes a machine learning component 624A, a validation component 624B, and a training component 624C. Memory 624 further includes model parameters 624D for one or more models or portions thereof (e.g., machine learning model parameters such as...). Figure 1 Machine learning model 105 Figure 1 130 revised machine learning models and / or Figure 2 The parameters of the machine learning model 205). The memory 624 also includes a fine-tuning dataset 624E (which can correspond to the parameters of the machine learning model 205). Figure 2 The fine-tuned dataset 235). Although not included in the illustrated example, in some respects, memory 624 may also include other data, such as training data (e.g., task description 110, unit tests 115, and / or...). Figure 1 and Figure 2 Solution 120). Although for the sake of conceptual clarity in Figure 6The components are depicted as discrete components, but in various respects, the illustrated components (and other components not depicted) may be implemented together or separately.

[0113] The processing system 600 further includes machine learning circuitry 626, verification circuitry 627, and training circuitry 628. The depicted circuitry and other circuitry not depicted can be configured to perform various aspects of the techniques described herein.

[0114] For example, machine learning component 624A and / or machine learning circuit 626 can be used to generate a synthesis procedure using a machine learning model, as discussed above. For example, machine learning component 624A and / or machine learning circuit 626 can use a machine learning model (e.g., Figure 1 Machine learning model 105 and / or Figure 2 The machine learning model 205 processes input text, such as task descriptions (e.g., Figure 1 and Figure 2 Task description 110 and / or unit test 115), to generate a synthesis program (e.g., Figure 2 (Program 210). In some respects, the machine learning component 624A and / or the machine learning circuit 626 can similarly generate updated prompts (e.g., via cascaded task descriptions (such as...). Figure 1 Task description 110), incorrect procedures (e.g., Figure 2 Procedure 210) and / or feedback (e.g., Figure 2 (Feedback 225). The machine learning component 624A and / or machine learning circuit 626 can then use these updated hints to generate a revised model, as discussed above.

[0115] Verification component 624B and / or verification circuit 627 (which may correspond to Figure 2 Verification component 215 can be used to verify whether the generated program (including revised or repaired programs) meets the programming task, as discussed above. For example, verification component 624B and / or verification circuit 627 can determine whether the generated program causes any compilation errors when compiled or interpreted, and / or whether it causes any runtime errors when executed. As another example, verification component 624B and / or verification circuit 627 can determine whether the generated program operates accurately or correctly, such as whether the program generates the correct output based on a given input (e.g., using unit testing).

[0116] Training component 624C and / or training circuit 628 (which may correspond to) Figure 2The training component 220 can be used to generate fine-tuning data (e.g., fine-tuning dataset 624E) and to use that fine-tuning data to update the parameters of the machine learning model, as discussed above. For example, the training component 624C and / or the training circuit 628 can generate paradigms that include generated program code (when the model generates a program that satisfies the input programming task) and / or ground truth code (when the model fails to generate a program that satisfies the task), and these paradigms can be used to refine the model to improve its accuracy.

[0117] Although for the sake of clarity Figure 6 While depicted as separate components and circuits, the machine learning circuit 626, the verification circuit 627, and the training circuit 628 may be implemented jointly or separately in other processing devices of the processing system 600, such as within the CPU 602, GPU 604, DSP 606, NPU 608, etc.

[0118] Generally speaking, the processing system 600 and / or its components can be configured to perform the methods described herein.

[0119] It is worth noting that, in other aspects, such as when the processing system 600 is a server computer, components of the processing system 600 may be omitted. For example, in other aspects, the multimedia component 610, the wireless connectivity component 612, the sensor processing unit 616, the ISP 618, and / or the navigation processor 620 may be omitted. Furthermore, various aspects of the processing system 600 may be distributed among multiple devices.

[0120] Example Terms

[0121] Specific implementation examples are described in the following numbered clauses: Clause 1: A method comprising: accessing a second input indicative of a second programming task; generating a second generated program based on processing the second input using the trained machine learning model; in response to determining that the second generated program fails to satisfy the second programming task: generating second feedback based on the determination that the second generated program fails to satisfy the second programming task; generating a second revised program based on processing the second feedback using the trained machine learning model; and in response to determining that the second revised program fails to satisfy the second programming task, further updating one or more parameters of the trained machine learning model based on a true value program for the second programming task.

[0122] Clause 2: The method according to Clause 1 further includes: accessing a second input indicating a second programming task; generating a second generated program based on processing the second input using the trained machine learning model; and in response to determining that the second generated program satisfies the second programming task, further updating one or more parameters of the trained machine learning model based on the second generated program.

[0123] Clause 3: The method according to any one of Clauses 1 to 2, the method further comprising: accessing a second input indicating a second programming task; generating a second generated program based on processing the second input using the trained machine learning model; in response to determining that the second generated program fails to satisfy the second programming task: generating second feedback based on the determination that the second generated program fails to satisfy the second programming task; generating a second revised program based on processing the second feedback using the trained machine learning model; and in response to determining that the second revised program fails to satisfy the second programming task, further updating one or more parameters of the trained machine learning model based on a true value program for the second programming task.

[0124] Clause 4: The method according to any one of Clauses 1 to 3, wherein the first input comprises a natural language description of the first programming task.

[0125] Clause 5: The method according to Clause 4, wherein the first input further includes unit testing to verify the accuracy of the program with respect to the first programming task.

[0126] Clause 6: The method according to any one of Clauses 1 to 5, wherein determining that the first generated program fails to satisfy the first programming task includes at least one of the following: (i) determining that the first generated program causes one or more compilation errors, (ii) determining that the first generated program causes one or more runtime errors, or (iii) determining that the first generated program fails to accurately handle one or more unit tests associated with the first programming task.

[0127] Clause 7: The method according to any one of Clauses 1 to 6, wherein the first feedback includes an indication that the first generated program is incorrect.

[0128] Clause 8: The method described in Clause 7, wherein the first feedback further includes one or more error interpretations for the first generated program.

[0129] Clause 9: The method according to any one of Clauses 1 to 8, wherein generating the first revised procedure comprises: generating a cascaded input based on cascading the first input, the first generated procedure, and the first feedback; and processing the cascaded input using the trained machine learning model.

[0130] Clause 10: A processing system comprising: a memory including computer-executable instructions; and one or more processors configured to execute the computer-executable instructions and cause the processing system to perform a method according to any one of Clauses 1 to 9.

[0131] Clause 11: A processing system comprising components for performing the method according to any one of Clauses 1 to 9.

[0132] Clause 12: A non-transitory computer-readable medium comprising computer-executable instructions that, when executed by one or more processors of a processing system, cause the processing system to perform the method according to any one of Clauses 1 to 9.

[0133] Clause 13: A non-transitory computer-readable medium encoding logic that, when executed by a processing system, causes the processing system to perform the method according to any one of Clauses 1 to 9.

[0134] Clause 14: An apparatus comprising logic circuitry configured to perform the method according to any one of Clauses 1 to 9.

[0135] Clause 15: A computer program product embodied on a computer-readable storage medium, said computer-readable storage medium including code for performing the method according to any one of Clauses 1 to 9.

[0136] Additional Notes

[0137] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. The examples discussed herein do not limit the scope, applicability, or aspects set forth in the claims. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. For example, the function and arrangement of the elements discussed may be changed without departing from the scope of this disclosure. Various processes or components may be omitted, substituted, or added as appropriate in various examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined in some other examples. For example, any number of aspects set forth herein may be used to implement an apparatus or practice. Moreover, the scope of this disclosure is intended to cover such apparatuses or methods practiced using other structures, functionalities, or structures and functionalities that complement or replace the various aspects of this disclosure set forth herein. It should be understood that any aspect of the disclosure herein may be embodied by one or more elements of these claims.

[0138] As used herein, the term “exemplary” means “serving as an example, instance, or illustration.” Any aspect described as “exemplary” in this document is not necessarily to be construed as preferred or superior to other aspects.

[0139] As used in this article, the phrase “at least one of” in a list of items refers to any combination of these entries, including a single member. As an example, “at least one of a, b, or c” is intended to cover: a, b, c, ab, ac, bc, and abc, as well as any combination with multiple identical elements (e.g., aa, aaa, aab, aac, abb, acc, bb, bbb, bbb, bbc, cc, and ccc, or any other ordering of a, b, and c).

[0140] As used herein, the term "determine" encompasses a wide variety of actions. For example, "determine" can include calculation, computation, processing, derivation, research, searching (e.g., looking in a table, database, or other data structure), ascertainment, and similar actions. Additionally, "determine" can include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), etc. Furthermore, "determine" can include parsing, selecting, picking, building, etc.

[0141] The methods disclosed herein include one or more steps or actions for implementing the methods. The steps and / or actions of the methods may be interchanged without departing from the scope of the claims. In other words, unless a specific order of steps or actions is specified, the order and / or use of a particular step and / or action may be modified without departing from the scope of the claims. Furthermore, the various operations of the methods described above may be performed by any suitable component capable of performing the corresponding function. This component may include various hardware and / or software components and / or modules, including but not limited to circuits, application-specific integrated circuits (ASICs), or processors. Generally, where the operations illustrated in the figures are present, those operations may have corresponding components with similar numbering plus functional components.

[0142] The following claims are not intended to be limited to the aspects shown herein, but should be given the full scope consistent with the language of the claims. Within the claims, unless specifically stated otherwise, reference to the singular form of an element is not intended to mean “one and only one,” but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more. No element of any claim should be interpreted in accordance with 35 USC § 112(f) unless the element is expressly referred to by the phrase “for a component of,” or, in the case of a method claim, by the phrase “for a step of.” All structural and functional equivalents of the elements throughout the various aspects described herein that are known to a person of ordinary skill in the art, or will be known later, are expressly incorporated herein by reference and are intended to be covered by the claims. Furthermore, nothing disclosed herein is intended to be offered to the public, whether or not such disclosure is explicitly stated in the claims.

Claims

1. A processing system, the processing system comprising: One or more memories, the one or more memories including processor-executable instructions; and One or more processors, the one or more processors being configured to execute processor-executable instructions and enable the processing system: Access indicates the first input of the first programming task; Generate a first generated program, wherein, in order to generate the first generated program, the one or more processors are configured to execute processor-executable instructions to cause the processing system to process the first input using a trained machine learning model; It is determined that the first generated program failed to fulfill the first programming task; A first feedback is generated based on the determination that the first generated program failed to meet the first programming task; Generate a first revised program, wherein, in order to generate the first revised program, the one or more processors are configured to execute processor-executable instructions to cause the processing system to process the first feedback using the trained machine learning model; Determine that the first revised program satisfies the first programming task; as well as In response to the determination that the first revised program satisfies the first programming task, one or more parameters of the trained machine learning model are updated based on the first revised program.

2. The processing system of claim 1, wherein the one or more processors are configured to further execute processor-executable instructions to cause the processing system to: Access the second input indicating the second programming task; To generate a second generated program, wherein, in order to generate the second generated program, the one or more processors are configured to execute processor-executable instructions to cause the processing system to process the second input using the trained machine learning model; Determine that the second generated program satisfies the second programming task; as well as In response to the determination that the second generated program satisfies the second programming task, one or more parameters of the trained machine learning model are further updated based on the second generated program.

3. The processing system of claim 1, wherein the one or more processors are configured to further execute processor-executable instructions to cause the processing system to: Access the second input indicating the second programming task; To generate a second generated program, wherein, in order to generate the second generated program, the one or more processors are configured to execute processor-executable instructions to cause the processing system to process the second input using the trained machine learning model; It was determined that the second generated program failed to fulfill the second programming task; A second feedback is generated based on the determination that the second generated program fails to meet the second programming task; Generate a second revised program, wherein, in order to generate the second revised program, the one or more processors are configured to execute processor-executable instructions to cause the processing system to process the second feedback using the trained machine learning model; It was determined that the second revised program failed to meet the second programming task; as well as In response to the second revised procedure failing to meet the determination of the second programming task, one or more parameters of the trained machine learning model are further updated based on the true value procedure for the second programming task.

4. The processing system according to claim 1, wherein the first input includes a natural language description of the first programming task.

5. The processing system of claim 4, wherein the first input further includes unit tests to verify the accuracy of the program with respect to the first programming task.

6. The processing system of claim 1, wherein, in order to determine that the first generated program fails to satisfy the first programming task, the one or more processors are configured to execute processor-executable instructions to cause the processing system to: (i) Determine that the first generated program causes one or more compilation errors. (ii) Determine that the first generated program causes one or more runtime errors, or (iii) Determine that the first generated program fails to accurately handle one or more unit tests associated with the first programming task.

7. The processing system of claim 1, wherein the first feedback includes an indication that the first generated program is incorrect.

8. The processing system of claim 7, wherein the first feedback further includes one or more error explanations for the first generated program.

9. The processing system of claim 1, wherein, in order to generate the first revised program, the one or more processors are configured to execute processor-executable instructions to cause the processing system to: Generate a cascaded input, wherein, in order to generate the cascaded input, the one or more processors are configured to execute processor-executable instructions to cause the processing system to cascade the first input, the first generated program, and the first feedback; and The trained machine learning model is used to process the cascaded inputs.

10. A processor-implemented method, the processor-implemented method comprising: Access indicates the first input of the first programming task; The first generated program is generated by processing the first input using a trained machine learning model; In response to determining that the first generated program failed to fulfill the first programming task: A first feedback is generated based on the determination that the first generated program failed to meet the first programming task; as well as The first revised program is generated based on processing the first feedback using the trained machine learning model. as well as In response to determining that the first revised program satisfies the first programming task, one or more parameters of the trained machine learning model are updated based on the first revised program.

11. The processor-implemented method according to claim 10, further comprising: Access the second input indicating the second programming task; The second generated program is generated by processing the second input using the trained machine learning model. as well as In response to determining that the second generated program satisfies the second programming task, one or more parameters of the trained machine learning model are further updated based on the second generated program.

12. The processor-implemented method according to claim 10, further comprising: Access the second input indicating the second programming task; The second generated program is generated by processing the second input using the trained machine learning model. In response to determining that the second generated program failed to fulfill the second programming task: A second feedback is generated based on the determination that the second generated program fails to meet the second programming task; as well as The second revised program is generated based on processing the second feedback using the trained machine learning model. as well as In response to determining that the second revised procedure fails to meet the second programming task, one or more parameters of the trained machine learning model are further updated based on the true value procedure for the second programming task.

13. The processor-implemented method of claim 10, wherein the first input comprises a natural language description of the first programming task.

14. The processor-implemented method of claim 13, wherein the first input further includes unit testing to verify the accuracy of the program with respect to the first programming task.

15. The processor-implemented method of claim 10, wherein determining that the first generated program fails to satisfy the first programming task includes at least one of the following: (i) Determine that the first generated program causes one or more compilation errors. (ii) Determine that the first generated program causes one or more runtime errors, or (iii) Determine that the first generated program fails to accurately handle one or more unit tests associated with the first programming task.

16. The processor-implemented method of claim 10, wherein the first feedback includes an indication that the first generated program is incorrect.

17. The processor-implemented method of claim 16, wherein the first feedback further comprises one or more error interpretations for the first generated program.

18. The processor-implemented method of claim 10, wherein generating the first revised program comprises: The cascaded input is generated based on the first input, the first generated program, and the first feedback. as well as The trained machine learning model is used to process the cascaded inputs.

19. One or more non-transitory computer-readable media, the one or more non-transitory computer-readable media comprising computer-executable instructions, which, when executed by one or more processors of one or more processing systems, cause the one or more processing systems to: Access indicates the first input of the first programming task; Generate a first generated program, wherein, in order to generate the first generated program, the computer-executable instructions, when executed by the one or more processors, cause the one or more processing systems to process the first input using a trained machine learning model; It is determined that the first generated program failed to fulfill the first programming task; A first feedback is generated based on the determination that the first generated program failed to meet the first programming task; Generate a first revised program, wherein, in order to generate the first revised program, the computer-executable instructions, when executed by the one or more processors, cause the one or more processing systems to use the trained machine learning model to process the first feedback; Determine that the first revised program satisfies the first programming task; as well as In response to the determination that the first revised program satisfies the first programming task, one or more parameters of the trained machine learning model are updated based on the first revised program.

20. The one or more non-transitory computer-readable media of claim 19, wherein the computer-executable instructions, when executed by the one or more processors, further cause the one or more processing systems to: Access the second input indicating the second programming task; To generate a second generated program, wherein, in order to generate the second generated program, the computer-executable instructions, when executed by the one or more processors, cause the one or more processing systems to use the trained machine learning model to process the second input; Determine that the second generated program satisfies the second programming task; as well as In response to the determination that the second generated program satisfies the second programming task, one or more parameters of the trained machine learning model are further updated based on the second generated program.

21. The one or more non-transitory computer-readable media of claim 19, wherein the computer-executable instructions, when executed by the one or more processors, further cause the one or more processing systems to: Access the second input indicating the second programming task; To generate a second generated program, wherein, in order to generate the second generated program, the computer-executable instructions, when executed by the one or more processors, cause the one or more processing systems to use the trained machine learning model to process the second input; It was determined that the second generated program failed to fulfill the second programming task; A second feedback is generated based on the determination that the second generated program fails to meet the second programming task; as well as Generate a second revised program, wherein, in order to generate the second revised program, the computer-executable instructions, when executed by the one or more processors, cause the one or more processing systems to use the trained machine learning model to process the second feedback; It was determined that the second revised program failed to meet the second programming task; as well as In response to the second revised procedure failing to meet the determination of the second programming task, one or more parameters of the trained machine learning model are further updated based on the true value procedure for the second programming task.

22. One or more non-transitory computer-readable media according to claim 19, wherein the first input comprises a natural language description of the first programming task.

23. One or more non-transitory computer-readable media according to claim 22, wherein the first input further includes unit tests to verify the accuracy of the program with respect to the first programming task.

24. The one or more non-transitory computer-readable media of claim 19, wherein, in order to determine that the first generated program has failed to satisfy the first programming task, the computer-executable instructions, when executed by the one or more processors, cause the one or more processing systems to: (i) Determine that the first generated program causes one or more compilation errors. (ii) Determine that the first generated program causes one or more runtime errors, or (iii) Determine that the first generated program fails to accurately handle one or more unit tests associated with the first programming task.

25. One or more non-transitory computer-readable media according to claim 19, wherein the first feedback includes an indication that the first generated program is incorrect.

26. The one or more non-transitory computer-readable media of claim 25, wherein the first feedback further comprises one or more error interpretations for the first generated program.

27. The one or more non-transitory computer-readable media of claim 19, wherein generating the first revised program comprises, when executed by the one or more processors, causing the one or more processing systems to: Generate a cascaded input, wherein, in order to generate the cascaded input, the computer-executable instructions, when executed by the one or more processors, cause the one or more processing systems to cascade the first input, the first generated program, and the first feedback; and The trained machine learning model is used to process the cascaded inputs.

28. A processing system, the processing system comprising: A component used to access the first input indicating the first programming task; A component for generating a first generated program based on processing the first input using a trained machine learning model; A component used to determine that the first generated program failed to meet the first programming task; A component for generating first feedback based on the determination that the first generated program failed to meet the first programming task; and A component for generating a first revised program based on processing the first feedback using the trained machine learning model; A component for determining whether the first revised program satisfies the first programming task; and The component is used to update one or more parameters of the trained machine learning model based on the first revised program in response to the determination that the first revised program satisfies the first programming task.

29. The processing system according to claim 28, further comprising: A component for accessing a second input that indicates a second programming task; A component for generating a second generated program based on processing the second input using the trained machine learning model; Components used to determine whether the second generated program satisfies the second programming task; and The component is used to: in response to the determination that the second generated program satisfies the second programming task, further update one or more parameters of the trained machine learning model based on the second generated program.

30. The processing system according to claim 28, wherein the processing system further comprises: A component for accessing a second input that indicates a second programming task; A component for generating a second generated program based on processing the second input using the trained machine learning model; Components used to determine that the second generated program fails to meet the second programming task; A component for generating second feedback based on the determination that the second generated program failed to meet the second programming task; A component for generating a second revised program based on processing the second feedback using the trained machine learning model; Components used to determine if the second revised program fails to meet the second programming task; and The component is used to: in response to the determination that the second revised program fails to meet the second programming task, further update one or more parameters of the trained machine learning model based on the true value program for the second programming task.

Images