Neural Code Generation with Candidate Filtering and Clustering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generating computer code from natural language descriptions is challenging due to the vast and sparse structured space of possible programs, where small edits can drastically change behavior, and evaluating partial or incorrect programs is difficult, especially in general-purpose programming languages like C++ or Python.
Innovation Solution
A system using generative neural networks, such as Transformer models, to generate a large set of program samples, followed by filtering and clustering, leveraging metadata conditioning and parallel processing on hardware like CPUs, GPUs, and TPUs to efficiently produce high-quality computer programs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If generative neural networks are used to generate computer programs from natural language descriptions, then the ability to generate diverse program candidates is improved, but the complexity of the system increases due to the need for filtering and clustering mechanisms
Solution Approach 1:
The system segments the code generation process into distinct phases: (1) generating diverse program candidates using generative neural networks, (2) filtering candidates based on task requirements, and (3) clustering similar programs. This segmentation allows each component to focus on specific functions, managing overall system complexity while maintaining diversity generation capability.
Solution Approach 2:
The patent introduces intermediate representations and mechanisms between the neural network and final code selection. Filtering mechanisms and clustering algorithms serve as intermediaries that process the raw diverse candidates into refined selections, reducing the direct complexity burden on the generative model while maintaining versatility.
2Manufacturing precision
If a large set of program samples is generated using generative neural networks, then the quality and diversity of candidates is improved, but the computational resources and time required for generation and evaluation increase
Solution Approach 1:
The system generates a large set of program samples (excessive action) to ensure sufficient quality and diversity, then applies filtering and clustering to select the optimal candidates. This approach accepts temporary resource expenditure during generation to achieve superior final output quality that would be impossible with fewer samples.
Solution Approach 2:
The generative neural network creates multiple copies (candidates) of programs from a single natural language description. By generating and evaluating multiple copies rather than attempting to generate the perfect program directly, the system improves quality through statistical advantage while managing computational resources through efficient filtering of inferior copies.
3Adaptability or versatility
If code generation is performed in a general-purpose programming language, then the versatility of the system is improved, but the difficulty of evaluating partial or incorrect programs increases
Solution Approach 1:
The patent introduces filtering mechanisms as intermediaries between code generation and evaluation. These filters process generated code before full evaluation, helping to identify and eliminate obviously incorrect or incomplete programs, thereby reducing the difficulty of detecting problematic code in general-purpose languages.
Solution Approach 2:
The system implements feedback loops where evaluation results feed back into the generation process. By analyzing which generated programs succeed or fail, the system can adjust future generation attempts, making evaluation of partial or incorrect programs more effective through iterative refinement rather than one-time judgment.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating computer code using neural networks. One of the methods includes receiving description data describing a computer programming task; receiving a first set of inputs for the computer programming task; generating a plurality of candidate computer programs by sampling a plurality of output sequences from a set of one or more generative neural networks; for each candidate computer program in a subset of the candidate computer programs and for each input in the first set: executing the candidate computer program on the input to generate an output; and selecting, from the candidate computer programs, one or more computer programs as synthesized computer programs for performing the computer programming task based at least in part on the outputs generated by executing the candidate computer programs in the subset on the inputs in the first set of inputs.