Code Synthesis Parameter Learning With Metadata-Filtered Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current state-of-the-art techniques for training language models for code synthesis focus on the quantity of training samples rather than the quality, resulting in less generalized and less accurate models due to the absence of natural language text in code snippets.
Innovation Solution
The approach involves generating a filtered dataset by extracting metadata from code snippets, such as docstrings and comments, to train a language model, and pruning code snippets without natural language text, thereby enhancing model generalization and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If training data quantity is increased, then model coverage is improved, but model accuracy and generalization deteriorate due to inclusion of low-quality code snippets without natural language text
Solution Approach 1:
The patent extracts and removes low-quality code snippets that lack natural language text (docstrings, comments) from the training dataset. By filtering out these problematic samples through quality assessment metrics, the training data quality is improved while maintaining sufficient quantity for comprehensive code coverage.
Solution Approach 2:
The patent applies different quality standards to different portions of the training data. High-quality code snippets with natural language text are retained and weighted more heavily, while low-quality snippets are filtered or down-weighted. This local differentiation of quality allows the model to learn from the most informative samples.
2Adaptability or versatility
If natural language text is added to code snippets, then model generalization is improved, but data processing complexity increases
Solution Approach 1:
The patent performs preliminary extraction of natural language text (docstrings, comments) from code snippets during the data preparation phase. By pre-processing and attaching these text descriptions to code snippets before training, the model receives enriched input that improves generalization without requiring complex processing during training.
Solution Approach 2:
The patent introduces natural language text as an intermediary between the code snippet and the model. This text layer (docstrings, comments) serves as a bridge that provides semantic meaning and context, enabling the model to better understand and generalize from code patterns without directly analyzing raw code structure.
Data Source
AI summary
According to an aspect of an embodiment, operations for deep parameter learning for code synthesis are provided. The operations may include receiving a source code file and generating an abstract syntax tree (AST). The operations may further include determining a set of classes, and functions/procedures from the computer-executable code and extracting metadata associated to each component. The operations may further include selecting a subset of functions for which descriptions in the extracted metadata satisfy filtering criteria and updating the computer-executable code by filtering lines of code (LoCs) corresponding to the subset of functions/procedures. The operations may further include generating a dataset of code features and respective metadata features that includes a deep connection between parameters and its usage based on the updated computer-executable code and the metadata generation task. The operations may further include training a language model on a sequence-to-sequence generation task, based on the dataset.