Grammar-Compliant Code Generation via Probability Masking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Generative language models may generate programming language code that is not syntactically compliant with the grammar of the programming language, leading to errors in compilation or execution.
Innovation Solution
A generative language model is modified to ensure that only grammar-compliant tokens are selected as the next token in a sequence, by applying a mask to reduce or zero the probability of tokens not compliant with the grammar.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a generative language model is trained using programming language to generate code, then the model can produce programming language output, but the generated code may not be syntactically compliant with the grammar of the programming language
Solution Approach 1:
The patent applies a mask to the probability distribution before token selection to preliminarily eliminate invalid tokens. This preliminary action ensures that only grammar-compliant tokens can be selected, preventing syntax errors before they occur in the generated code.
Solution Approach 2:
The patent introduces a mask as an intermediary component between the generative language model's probability output and the token selection process. This mask acts as a mediator that filters out invalid tokens while preserving valid ones, ensuring grammar compliance without modifying the underlying language model.
2Productivity
If the generative language model selects tokens based on highest probability without constraints, then generation speed is maintained, but syntax errors may occur in the generated code
Solution Approach 1:
The mask is applied in advance to the probability distribution, eliminating the need for post-generation syntax checking or correction. This preliminary filtering maintains generation speed by preventing invalid token selection from the outset while ensuring syntax accuracy.
Solution Approach 2:
The patent modifies the probability parameters by applying a mask that sets invalid token probabilities to zero or very low values. This parameter change ensures that the model selects from only valid tokens while maintaining the probabilistic selection mechanism that enables efficient code generation.
3Reliability
If a mask is applied to zero the probability of non-compliant tokens, then grammar compliance is ensured, but the model's flexibility in token selection is reduced
Solution Approach 1:
The mask applies different quality treatments to different tokens locally: valid tokens retain their original probability values and selection flexibility, while invalid tokens have their probabilities reduced to zero. This local differentiation ensures grammar compliance without unnecessarily restricting the model's flexibility for valid token selection.
Data Source
AI summary
A generative language model (e.g. large language model) may be used to generate programming language code. However, the generative language model may sometimes generate an output that is not compliant with the grammar of the programming language. In some embodiments herein, a generative language model may be modified to only generate an output that is grammar-compliant. A method may include generating a plurality of values using the generative language model, where each of the values is indicative of a probability of a respective token being a next token of a token sequence generated by the generative language model. A mask may be applied to the plurality of values. The mask may operate on each value that corresponds to a token not compliant with the grammar of the programming language to reduce or zero the probability of that token being the next token.