A Context-Aware Multi-Constraint Code Generation Method and System
By using dynamic activation of constraint subsets and context gating bias techniques, the shortcomings of existing code generation models in terms of security and correctness are solved. Applicable constraint activation and incremental updates under different constraint scenarios are realized, improving the security and efficiency of code generation. It is suitable for online IDEs and enterprise-level code generation platforms.
Patent Information
- Application Number
- CN202611323217.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-28
- Publication Date
- 2026-11-17
AI Technical Summary
Existing code generation models suffer from insufficient security and correctness when generating code. They struggle to dynamically activate applicable constraints under different constraint scenarios and lack an effective constraint update mechanism. This results in generated code containing syntax errors, type errors, and security risks, failing to meet the low-latency requirements of online code generation.
By dynamically activating a subset of constraints, introducing context-gated bias, unifying the transformation of verification feedback and implementing incremental updates, constructing a set of structured constraints, and dynamically adjusting the constraint strength during the generation process according to the current code context, sparse bias vector storage and context partitioning techniques are used to reduce interference from irrelevant constraints and decoding overhead.
It improves the security and correctness of code generation, reduces decoding overhead in scenarios with large vocabularies, and is suitable for online IDEs, code completion services, and enterprise-level code generation platforms. It also reduces the interference of irrelevant constraints on code semantics and the delay in the generation process.
Smart Images

Figure CN123114476A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of automatic code generation technology, and in particular relates to a context-aware, multi-constraint code generation method and system. Background Technology
[0002] With the development of large language models and code-specific models, automatic code generation based on natural language requirements has become an important auxiliary method in software development. Existing code generation models can generate corresponding code based on user-input natural language requirements, function signatures, contextual code snippets, project dependency information, or test cases, which plays a positive role in improving development efficiency.
[0003] However, existing code generation models still suffer from insufficient security and correctness. For example, they may generate code containing dangerous API calls, lacking input validation, SQL string concatenation, insecure deserialization, path traversal, or command injection; they may also generate code with syntax errors, type errors, missing dependencies, mismatched function signatures, unexpected return value structures, or failed unit tests. Therefore, introducing security constraints, correctness constraints, and engineering standard constraints into the code generation process is of great significance.
[0004] Existing improvement methods mainly include model training or fine-tuning, prompt word constraints, post-generation checks, and decoding stage constraint control. Among these, model training or fine-tuning is costly and difficult to adapt to continuously changing security rules and project environments; prompt word constraints lack stability, and the model may ignore some requirements; post-generation checks are a post-processing method, making it difficult to suppress erroneous code fragments in advance during the generation process; and while decoding stage constraint control can directly affect the generation probability of candidate tokens, it usually treats constraints as a static set applied uniformly to the generation process, failing to fully consider the applicable context of different constraints.
[0005] In real-world engineering scenarios, different constraints are not always applicable; rather, they often only take effect under specific language, syntax location, API call scenario, data source, or variable type conditions. For example, YAML security parsing constraints should primarily take effect in the YAML data parsing context, SQL injection protection constraints should primarily take effect in the database access or SQL construction context, and input validation constraints are typically applicable to function entry points or external data reading locations. Applying all constraints continuously throughout the entire generation process can easily introduce irrelevant constraints, leading to the generation of APIs or code snippets that do not conform to the current semantic context, and increasing the overhead of online matching, bias calculation, and state update.
[0006] Furthermore, generated code typically requires verification processes such as compilation checks, type checks, unit tests, static analysis, and security scans. Existing methods often re-input verification failure information as prompts to the model, lacking a mechanism to structurally transform verification feedback into decoded constraints and to perform local incremental updates to the existing constraint structure. If the constraint matching structure and bias vector are fully reconstructed after each new constraint, it will be difficult to meet the low-latency requirements of online code generation services.
[0007] Therefore, a new multi-constraint code generation method is needed that can identify the current code context during the code generation process, dynamically activate applicable constraints, and reduce interference from irrelevant constraints; can uniformly represent security rules, correctness rules, and verification feedback as constraints that can participate in decoding control; and can incrementally update the constraint structure after verification failure, thereby taking into account the security, correctness, and generation efficiency of the generated code. Summary of the Invention
[0008] This invention proposes a context-aware multi-constraint code generation method and system. By dynamically activating constraint subsets, introducing context gating bias, unifying transformation verification feedback, and implementing incremental updates and conflict arbitration, it effectively reduces decoding overhead in scenarios with large vocabularies and high concurrency while improving the security, correctness, and standardization of code generation.
[0009] To achieve the above objectives, the technical solution of the present invention includes the following:
[0010] A context-aware, multi-constraint code generation method, the method comprising: The system converts user-input natural language requirements, security rule bases, vulnerability rules, enterprise coding standards, target project dependency configurations, static analysis rules, type rules, unit test examples, function signatures, API documentation, and historical defect repair examples into structured constraints and generates a set of structured constraints. The structured constraint set is partitioned according to the applicable context of the structured constraints, and a corresponding constraint matching structure is constructed for each context partition. Each constraint matching state in the constraint matching structure is associated with one or more logit deviation values used to adjust candidate tokens. The constraint matching state is pre-compiled based on the token sequence of the constraint phrase and is used to indicate the matching progress between the currently generated token sequence and the constraint phrase. Before each token generation, based on the matching relationship between the current code context state and the applicable context of each structured constraint, the activated structured constraints and the context gating factor corresponding to the activated structured constraints are obtained, and the logit deviation value of the activated structured constraints for the next token is calculated based on the context gating factor and the generated token sequence. At each step of token generation, the large language model outputs the original logit vector of the next token based on the code generation request and the generated token sequence. After obtaining the logit deviation values of each activated structured constraint for the next token based on the current code context state, it generates the final deviation vector. The final deviation vector is superimposed on the original logit vector to obtain the adjusted logit vector, and the next token is obtained based on the adjusted logit vector. After updating the generated token sequence, current code context state, and constraint matching state based on the next token, the step of obtaining the activated structured constraints before each token generation step is re-executed, based on the matching relationship between the current code context state and the applicable context of each structured constraint, until the set conditions are met, and the generated code corresponding to the code generation request is output.
[0011] Furthermore, the fields in the structured constraints include constraint identifier, constraint phrase, constraint direction, constraint type, applicable context, triggering condition, priority, risk level, and historical feedback status; the constraint direction includes positive constraint representation and negative constraint representation, the positive constraint representing a token sequence or code snippet that is intended to increase the generation probability, and the negative constraint representing a token sequence or code snippet that is intended to decrease or prohibit the generation probability; the constraint type includes one or more of the following: security constraints, syntax correctness constraints, type correctness constraints, interface correctness constraints, test correctness constraints, dependency correctness constraints, performance constraints, and code style constraints.
[0012] Furthermore, the process of maintaining the current code context state includes: For each token generated, the syntax stack, indentation stack, bracket stack, string state, and comment state are updated based on the token to form the updated lightweight context state. The lightweight context state is used to determine whether the current code snippet meets the preset parsing trigger conditions and whether the current code snippet is parsable; wherein, the preset parsing trigger conditions include whether the current token is located at the statement boundary or code block boundary, whether the parentheses are in a closed state, whether the current generation position is outside the string or comment, and whether the number of generated tokens has reached the preset parsing interval; If the current code segment meets the preset parsing trigger conditions and the current code segment is parsed successfully, the current code context state is updated according to the parsing result; wherein, the parsing result includes the syntax tree, symbol table, type information and diagnostic information of the current code segment; If the current code snippet does not meet the preset parsing trigger conditions, or if the parsing of the current code snippet fails, local parsing, sliding window matching, heuristic syntax stack, or finite state machine are used to maintain the current code context state.
[0013] Furthermore, the structured constraint set is partitioned according to the applicable context of the structured constraints, and a corresponding constraint matching structure is constructed for each context partition, including: Based on the applicable context of the structured constraints, the constraint set is divided into multiple context constraint partitions; each context constraint partition corresponds to a type of code generation context, which includes Python-YAML parsing context, Python-SQL access context, Java-XML parsing context, file access context, network request context, function return context, or exception handling context; For each context constraint partition, the constraint phrases of the structured constraints are converted into a sequence of constraint tokens by the segmenter of the target large language model; wherein, the sequence of constraint tokens defines the path and termination node of the constraint matching state transition in the constraint matching structure, so that the constraint matching structure tracks the alignment progress of the tokens generated by the large language model with the constraint phrases in real time, thereby triggering the associated logit bias adjustment; Construct a corresponding constraint matching structure for the context constraint partition; wherein the constraint matching structure adopts a single constraint matching structure, a composite constraint matching structure, or a composite constraint matching structure with context gating; the state of the composite constraint matching structure with context gating includes the constraint matching state and the current code context state.
[0014] Furthermore, based on the matching relationship between the current code context state and the applicable context of each structured constraint, the activated structured constraints and their corresponding context gating factors are obtained, including: The current code context state is represented as a set of context features, and the applicable context of structured constraints is represented as a set of constraint conditions; wherein, the set of constraint conditions includes a set of necessary conditions and a set of unnecessary conditions; necessary conditions include one or more of the following: target language, syntax region, or API call scenario; unnecessary conditions include one or more of the following: data source, variable type, historical verification feedback status, or project configuration conditions; If any necessary condition in the set of necessary conditions is not satisfied by the current context feature set, the corresponding structured constraint will not be activated. If every necessary condition in the set of necessary conditions is satisfied by the current set of context features, the corresponding structured constraint is activated, and the value of the context gating factor is determined based on the number of matching non-necessary conditions, the weight of each condition, and the constraint risk level.
[0015] Furthermore, based on the context gating factor and the generated token sequence, the logit deviation value of the activated structured constraint for the next token is calculated; Use the generated token sequence as the current generation prefix, and obtain the constraint token sequence and constraint direction of the activated structured constraints; Calculate the matching length between the currently generated prefix and the constraint token sequence; Based on the matching length, constraint length, global maximum constraint length, constraint risk level, constraint priority, and context gating factor, calculate the logit deviation value of the activated structured constraint for the next token.
[0016] Furthermore, the process of generating the final deviation vector includes: Based on the constraint type, risk level, context relevance, verification feedback status, and user policy of each activated structured constraint, obtain the activation weight of the activated structured constraint. The final deviation vector is obtained by weighting the logit deviation values of each activated structured constraint for the next token according to the activation weight.
[0017] Furthermore, after generating the code corresponding to the output code generation request, the method further includes: The generated code undergoes compilation checks, interpreted execution checks, type checks, unit tests, static analysis, security scans, and dependency checks. If the check passes, the generated code will be used as the final code; If the check fails, the check result is parsed into structured error information, which is then converted into new constraints. A constraint matching structure fragment corresponding to the new constraint is constructed and merged into the constraint matching structure of the relevant context partition. After recalculating the logit deviation value corresponding to the affected state, code generation is performed again based on the code generation request.
[0018] A context-aware, multi-constraint code generation system, the system comprising: The structured constraint building module is used to convert user-input natural language requirements, security rule bases, vulnerability rules, enterprise coding standards, target project dependency configurations, static analysis rules, type rules, unit test examples, function signatures, API documentation, and historical defect repair examples into structured constraints and generate a set of structured constraints. The partition constraint compilation module is used to partition the structured constraint set according to the applicable context of the structured constraints, and construct a corresponding constraint matching structure for each context partition; wherein, each constraint matching state in the constraint matching structure is associated with one or more logit deviation values for adjusting candidate tokens, and the constraint matching state is pre-compiled based on the token sequence of the constraint phrase, used to indicate the matching progress between the currently generated token sequence and the constraint phrase; The dynamic activation constraint module is used to obtain the activated structured constraints and the context gating factor corresponding to the activated structured constraints based on the matching relationship between the current code context state and the applicable context of each structured constraint before each token generation. Based on the context gating factor and the generated token sequence, the module calculates the logit deviation value of the activated structured constraints for the next token. The online constraint decoding module is used to, at each token generation step, output the original logit vector of the next token based on the code generation request and the generated token sequence, and obtain the logit deviation values of each activated structured constraint for the next token according to the current code context state, and generate the final deviation vector; the final deviation vector is superimposed on the original logit vector to obtain the adjusted logit vector, and the next token is obtained by decoding based on the adjusted logit vector; after updating the generated token sequence, the current code context state and the constraint matching state based on the next token, the step of obtaining the activated structured constraints based on the matching relationship between the current code context state and the applicable context of each structured constraint before each token generation step is re-executed until the set conditions are met, and the generated code corresponding to the code generation request is output.
[0019] A computer device, characterized in that the computer device comprises: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the context-aware multi-constraint code generation method described above.
[0020] Compared with the prior art, the present invention has at least the following beneficial effects.
[0021] 1. Existing methods typically apply constraints as a static set uniformly throughout the generation process, making it difficult to dynamically determine applicability based on the current code context. This can easily lead to irrelevant constraints interfering with code semantics, API selection, and structure. To address this, this invention dynamically activates a subset of relevant constraints, rather than applying all constraints fixedly throughout the entire generation process. This effectively reduces the interference of irrelevant safety, correctness, or engineering specification rules on code semantics.
[0022] 2. Existing methods lack a continuous identification mechanism for multiple dimensions such as language type, syntactic region, and API call scenario during the autoregressive generation process of large models, making it difficult to activate corresponding constraints at specific locations. This invention introduces a context-gated bias vector, which dynamically correlates the strength of constraints with elements such as language type, syntactic region, and API call scenario, thereby precisely strengthening the corresponding constraints at more appropriate code locations.
[0023] 3. Existing methods struggle to uniformly represent and transform various constraints, such as security, correctness, engineering specifications, API usage, and verification feedback, into control signals that can participate in decoding. This invention achieves a unified representation of these multiple types of constraints as positive / negative constraints that can directly participate in decoding control, and transforms them into structured decoding conditions.
[0024] 4. After compilation, testing, or scanning fails, existing methods typically only re-enter the error message as a prompt, lacking a structured transformation and constraint update mechanism for the failure information. This invention directly transforms various verification results into new decoding constraints, enabling feedback information to influence subsequent token generation in real time. Furthermore, by incrementally updating, it only adjusts the constraint state of the affected parts, avoiding full reconstruction and significantly reducing iterative repair latency.
[0025] 5. Existing methods struggle to arbitrate conflicts based on type, risk level, etc., when multiple constraints act simultaneously, easily leading to constraint cancellation, over-enhancement, or semantic drift. This invention addresses this by employing a constraint conflict arbitration mechanism that handles conflicts based on constraint type, risk level, contextual relevance, and historical validation results, effectively reducing the cancellation or aberrant enhancement of bias vectors.
[0026] 6. In high-concurrency code generation scenarios, existing constraint decoding methods tend to incur high overhead in online matching, computation, and updating. This invention effectively reduces the storage and computation overhead in large-vocabulary scenarios through optimization techniques such as sparse bias vector storage, context partitioning, hot state caching, and local logit updates, making it suitable for online IDEs, code completion services, and enterprise-level secure code generation platforms. Attached Figure Description
[0027] Figure 1 This is the overall flowchart of the context-aware multi-constraint code generation method.
[0028] Figure 2 This diagram illustrates the relationship between structured constraints, code context state, and context gating factors.
[0029] Figure 3 A schematic diagram illustrating the process of constructing a context partition constraint matching structure. Detailed Implementation
[0030] The technical solution of the present invention will be further described below with reference to specific embodiments. The examples given are only used to explain the technical solution of this application and are not intended to limit the scope of this application.
[0031] The overall architecture of this invention includes an offline phase, an online phase, and a verification feedback phase. The offline phase primarily completes constraint rule structuring, context applicability condition annotation, context partitioning, constraint matching structure construction, deviation vector pre-computation, sparse storage, and cache initialization. The online phase primarily completes code context state identification, activation constraint selection, context gating deviation vector generation, conflict arbitration, logit adjustment, token sampling, code context state updating, and constraint matching structure state updating. The verification feedback phase primarily completes generated code inspection, error message parsing, verification feedback constraint generation, partial merging of constraint matching structures, partial updating of deviation vectors, and local code repair.
[0032] This invention can be applied to any large language model with a token-level decoding interface, including code-specific models, general-purpose large language models, private deployment models, and cloud-based code generation models. Target programming languages can include Python, Java, JavaScript, TypeScript, C, C++, Go, Rust, PHP, C#, SQL, etc. Target runtime scenarios can include function generation, class generation, interface implementation, unit test generation, code completion, vulnerability patching, code migration, and code refactoring.
[0033] Specifically, the context-aware multi-constraint code generation method of the present invention, such as Figure 1 As shown, it includes the following content.
[0034] (a) Code generation request.
[0035] Obtain the code generation request input by the user. The code generation request includes one or more of the following: natural language code generation requirements, target programming language, target runtime environment, project dependency information, function signature, existing context code, input / output examples, unit test examples, security specifications, code style specifications, and user-defined constraints.
[0036] (ii) Construction of structured constraints.
[0037] This invention first obtains the original constraint sources. These original constraint sources include user-input natural language requirements, security rule bases, vulnerability rules, enterprise coding standards, target project dependency configurations, static analysis rules, type rules, unit test examples, function signatures, API documentation, and historical defect repair examples.
[0038] This invention converts the original constraints into structured constraints. Each structured constraint includes one or more of the following fields: Constraint identifier, used to uniquely identify the constraint; A constraint phrase is used to represent a sequence of tokens whose generation probability is to be increased or decreased. The token sequence includes secure APIs, dangerous APIs, parameter validation fragments, exception handling fragments, import statements, return statement patterns, or specific code structures. The constraint direction is used to indicate whether it is a positive or negative constraint. A positive constraint indicates a code snippet that the model is expected to generate, while a negative constraint indicates a code snippet that the model is expected to avoid generating. Constraint type, used to represent one or more of the following: security constraints, syntax correctness constraints, type correctness constraints, interface correctness constraints, test correctness constraints, dependency correctness constraints, performance constraints, and code style constraints; The applicable context indicates the language, framework, syntax region, API call scenario, data source, variable type, and risk scenario to which the constraint applies; Triggering conditions are used to indicate the conditions that must be met for a constraint to be activated; Priority, used to indicate the processing order when multiple constraints conflict; Risk level, used to indicate the severity of the safety or correctness risk corresponding to the constraint; Historical feedback status is used to record whether the constraint was triggered by a verification failure, the number of times it was triggered, the time of the most recent trigger, and the effect of the repair.
[0039] Specifically, for secure Python YAML parsing scenarios, the structured constraints include: a negative constraint phrase "yaml.load()", a positive constraint phrase "yaml.safe_load()", an applicable context of "Python language, YAML parsing, external input data or untrusted data source", a risk level of "high", and a priority of "relatively high".
[0040] (iii) Context state maintenance.
[0041] During code generation, the current code context state is maintained based on the generated token sequence, code syntax structure, indentation level, bracket matching status, string status, comment status, function call status, variable definition status, data source, and API call pattern. The code context state includes one or more of the following: language state, syntax region state, structure state, call status, data source state, variable type state, dependency state, and verification feedback state.
[0042] The language state indicates the programming language and version of the current code, including Python, Java, JavaScript, C++, Go, or Rust.
[0043] The syntax region state is used to indicate the current position of the token, including the import region, class definition region, function definition region, function body region, conditional branch region, loop region, exception handling region, string region, comment region, parameter list region, and return statement region.
[0044] The structure state is used to represent the current state of parentheses, indentation, code blocks, string boundaries, and statement boundaries, including whether it is currently in an unclosed parenthesis, in a string literal, or in a function call parameter list.
[0045] The call status is used to indicate whether an API call, function call, class instantiation, database query, file access, network access, serialization operation, or deserialization operation is currently being generated.
[0046] The data source status is used to indicate whether the current variable or parameter comes from external input, untrusted input, file, network, database, user form, environment variable, or command line parameter.
[0047] The variable type state is used to indicate the inferred type of the current variable, parameter, return value, or intermediate expression, including string, integer, list, dictionary, file object, database connection, and HTTP request object.
[0048] The dependency state represents the libraries, modules, classes, and functions that are already imported or available in the current code. When a positive constraint needs to call a library that has not yet been imported, this invention generates a supplementary dependency constraint.
[0049] The verification feedback status is used to indicate the result of the most recent compilation, testing, type checking, or security scan, as well as the error type and error location related to the current code region.
[0050] In one embodiment, the current code context state is maintained through incremental parsing. For each generated token, the present invention first updates the syntax stack, indentation stack, bracket stack, string state, and comment state based on the token, forming an updated lightweight context state.
[0051] Subsequently, the present invention determines whether the current code segment meets the preset parsing trigger conditions based on the lightweight context state. The parsing trigger conditions include whether the current token is located at the statement boundary or code block boundary, whether the parentheses are closed, whether the current generation position is outside the string or comment, and whether the number of generated tokens has reached the preset parsing interval, etc.
[0052] When the current code segment meets the parsing trigger condition, this invention calls a lightweight syntax parser or language server to parse the current code segment. If the parsing is successful, the syntax tree, symbol table, type information, and diagnostic information of the current code segment are obtained, and the code context state is updated according to the parsing result. If the current code segment does not meet the parsing trigger condition, or the parsing fails, an approximate context state is maintained using local parsing, sliding window matching, heuristic syntax stack, or finite state machine methods.
[0053] (iv) Partition constraint compilation.
[0054] Based on the applicable context of the structured constraints, the constraint set is divided into multiple context constraint partitions. For each context constraint partition, the constraint phrase is converted into a token sequence by the tokenizer of the target large language model, and a corresponding constraint matching structure is constructed for that context constraint partition.
[0055] The constraint matching structure can be a deterministic finite automaton, a composite deterministic finite automaton, a Trie tree, a multi-pattern matching automaton with failure pointers, a hash state machine, or other data structures capable of representing the matching state of a token sequence. Each matching state is associated with one or more bias values or bias vectors used to adjust the candidate token logit.
[0056] Specifically, in the offline phase, this invention first partitions the structured constraint set according to the applicable context of the constraints, and then constructs a corresponding constraint matching structure for each context partition. Each partition corresponds to a type of code generation context, including Python-YAML parsing context, Python-SQL access context, Java-XML parsing context, file access context, network request context, function return context, and exception handling context, etc.
[0057] For each context partition, this invention converts the constraint phrases in that partition into token sequences using the segmenter corresponding to the target large language model, and constructs a constraint matching structure corresponding to that context partition. The constraint matching structure records the matching state between the constraint phrases and the generated token sequences, and its state transitions represent changes in the matching state after adding a new token.
[0058] The constraint matching structure can be a single constraint matching structure, a composite constraint matching structure, or a composite constraint matching structure with context gating.
[0059] For a single constraint phrase, this invention constructs a deterministic finite automaton corresponding to that constraint phrase. For a constraint token sequence of length L, the deterministic finite automaton includes L+1 primary matching states, each state representing the length of the longest matching prefix between the currently generated token sequence and the constraint token sequence. Tokens outside the constraint phrase are mapped to a uniform OTHER transition to reduce the size of the state transition table.
[0060] For multiple constraint phrases within the same context partition, this invention merges multiple single-constraint matching structures into a composite constraint matching structure. The state of the composite constraint matching structure represents a combination of the states of multiple single-constraint matching structures. This invention prunes unreachable states in the composite constraint matching structure and pre-calculates the corresponding deviation vector for each reachable state.
[0061] To reduce the state scale caused by merging global constraints, this invention constructs multiple composite constraint matching structures according to context partitions, instead of merging all constraints into a single global composite constraint matching structure. In the online phase, this invention selects one or more relevant context partitions' composite constraint matching structures for querying based on the current code context state, thereby reducing the number of states in a single matching structure and lowering the cost of state transitions and deviation vector queries.
[0062] When it is necessary to model the context state and the constraint matching state in a unified manner, this invention constructs a composite constraint matching structure with context gating. The state of this composite constraint matching structure includes the constraint matching state and the context state, and the final deviation vector is jointly determined by the basic deviation vector corresponding to the constraint matching state and the context gating factor.
[0063] (v) Dynamic activation constraints.
[0064] like Figure 2As shown, during the online generation phase, before each token generation step, this invention calculates the context gating factor for each constraint based on the matching relationship between the current code context state and the applicable context of each structured constraint. The context gating factor is used to indicate whether the constraint is activated in the current code context and the strength of its activation; it can be a binary variable or a continuous variable.
[0065] To calculate the context gating factor, this invention represents the current code context state as a set of context features and the applicable context of structured constraints as a set of constraint conditions. The set of constraint conditions includes a set of necessary conditions and a set of unnecessary conditions. If any condition in the set of necessary conditions is not satisfied by the current set of context features, the corresponding constraint is not activated; if all necessary conditions are satisfied, the continuous context gating factor is determined based on the number of matching unnecessary conditions, the weight of each condition, and the constraint risk level. The necessary conditions include one or more of the following: target language, syntax region, or API call scenario; the unnecessary conditions include one or more of the following: data source, variable type, historical verification feedback status, or project configuration conditions.
[0066] For example, when the current code state is "Python function body, processing external input, parsing YAML string", constraints related to YAML security parsing are activated; constraints related to SQL injection protection, XML parsing, or file path traversal are not activated or remain inactive.
[0067] Therefore, this invention avoids continuously using all constraints throughout the generation process, but instead activates a specific subset of constraints in a specific context, thereby reducing the probability of irrelevant constraints introducing erroneous APIs or code snippets.
[0068] (vi) Generation of gating deviation vector This invention pre-calculates a basic deviation vector for each state of the constraint matching structure. The basic deviation vector represents the logit value that should be increased or decreased for each candidate token in that state.
[0069] In each decoding step, the token sequence that has been generated and confirmed in the previous decoding step is used as the current generation prefix. For the i-th constraint, its constraint token sequence is denoted as... The constraint direction is denoted as If the constraint is a positive constraint, then If the constraint is positive, then... The result is negative. This invention is based on the current generated prefix and... The deviation value of the candidate next token is calculated based on the matching length, constraint length, global maximum constraint length, constraint risk level, constraint priority, and context gating factor.
[0070] The deviation value is determined by one or more of the following factors: A. Matching length factor, used to reflect the closeness of the currently generated content to the constraint phrase; B. Length normalization factor, used to prevent longer or shorter constraints from being unfairly reinforced; C. Constraint direction factor, used to distinguish between positive and negative constraints; D. Context gating factors, used to determine the activation strength of constraints in the current context; E. Priority factors, used to distinguish the importance of different constraints such as safety, correctness, and style; F. Risk level factor, used to strengthen constraints related to high-risk vulnerabilities or serious correctness errors; G. Feedback factors, used to reinforce constraints triggered by recent validation failures.
[0071] To avoid semantic drift in model generation due to excessive constraint bias, this invention uses bounded functions to compress the bias values. These bounded functions include the sigmoid function, tanh function, piecewise linear function, or maximum truncation function. This invention also sets a global temperature coefficient or constraint strength coefficient to adjust the impact of constraints on the original logit.
[0072] The final deviation vector is obtained by weighted aggregation of multiple active constraint deviation vectors. For inactive constraints, their context gating factor is zero, and these constraints do not participate in the calculation of the final deviation vector; for low-relevance constraints, their context gating factor is small, and these constraints only have a weak effect.
[0073] (vii) Binding conflict arbitration.
[0074] In multi-constraint scenarios, different constraints may have different effects on the same candidate token in different directions or with different strengths. For example, security constraints may reduce the probability of generating a dangerous API, while correctness constraints may tend to generate the same token due to test cases or function signature requirements; code style constraints may require simplified code, while security constraints may require increased exception handling or input validation. If all deviation vectors are summed directly, problems such as mutual cancellation, over-enhancement, or generation of semantically unstable tokens can easily occur.
[0075] To address this, the present invention establishes a constraint conflict arbitration mechanism. When multiple constraints cause deviations of different directions or strengths for the same candidate token, the constraint conflict arbitration mechanism determines the final deviation value based on the constraint type, risk level, contextual relevance, verification feedback status, and user strategy.
[0076] Specifically, the constraint conflict arbitration mechanism first identifies the set of constraints that produce non-zero deviations for the same candidate token, and then determines the final deviation value based on the following factors: 1) Constraint type priority, where compile correctness and type correctness constraints take precedence over code style constraints, high-risk security constraints take precedence over ordinary performance constraints, and user-specified hard constraints take precedence over automatically inferred soft constraints; 2) Risk level, where higher-risk security vulnerabilities have higher weighting; 3) Context relevance, where constraints that are highly relevant to the current code context have higher weights, and constraints that are weakly relevant to the current context have lower weights; 4) Verify the feedback status, where constraints triggered by the most recent compilation failure, test failure, or static analysis alarm have higher weight in the subsequent repair phase; if a constraint fails to improve the verification results after multiple applications, its weight is reduced or an alternative constraint is triggered. 5) User policies, which include security priority, correctness priority, speed priority and conservative generation. Different user policies correspond to different arbitration rules.
[0077] When determining the final deviation value, this invention calculates a weighted sum of multiple deviation values for the same token according to the factors mentioned above, and then normalizes or truncates the weighted sum. When positive and negative constraints directly conflict, this invention selects the constraint with the highest priority as the dominant constraint and suppresses lower priority constraints. For example, for high-risk security constraints, their weight is increased; for correctness constraints triggered by recent verification failures, their weight is increased; for constraints that have a continuous effect but have not improved the verification results, their weight is decreased.
[0078] Furthermore, this invention also marks conflict tokens as tokens to be verified and focuses on verifying their impact during the post-generation inspection phase.
[0079] (viii) Online constraint decoding process.
[0080] During the online code generation phase, this invention receives code generation requests and initializes the code context state, constraint matching structure state, and cache state.
[0081] The large language model outputs the original logit vector of the next token based on natural language requirements, existing code context, and the generated token sequence. This invention queries the applicable context partition based on the current code context state and obtains the matching status and basic deviation vector of the activated constraints in the corresponding partition.
[0082] Subsequently, this invention calculates the context gating factor, generates a context gating bias vector, and performs constraint conflict arbitration and bias vector aggregation on multiple context gating bias vectors to form the final bias vector. For the final bias vector represented by a sparse structure, this invention identifies the tokens corresponding to the non-zero bias terms as affected tokens, performs only a local addition update on their logit values, and does not perform a full update on the complete vocabulary vector. For the combination of cache hit matching state and context state, this invention directly reads the final bias vector from the cache.
[0083] The final bias vector is superimposed on the original logit vector to obtain the adjusted logit vector. Then, the adjusted logit is converted into a probability distribution using softmax, and the next token is selected using greedy decoding, top-k sampling, top-p kernel sampling, temperature sampling, beam search, or speculative decoding.
[0084] After generating the next token, this invention synchronously updates the generated token sequence, code context state, constraint matching structure state, sparse cache state, and verification-related state. If token generation changes the current context—for example, entering a function call parameter list, leaving a string region, entering an exception handling block, starting SQL string generation, or completing an API call—then the activation constraint set is recalculated. This process iterates until an end-of-line character is generated, the maximum length is reached, the objective function is completed, or other termination conditions are met.
[0085] (ix) Verify feedback-driven incremental constraint updates.
[0086] This invention performs verification checks after code generation and performs incremental constraint updates based on verification feedback. The verification checks include one or more of the following: compilation checks, interpreter syntax checks, type checks, unit tests, static analysis, security scanning, dependency checks, and runtime sandbox checks.
[0087] When the verification check passes, the present invention outputs the final code. When the verification check fails, the present invention parses the failure result into structured error information. The structured error information includes error type, error location, error message, related variables, related APIs, related test cases, expected output, actual output, security alarm number, and alarm severity level.
[0088] This invention generates new constraints based on structured error information. Specifically, when the error type is missing dependency or undefined name, this invention generates positive constraints for import statements or API replacement constraints; when the error type is type mismatch, this invention generates positive constraints for type conversion, return value structure adjustment, or parameter processing; when the error type is unit test assertion failure, this invention generates local repair constraints based on the failed input, expected output, and actual output; when the error type is unsafe API usage, this invention generates negative constraints for dangerous APIs and positive constraints for safe APIs; when the error type is missing exception handling, this invention generates positive constraints for exception handling logic or error return logic; when the error type is unvalidated external input, this invention generates positive constraints for input validation, boundary checks, or whitelist validation.
[0089] For new constraints, this invention does not reconstruct the constraint matching structure of all context partitions in its entirety, but instead performs incremental updates. The incremental updates include: determining the context partition to which the new constraint applies; segmenting the new constraint phrase into a token sequence; constructing the matching structure fragment corresponding to the new constraint; partially merging the new matching structure fragment with the original constraint matching structure of the target context partition; identifying the set of affected states; recalculating only the basic deviation vector and cached items for the affected states; retaining the constraint structures of unaffected states and other context partitions; and updating the constraint index, context gating index, and conflict arbitration parameters.
[0090] If verification fails but a specific function, code block, or statement can be located, this invention performs local repair from the vicinity of the error location, related function, or related statement, rather than regenerating the entire code. During local repair, this invention inputs the code snippets before and after the error location, the error message, and the newly added constraints into the large language model, and uses the updated constraint decoding structure during the generation of the repaired snippet. If verification fails and the error cannot be located, or if the error involves the global structure, the complete code is regenerated.
[0091] (x) Sparse storage and hot state caching of constraint deviation vectors.
[0092] The constraint deviation vectors described in this section include the basic deviation vectors corresponding to the constraint matching state, the context-gated deviation vectors formed by weighting the context gating factors, and the final deviation vectors formed by conflict arbitration and aggregation, but do not include the feature representation of the code context state itself.
[0093] When the vocabulary of a large language model is substantial, storing a deviation vector of the entire vocabulary dimension for each constraint matching state would result in high storage, data transfer, and online update overhead. To reduce deployment costs, this invention employs a sparse deviation vector storage and hot-state caching mechanism, optimizing the storage format, update range, and caching method of the deviation vector.
[0094] This invention represents the constraint bias vector as a sparse structure, storing only the token IDs of non-zero bias tokens and their corresponding bias values. For tokens unaffected by constraints, no bias value is stored. During the online decoding phase, based on the final bias vector of the sparse representation, only the tokens recorded within it are locally updated using logit, without performing a full addition operation on the logit vector of the complete vocabulary dimension.
[0095] To further reduce storage requirements, this invention can filter sparse deviation terms based on their deviation values, retaining only the top-k tokens with the highest absolute deviation values for each constraint matching state. The value of the top-k is dynamically determined based on the constraint type, context partition, model vocabulary size, online latency requirements, and storage resource limitations. Tokens with deviation values below a preset threshold can be either not stored or excluded from logit updates during the online decoding phase.
[0096] For cases where the deviation vectors between adjacent states or parent-child states have high similarity, this invention employs a differential storage method. Specifically, if the deviation vector of the current state differs only slightly from the deviation vectors of its parent state, previous state, or adjacent state, only the difference terms are stored. During the online phase, the required deviation vector for the current state is recovered based on the basic deviation vector and the difference terms, thereby reducing redundant storage.
[0097] This invention can also perform low-precision quantization on deviation values. The low-precision quantization includes float16, bfloat16, int8, or piecewise quantization representation. By reducing the precision of the deviation value representation, memory usage and data transfer costs between the CPU, GPU, or other computing devices can be reduced. During the quantization process, different quantization precisions can be set according to the constraint type, risk level, and deviation value range to avoid excessively weakening the deviation effect of high-priority constraints.
[0098] This invention maintains a hot state cache to store frequently occurring context states, constraint matching states, and deviation vector combinations. During the online phase, when the current context state and constraint matching state hit the hot state cache, the final deviation vector in the cache is directly read, without needing to re-execute context gating calculation, constraint conflict arbitration, and deviation vector aggregation. For states with low access frequency or those that have been missed for a long time, the corresponding cache entries can be released according to a cache eviction policy.
[0099] In batch decoding scenarios, multiple code generation requests may be in the same or similar context states. This invention shares the context partition index, constraint matching structure transfer table, sparse deviation vector, and hot state cache for such requests. For requests in the same batch with the same context state and constraint matching state, deviation vector queries and logit local updates can be merged, thereby reducing the overhead of redundant computations in high-concurrency code generation services.
[0100] In summary, this invention synchronously maintains the code context state and constraint matching state during each token generation step; calculates the context gating factor for each structured constraint based on the code context state; selects the constraint matching structure in the relevant context partition based on the context gating factor and generates a gating deviation vector; superimposes the gating deviation vector onto the original logit vector output by the large language model to adjust the generation probability of the next token; after the generated code undergoes compilation checks, type checks, unit tests, static analysis, or security scans, the verification feedback is parsed as a new constraint, and only the affected matching state and deviation vector in the relevant context partition are incrementally updated.
[0101] The application scenarios of the present invention will be illustrated below through two specific embodiments.
[0102] Example 1: YAML secure parsing code generation.
[0103] In this embodiment, the user inputs a natural language requirement: "Write a Python function to read the user-uploaded YAML configuration file and return a configuration dictionary." This invention identifies Python as the target language, YAML parsing as the task, and the data source as a user-uploaded file, classifying it as an external input scenario.
[0104] This invention selects YAML security parsing-related constraints from a constraint library. Negative constraints include "yaml.load()", and positive constraints include "yaml.safe_load()". Furthermore, this invention recognizes that using "yaml.safe_load()" requires the existence of an "import yaml" statement or an equivalent import statement, thus generating dependency correctness constraints.
[0105] During online generation, when the model enters file reading and YAML parsing related syntax areas, the context gating factor of the YAML safe parsing constraints is increased, applying a positive bias to tokens related to "yaml.safe_load()" and a negative bias to tokens related to "yaml.load()". When the model generates other unrelated code areas, such as function definitions, ordinary variable assignments, or return statements, the YAML constraint gating factor is decreased to avoid unnecessary insertion of YAML parsing APIs.
[0106] After generation, this invention performs interpreter syntax checks and security scans. If a missing import statement is found, resulting in an undefined name error, this invention converts the error into an "import yaml" positive constraint, incrementally updates the constraint matching structure of the Python-YAML context partition, and then performs local repairs from the beginning of the import region or function.
[0107] Example 2: Correction fix driven by unit test feedback.
[0108] In this embodiment, the user requests the generation of a function and provides input / output examples. After the initial code generation, unit testing is performed, and it is found that the return value type does not meet expectations under a certain boundary input. By parsing the test failure information, the failed test case, expected return value, actual return value, and error location are obtained.
[0109] This invention transforms the test failure into a correctness constraint. For example, if a list is expected but a string is actually returned, a positive constraint for return value wrapping or type conversion is generated; if an empty list is expected to be returned for empty input but an exception is actually thrown, a positive constraint for boundary condition handling is generated.
[0110] The newly added constraint is marked as a test correctness constraint, applicable to the function entry boundary judgment region or return statement region. This invention incrementally compiles this new constraint into the corresponding context partition and increases its context gating factor during the local repair phase. If the error location corresponds to a return statement, local repair is performed near the return statement; if the error involves boundary input processing, local repair is performed from the function entry condition judgment region. After repair, this invention executes unit tests again; if the test passes, the temporary weight of the feedback constraint in subsequent generation processes is reduced, or the feedback constraint is solidified as a local correctness constraint for the current task; if the test still fails, new constraints are generated again based on new test failure information, and incremental updates are performed on the relevant context partitions.
[0111] Although specific embodiments of the system have been disclosed for illustrative purposes to aid in understanding and implementing the system, those skilled in the art will understand that various substitutions, variations, and modifications are possible without departing from the spirit and scope of the system and the appended claims. Therefore, the system should not be limited to the content disclosed in the preferred embodiments, and the scope of protection claimed by the system is determined by the scope defined in the claims.
Claims
1. A context-aware, multi-constraint code generation method, characterized in that, The method includes: The system converts user-input natural language requirements, security rule bases, vulnerability rules, enterprise coding standards, target project dependency configurations, static analysis rules, type rules, unit test examples, function signatures, API documentation, and historical defect repair examples into structured constraints and generates a set of structured constraints. The structured constraint set is partitioned according to the applicable context of the structured constraints, and a corresponding constraint matching structure is constructed for each context partition. Each constraint matching state in the constraint matching structure is associated with one or more logit deviation values used to adjust candidate tokens. The constraint matching state is pre-compiled based on the token sequence of the constraint phrase and is used to indicate the matching progress between the currently generated token sequence and the constraint phrase. Before each token generation, based on the matching relationship between the current code context state and the applicable context of each structured constraint, the activated structured constraints and the context gating factor corresponding to the activated structured constraints are obtained, and the logit deviation value of the activated structured constraints for the next token is calculated based on the context gating factor and the generated token sequence. At each step of token generation, the large language model outputs the original logit vector of the next token based on the code generation request and the generated token sequence. After obtaining the logit deviation values of each activated structured constraint for the next token based on the current code context state, it generates the final deviation vector. The final deviation vector is superimposed on the original logit vector to obtain the adjusted logit vector, and the next token is obtained based on the adjusted logit vector. After updating the generated token sequence, current code context state, and constraint matching state based on the next token, the step of obtaining the activated structured constraints before each token generation step is re-executed, based on the matching relationship between the current code context state and the applicable context of each structured constraint, until the set conditions are met, and the generated code corresponding to the code generation request is output.
2. The method according to claim 1, characterized in that, The fields in the structured constraints include constraint identifier, constraint phrase, constraint direction, constraint type, applicable context, triggering condition, priority, risk level, and historical feedback status; the constraint direction includes positive constraint representation and negative constraint representation, the positive constraint representing a token sequence or code snippet that is intended to increase the generation probability, and the negative constraint representing a token sequence or code snippet that is intended to decrease or prohibit the generation probability; the constraint type includes one or more of the following: security constraints, syntax correctness constraints, type correctness constraints, interface correctness constraints, test correctness constraints, dependency correctness constraints, performance constraints, and code style constraints.
3. The method according to claim 1, characterized in that, The process of maintaining the current code context state includes: For each token generated, the syntax stack, indentation stack, bracket stack, string state, and comment state are updated based on the token to form the updated lightweight context state. The lightweight context state is used to determine whether the current code snippet meets the preset parsing trigger conditions and whether the current code snippet is parsable; wherein, the preset parsing trigger conditions include whether the current token is located at the statement boundary or code block boundary, whether the parentheses are in a closed state, whether the current generation position is outside the string or comment, and whether the number of generated tokens has reached the preset parsing interval; If the current code segment meets the preset parsing trigger conditions and the current code segment is parsed successfully, the current code context state is updated according to the parsing result; wherein, the parsing result includes the syntax tree, symbol table, type information and diagnostic information of the current code segment; If the current code snippet does not meet the preset parsing trigger conditions, or if the parsing of the current code snippet fails, local parsing, sliding window matching, heuristic syntax stack, or finite state machine are used to maintain the current code context state.
4. The method according to claim 1, characterized in that, The structured constraint set is partitioned according to the applicable context of the structured constraints, and a corresponding constraint matching structure is constructed for each context partition, including: Based on the applicable context of the structured constraints, the constraint set is divided into multiple context constraint partitions; each context constraint partition corresponds to a type of code generation context, which includes Python-YAML parsing context, Python-SQL access context, Java-XML parsing context, file access context, network request context, function return context, or exception handling context; For each context constraint partition, the constraint phrases of the structured constraints are converted into a sequence of constraint tokens by the segmenter of the target large language model; wherein, the sequence of constraint tokens defines the path and termination node of the constraint matching state transition in the constraint matching structure, so that the constraint matching structure tracks the alignment progress of the tokens generated by the large language model with the constraint phrases in real time, thereby triggering the associated logit bias adjustment; Construct a corresponding constraint matching structure for the context constraint partition; wherein the constraint matching structure adopts a single constraint matching structure, a composite constraint matching structure, or a composite constraint matching structure with context gating; the state of the composite constraint matching structure with context gating includes the constraint matching state and the current code context state.
5. The method according to claim 1, characterized in that, Based on the matching relationship between the current code context state and the applicable context of each structured constraint, obtain the activated structured constraints and their corresponding context gating factors, including: The current code context state is represented as a set of context features, and the applicable context of structured constraints is represented as a set of constraint conditions; wherein, the set of constraint conditions includes a set of necessary conditions and a set of unnecessary conditions; necessary conditions include one or more of the following: target language, syntax region, or API call scenario; unnecessary conditions include one or more of the following: data source, variable type, historical verification feedback status, or project configuration conditions; If any necessary condition in the set of necessary conditions is not satisfied by the current context feature set, the corresponding structured constraint will not be activated. If every necessary condition in the set of necessary conditions is satisfied by the current set of context features, the corresponding structured constraint is activated, and the value of the context gating factor is determined based on the number of matching non-necessary conditions, the weight of each condition, and the constraint risk level.
6. The method according to claim 1, characterized in that, Based on the context gating factor and the generated token sequence, calculate the logit deviation value of the activated structured constraint for the next token; Use the generated token sequence as the current generation prefix, and obtain the constraint token sequence and constraint direction of the activated structured constraints; Calculate the matching length between the currently generated prefix and the constraint token sequence; Based on the matching length, constraint length, global maximum constraint length, constraint risk level, constraint priority, and context gating factor, calculate the logit deviation value of the activated structured constraint for the next token.
7. The method according to claim 1, characterized in that, The process of generating the final deviation vector includes: Based on the constraint type, risk level, context relevance, verification feedback status, and user policy of each activated structured constraint, obtain the activation weight of the activated structured constraint. The final deviation vector is obtained by weighting the logit deviation values of each activated structured constraint for the next token according to the activation weight.
8. The method according to any one of claims 1 to 7, characterized in that, After generating the code corresponding to the output code generation request, the method further includes: The generated code undergoes compilation checks, interpreted execution checks, type checks, unit tests, static analysis, security scans, and dependency checks. If the check passes, the generated code will be used as the final code; If the check fails, the check result is parsed into structured error information, which is then converted into new constraints. A constraint matching structure fragment corresponding to the new constraint is constructed and merged into the constraint matching structure of the relevant context partition. After recalculating the logit deviation value corresponding to the affected state, code generation is performed again based on the code generation request.
9. A context-aware, multi-constraint code generation system, characterized in that, The system includes: The structured constraint building module is used to convert user-input natural language requirements, security rule bases, vulnerability rules, enterprise coding standards, target project dependency configurations, static analysis rules, type rules, unit test examples, function signatures, API documentation, and historical defect repair examples into structured constraints and generate a set of structured constraints. The partition constraint compilation module is used to partition the structured constraint set according to the applicable context of the structured constraints, and construct a corresponding constraint matching structure for each context partition; wherein, each constraint matching state in the constraint matching structure is associated with one or more logit deviation values for adjusting candidate tokens, and the constraint matching state is pre-compiled based on the token sequence of the constraint phrase, used to indicate the matching progress between the currently generated token sequence and the constraint phrase; The dynamic activation constraint module is used to obtain the activated structured constraints and the context gating factor corresponding to the activated structured constraints based on the matching relationship between the current code context state and the applicable context of each structured constraint before each token generation. Based on the context gating factor and the generated token sequence, the module calculates the logit deviation value of the activated structured constraints for the next token. The online constraint decoding module is used to, at each token generation step, output the original logit vector of the next token based on the code generation request and the generated token sequence, and obtain the logit deviation values of each activated structured constraint for the next token according to the current code context state, and generate the final deviation vector; the final deviation vector is superimposed on the original logit vector to obtain the adjusted logit vector, and the next token is obtained by decoding based on the adjusted logit vector; after updating the generated token sequence, the current code context state and the constraint matching state based on the next token, the step of obtaining the activated structured constraints based on the matching relationship between the current code context state and the applicable context of each structured constraint before each token generation step is re-executed until the set conditions are met, and the generated code corresponding to the code generation request is output.
10. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the context-aware multi-constraint code generation method as described in any one of claims 1-8.
