Program analysis agent hallucination suppression method and system based on structured prompt words

Abstract

This invention belongs to the field of software engineering technology, specifically disclosing a method and system for suppressing hallucinations in program analysis agents based on structured cue words. The method includes: extracting structured feature information from the code to be analyzed; constructing multi-dimensional structured cue words based on the extracted structured features; inputting the structured cue words into a large language model for analysis; performing multi-stage intermediate result verification on the analysis results of the large language model; locating the hallucination source based on the verification results; constructing correction cue words to guide the large language model in self-correction; and aggregating the verified analysis results to generate a final program analysis report. This invention significantly improves the accuracy of the large language model's understanding of code structure and semantics, reducing the probability of hallucinations from the source.

Description

Technical Field

[0001] This invention relates to the field of software engineering technology, and in particular to a method and system for suppressing hallucinations in program analysis agents based on structured cue words. Background Technology

[0002] In existing technologies, solutions to the illusion problem of large language models mainly focus on general dialogue scenarios, including knowledge injection methods based on retrieval-enhanced generation (RAG), reasoning enhancement methods based on chain-of-thought, and reinforcement learning (RLHF) methods based on human feedback.

[0003] However, these methods have limited effectiveness in the specific field of program analysis, mainly because: program code has strict structured features and precise semantic requirements, and general prompt word design methods cannot effectively capture the key features of the code; program analysis tasks need to deal with complex data flow, control flow and dependencies, and existing methods lack targeted information organization mechanisms; the intermediate results of program analysis are verifiable, but existing methods have failed to make full use of this feature for hallucination detection and correction. Summary of the Invention

[0004] To address the aforementioned issues, this invention proposes a method and system for suppressing hallucinations in program analysis agents based on structured cue words. It designs multi-dimensional structured cue words to improve the accuracy of large language models in understanding code structure and semantics, performs multi-dimensional automated verification of model outputs, and achieves automatic correction of hallucinations through a feedback-guided iterative correction strategy.

[0005] In some implementations, the following technical solutions are adopted: A method for suppressing hallucinations in a program analysis agent based on structured cue words, comprising: Extract the structured feature information of the code to be analyzed, including lexical features, syntactic features, semantic features, and developer intent features; Based on the extracted structured features, multi-dimensional structured prompt words are constructed; the structured prompt words include: task instruction blocks, code content blocks, structural feature blocks, context enhancement blocks, and verification constraint blocks; The structured prompt words are input into a large language model for analysis, and the analysis results of the large language model are verified through multiple stages of intermediate results. Based on the verification results, the source of the hallucination is located, and correction prompts are constructed to guide the large language model to perform self-correction. The validated analysis results are aggregated to generate the final program analysis report.

[0006] As a further embodiment, the lexical features include identifiers, keywords, operators, and literals; The syntactic features include function definitions, class definitions, control flow structures, and expression structures; The semantic features include variable definition-usage chains, function call relationships, and dependency graphs; The developer intent features include code comments, docstrings, and natural language descriptions of naming conventions.

[0007] As a further embodiment, the task instruction block includes an analysis task type identifier, target description, output format requirements, and constraints. The code content block includes the original text of the code to be analyzed, and is annotated with code metadata; The structural feature blocks include the structured analysis results of the AST summary, control flow graph summary, and data flow graph summary; The context enhancement block includes relevant code snippets, API documentation, and historical analysis records. The verification constraint block includes multiple verification rules.

[0008] As a further step, the analysis results of the large language model are validated through multiple stages of intermediate results, including: Syntax consistency verification is used to verify whether the code elements involved in the output of a large language model exist in the original code; Type consistency verification is used to verify whether the type information inferred by the large language model is consistent with the results of static type analysis. Control flow consistency verification is used to verify whether the execution path described by the large language model exists in the control flow graph; Data flow consistency verification is used to verify whether the data transmission relationships described by the large language model are consistent with the data flow analysis results. Logical consistency verification is used to verify whether the logical relationships in the reasoning chain of a large language model are self-consistent.

[0009] As a further solution, based on the specific type of verification failure, the information fragment that caused the hallucination is located, and the hallucination information fragment, the reason for the verification failure, and the correct reference information are encapsulated into correction prompt words. The correction prompt words are then input into the large language model to guide the large language model to correct the erroneous output.

[0010] As a further step, if all verifications pass, the analysis results are aggregated to generate a program analysis report; the program analysis report includes at least the analysis conclusions, confidence scores, and supporting evidence chain information, wherein the confidence scores are obtained based on a weighted calculation of the verification pass rates.

[0011] As a further solution, the confidence score is calculated as follows: ; in, For the first i The status of the verification is indicated by 1 for successful verification and 0 for failed verification. For the first i The weights of each validation item are pre-set based on the importance of the validation type.

[0012] In other embodiments, the following technical solutions are adopted: A hallucination suppression system for a program analysis intelligent agent based on structured cue words, comprising: The feature extraction module is configured to extract structured feature information of the code to be analyzed, including lexical features, syntactic features, semantic features, and developer intent features. The prompt word construction module is configured to construct multi-dimensional structured prompt words based on extracted structured features; the structured prompt words include: task instruction blocks, code content blocks, structural feature blocks, context enhancement blocks, and verification constraint blocks; The analysis and verification module is configured to input structured prompt words into a large language model for analysis and to perform multi-stage intermediate result verification on the analysis results of the large language model. The hallucination correction module is configured to locate the hallucination source based on the verification results and construct correction prompts to guide the large language model to perform self-correction. The results aggregation module is configured to aggregate validated analysis results to generate a final program analysis report.

[0013] In other embodiments, the following technical solutions are adopted: A terminal device includes a processor and a memory, the processor being used to implement instructions; the memory being used to store multiple instructions adapted for loading and execution by the processor of the aforementioned method for suppressing hallucinations in intelligent agents based on structured cue words.

[0014] In other embodiments, the following technical solutions are adopted: A computer-readable storage medium storing a plurality of instructions adapted for loading and execution by a processor of a terminal device of the aforementioned method for suppressing hallucinations in intelligent agents based on structured cue words.

[0015] Compared with the prior art, the beneficial effects of the present invention are: (1) This invention constructs structured prompt words from multiple dimensions such as task instructions, code content, structural features, context enhancement and verification constraints. It organizes the lexical features, syntactic features, semantic features and developer intent features of the code in an orderly manner in the form of XML format tags, which significantly improves the accuracy of the large language model in understanding the code structure and semantics, and reduces the probability of hallucination from the source.

[0016] (2) This invention achieves multi-dimensional automated verification of model output through an intermediate result verification mechanism based on a program analysis toolchain, enabling timely detection and location of hallucination information; furthermore, through a feedback-guided iterative correction strategy, it achieves automatic correction of hallucinations, improving the system's adaptability and output reliability; and through a confidence scoring mechanism, it provides quantifiable reliability indicators for the analysis results, facilitating user evaluation and decision-making. The overall solution of this invention is specifically designed for program analysis scenarios, making full use of the structured features of code and the verification capabilities of program analysis tools, and has better performance in program analysis tasks compared to general hallucination suppression methods.

[0017] Other features and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0018] Figure 1 This is a flowchart of the hallucination suppression method for intelligent agents based on structured cue words in an embodiment of the present invention; Figure 2 This is a schematic diagram of the organization structure of multi-dimensional structured prompt words in an embodiment of the present invention; Figure 3 This is a schematic diagram of the intermediate result verification and hallucination detection process in an embodiment of the present invention; Figure 4 This is a schematic diagram of the hallucination correction and iterative optimization process in an embodiment of the present invention; Figure 5 This is a modular framework diagram of the hallucination suppression system for a program analysis intelligent agent based on structured prompt words in an embodiment of the present invention. Detailed Implementation

[0019] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0020] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0021] Terminology Explanation: Program analysis refers to the process of examining and reasoning about software source code or binary code using automated techniques. Its purpose is to discover potential defects, security vulnerabilities, performance issues, or verify whether the program meets specific correctness attributes. The content analyzed in program analysis includes: the syntax structure of the code, the definition and usage of variables, the control flow path of the program, the way data is passed within the program, and whether the code conforms to safe coding standards.

[0022] Model illusion refers to the phenomenon where large language models generate outputs that are inconsistent with reality, contain logical errors, or are fabricated out of thin air, leading to biased prediction results. For example, in program analysis scenarios, common model illusions include: code logic illusion, identifier illusion, knowledge illusion, and reasoning illusion. Code logic illusion refers to the model's misunderstanding of the code's execution logic, generating analysis conclusions that do not match actual code behavior. Identifier illusion refers to the model fabricating non-existent variable names, function names, or class names, causing the analysis results to reference fictitious code elements. Knowledge illusion refers to the model generating conclusions unsuitable for the current code analysis scenario based on outdated or erroneous knowledge from the training data. Reasoning illusion refers to the model accumulating errors during multi-step reasoning, causing the final analysis results to deviate significantly from reality.

[0023] Example 1 In one or more embodiments, a method for suppressing hallucinations in a program analysis agent based on structured prompt words is disclosed. Taking C language memory security vulnerability detection as an example, this method is applied to a code vulnerability detection scenario, combined with... Figure 1 Specifically, it includes the following processes: S101: Extract the structured feature information of the code to be analyzed.

[0024] In this embodiment, the input code to be analyzed is preprocessed to extract the structured feature information of the code and construct a multi-level code representation model.

[0025] The process of extracting structured feature information specifically includes: (1) Lexical analysis layer processing: Perform lexical analysis on the source code, extract lexical units such as identifiers, keywords, operators, and literals, and construct lexical feature vectors; As a specific implementation method, lexical analysis is performed using the Clang frontend (an open-source C / C++ / Objective-C compiler frontend) to extract all identifiers (e.g., variable names, function names, type names), keywords (e.g., if (conditional branch), while (conditional loop), for (counting loop), return (function return), etc.), operators (e.g., +, -, *, / , =, &, etc.), and literals (integers, floating-point numbers, strings). For memory safety analysis, keywords related to memory operations are specifically marked, such as malloc (memory allocation), free (delete), realloc (reallocate / copy), memcpy (copy), strcpy (string copy function), etc.

[0026] (2) Syntax structure layer processing: Construct an abstract syntax tree (AST) and extract syntactic features such as function definition, class definition, control flow structure, and expression structure; As a specific implementation method, an abstract syntax tree is constructed to extract syntactic features such as function definitions (function name, parameter list, return type), variable declarations (variable name, type, initial value), and control flow structures (conditional branches, loops, jumps). For memory safety analysis, particular attention is paid to pointer declarations, array definitions, and dynamic memory allocation statements.

[0027] (3) Semantic information layer processing: Perform data flow analysis and control flow analysis to extract semantic features such as variable definition-usage chains, function call relationships, and dependency graphs; As a specific implementation method, the LLVM (Low-Level Virtual Machine, a compiler infrastructure) static analysis framework is used for data flow analysis, constructing a variable definition-use chain, identifying possible pointer sets (points-to analysis), and extracting function call graphs and inter-procedural dependencies. For memory safety analysis, the focus is on memory allocation-deletion pairings, pointer lifetimes, and array boundary information.

[0028] (4) Contextual information layer processing: Collect natural language description information such as code comments, docstrings, and naming conventions, and extract developer intent features; As a specific implementation method, comments, function docstrings, variable naming conventions, etc., are extracted from the source code, and natural language processing technology is used to parse the semantic information, such as memory ownership transfer instructions and buffer size conventions.

[0029] S102: Based on the extracted structured features, use XML (Extensible Markup Language) format tagging technology to construct multi-dimensional structured prompting words.

[0030] In the prior art, the design of prompting words for large language models usually adopts the method of natural language description, directly inputting the task requirements and the content to be analyzed in an unstructured text form into the model; such unstructured prompting words often have problems such as loose information organization and lack of a clear hierarchical structure, and the key features of the code are not explicitly extracted and presented, unable to effectively capture the key features of the code; moreover, there is a lack of a constraint and verification mechanism for the model output.

[0031] In this embodiment, structured prompting words are constructed from multiple dimensions such as task instructions, code content, structural features, context enhancement, and verification constraints, and the lexical features, syntactic features, semantic features, and developer intention features of the code are orderly organized in the form of XML format tagging, significantly improving the accuracy of the large language model's understanding of the code structure and semantics, and reducing the probability of hallucinations from the source.

[0032] Specifically, the structured prompting words include: task instruction block, code content block, structure feature block, context enhancement block, and verification constraint block; combined Figure 2 , the content of each part of the component is as follows: (1) Task Instruction Block: Encapsulated with the <task (task)> tag, containing information such as the analysis task type identifier, target description, output format requirements, constraint conditions, etc.; (2) Code Content Block: Encapsulated with the <code (code)> tag, containing the original text of the code to be analyzed, and annotating meta-information such as the code language, version, file path, etc. through the <metadata (metadata)> sub-tag; (3) Structure Feature Block: Encapsulated with the <structure (structure)> tag, containing structured analysis results such as AST summary, control flow graph summary, data flow graph summary, etc.; (4) Context Enhancement Block: Encapsulated with the <context (context)> tag, containing auxiliary information such as relevant code snippets, API documentation, historical analysis records, etc.; (5)Validation Constraint Block: Encapsulated with the <constraint> tag, it contains validation rules such as type constraints, value range constraints, and logical consistency constraints.

[0033] The structured prompt words constructed in this embodiment can improve the model's understanding accuracy, clearly defining the boundaries of different types of information through XML tags; explicitly providing preprocessing results such as AST summaries, control flow graph summaries, and data flow graph summaries, the validation constraint block presets the rules that the output must satisfy, and the hierarchical nested structure matches the hierarchical structure of the code itself.

[0034] S103: Input the structured prompt words into the large language model for analysis, and perform multi-stage intermediate result verification on the analysis results of the large language model.

[0035] In this embodiment, the large language model can be selected from GPT series (generative pre-trained models), Claude series (alignment models based on constitutional AI), or Llama series (open-source base language models), etc.

[0036] The intermediate result verification mechanism of this embodiment adopts an automated verification method based on a program analysis tool chain, combined with Figure 3 , and the specific verification process is as follows: (1)Syntax consistency verification: Verify whether the code elements (variable names, function names, class names, etc.) involved in the model output exist in the original code; Specifically, syntax consistency verification can be implemented through a symbol table validator, that is, based on the symbol table constructed by the compiler front end, verify the validity of the identifiers in the model output; specifically, first, use the compiler front end to perform lexical analysis and syntax analysis on the source code to construct a symbol table, which records all identifiers and their attribute information in the source code. Then, extract all the identifiers involved from the output text of the large language model and look them up and match them one by one in the symbol table.

[0037] The condition for passing the verification is that each code element mentioned in the model output can find a corresponding record in the symbol table.

[0038] The situations of verification failure include: (1-1)There are variable names or function names in the model output that do not exist in the source code; (1-2)The line numbers of the code referenced by the model exceed the actual range of the source code; (1-3)The type of the code element described by the model does not match the record in the symbol table.

[0039] (2) Type consistency verification: Verify whether the type information inferred by the model is consistent with the results of static type analysis; Specifically, type consistency verification can be performed through a type checker, which verifies the correctness of the type information inferred by the model based on the type inference engine.

[0040] First, static type analysis tools (such as the Clang static analyzer) are used to perform type inference on the source code, constructing a type information table that records the type information of each variable, expression, and function. Then, type assertions (such as "variable x is a pointer type" and "function f returns an integer") are extracted from the output of the large language model, and these assertions are compared with the records in the type information table.

[0041] The validation is successful if all descriptions of types in the model output are consistent with the static type analysis results.

[0042] Verification failure scenarios include: (2-1) The model claims that a certain variable is a pointer type, but static analysis shows that the variable is an integer type or other non-pointer type; (2-2) The return type of the function inferred by the model does not match the return type in the function declaration; (2-3) The type conversion operation described in the model is illegal or does not have an implicit conversion path in the C language specification; (2-4) The model claims that the type of a certain expression is inconsistent with the actual inferred type.

[0043] For example, if the model output claims that "the variable size is an unsigned integer type", but the source code declares size as int size (a signed integer), then the type consistency verification fails.

[0044] (3) Control flow consistency verification: Verify whether the execution path described by the model exists in the control flow graph; Specifically, control flow consistency verification can be performed using a path validator. First, a control flow graph (CFG) is constructed based on the source code, where nodes represent basic code blocks (i.e., sequentially executed statement sequences), and directed edges represent possible control transfer relationships (such as conditional branches, loop jumps, function calls, etc.). Then, the execution path described by the model (e.g., "entering the code block on line 5 when the condition on line 3 is true") is mapped to a sequence of nodes on the control flow graph, and a graph traversal algorithm (such as depth-first search) is used to verify whether the path is reachable.

[0045] Optionally, in this step, constraints on the execution path (such as loop boundary conditions, branch decision conditions, etc.) can be further collected, and symbolic execution techniques can be used to verify the feasibility of the path. Specifically, the constraints on the path are encoded into logical formulas, and the Z3 solver (a type of SMT constraint solver) is used to calculate whether the formulas can be satisfied; if the constraints cannot be satisfied, it means that the path cannot be triggered in actual execution.

[0046] The conditions for successful verification are: the execution path described by the model exists in the control flow graph, and the constraints on the path can be satisfied (if symbolic execution verification is enabled).

[0047] Verification failure scenarios include: (3-1) The execution order of the code blocks described by the model does not have corresponding directed edges connected in the control flow graph, i.e., the path is unreachable; (3-2) The model claims that a certain code block will be executed, but the code block is dead code (unreachable node) in the control flow graph. (3-3) The number of times the loop described by the model is executed violates the loop boundary conditions (e.g., claiming that the loop is executed a negative number of times). (3-4) When symbolic execution verification is enabled, the constraints on the path are determined by the Z3 solver to be unsatisfactory, indicating that the path conditions are contradictory and cannot actually be triggered.

[0048] For example, if the model output claims that "when size is negative, the program will execute the memcpy operation on line 4", but the memcpy operation on line 4 is located inside the if (size>0) conditional branch in the source code, then the control flow graph shows that the path is unreachable, and the control flow consistency verification is determined to be a failure.

[0049] (4) Data flow consistency verification: Verify whether the data transmission relationship described by the model is consistent with the data flow analysis results; Specifically, data flow consistency verification can be performed using a data flow validator. First, the source code is defined using a data flow analysis tool – chain analysis is used to record where each variable is defined (assigned), where it is used (read), and how data is passed between variables. Then, descriptions of data flow are extracted from the model output (such as "the value of variable a is passed to variable b," "the data input to the function parameter flows to the buffer," etc.), and these descriptions are compared with the data flow analysis results.

[0050] The validation is successful if the data transfer relationship described by the model is consistent with the definition-chain analysis results, meaning that the source and destination of the data can be found in the analysis results.

[0051] Verification failure scenarios include: (4-1) The model claims that the value of variable a comes from variable b, but the definition - using chain analysis shows that the definition point of a is independent of b; (4-2) There are breakpoints in the data transmission path described by the model, that is, there is no corresponding usage record for a certain variable in the transmission chain; (4-3) The model claims that a certain variable is used at a specific location, but that location is actually the definition point of the variable, not the usage point; (4-4) The cross-function data transfer relationship described by the model does not match the results of the inter-process data flow analysis.

[0052] For example, if the model output claims that "the data of the target buffer in line 4 (memcpy) comes from the global variable", but the data flow analysis shows that the target buffer was not allocated with the global variable after line 2 (malloc), then the data flow consistency verification is deemed to have failed.

[0053] (5) Logical consistency verification: Verify whether the logical relationships between each step in the model's reasoning chain are self-consistent; Specifically, logical consistency verification can be performed using a logical reasoning verifier. First, the reasoning chain is extracted from the model output; that is, the complete process by which the model, starting from observed code features, goes through several reasoning steps to arrive at an analytical conclusion. Then, each step in the reasoning chain is formalized as a logical proposition, and the logical implication relationship between adjacent steps is examined to see if it holds, and whether there are any logical contradictions in the entire reasoning chain.

[0054] The conditions for successful verification are: the logical relationships between each step in the model's reasoning chain are self-consistent, that is, the premises can correctly lead to the conclusion, and there are no internal contradictions in the entire reasoning process.

[0055] Verification failure scenarios include: (5-1) The reasoning in the model has logical jumps, that is, it draws the conclusion C directly from the premise A, but there is no valid logical implication relationship between A and C; (5-2) The reasoning chain of the model contains internal contradictions, that is, it makes contradictory assertions about the same fact in different steps; (5-3) The conclusion of the model is incompatible with the premises, that is, under the given premises, the conclusion cannot be logically valid; (5-4) The model uses incorrect inference rules, such as misusing sufficient conditions as necessary conditions, or the sample is insufficient to support the conclusion in inductive reasoning.

[0056] For example, if the model output claims "because malloc may return NULL (null pointer), null pointer dereferencing will definitely occur in line 4," this reasoning contains a logical error—malloc returning NULL is a possible condition, and cannot lead to the necessary conclusion that it "will definitely occur." The correct reasoning should be "when malloc returns NULL, null pointer dereferencing may occur in line 4." In this case, the logical consistency verification fails, and the model's reasoning statement needs to be corrected.

[0057] As a concrete example, after the large language model generates analysis results based on structured prompts, it triggers an intermediate result verification process. Assume the model outputs the following analysis results: Test results: 1. Buffer overflow vulnerability: Location: Line 4, memcpy call; Trigger condition: When size > 100; Recommended fix: Add boundary checks for sizes <= 100; 2. Memory leak: Location: buffer variable; Triggering condition: free was not called when the function ended; Recommended fix: Add `free(buffer)` before the function returns.

[0058] The verification engine performs multi-dimensional verification on the above output: (1) Syntax consistency verification: Verify that identifiers such as memcpy, buffer, and size exist in the source code - pass.

[0059] (2) Type consistency verification: Verify whether the type of size is int (integer) and whether the type of buffer is char* (character pointer) - pass.

[0060] (3) Control flow consistency verification: Verify whether memcpy in line 4 is indeed executed within the conditional branch - pass.

[0061] (4) Data flow consistency verification: Verify whether the buffer is defined in line 2 and used in line 4 - pass.

[0062] (5) Logical consistency verification: Verify the logical reasoning that “size>100 causes buffer overflow” – pass (buffer size is 100 bytes, memcpy copies size bytes, and overflow does occur when size>100).

[0063] Once all validations pass, the analysis result is marked as a reliable output.

[0064] This embodiment achieves multi-dimensional automated verification of model output through an intermediate result verification mechanism based on a program analysis toolchain, enabling timely detection and accurate location of hallucination information.

[0065] S104: Based on the verification results, locate the source of the illusion and construct correction prompts to guide the large language model to perform self-correction.

[0066] In this embodiment, when a potential hallucination is detected in step S103, a hallucination correction mechanism is triggered, combined with... Figure 4 The specific process is as follows: S1041: Locating the source of hallucination: Based on the specific type of verification failure, locate the information fragment that caused the hallucination; S1042: Construct correction prompts: Encapsulate hallucination information, verification failure reasons, and correct reference information into correction prompts; Among them, the illusion information is extracted from the specific content fragments of the verification failure in the verification process of step S103; the reason for the verification failure is automatically generated by the verifier, which verification check the output fragment failed to pass and the specific inconsistencies; the correct reference information is extracted from the intermediate results of the program analysis toolchain, including the correct identifier list in the symbol table, the correct type information obtained from static type analysis, and the actual reachable path in the control flow graph.

[0067] S1043: Trigger model self-correction: Input correction prompts into the large language model to guide the model to correct erroneous outputs; S1044: Iterative Verification: Repeat the verification process of step S103 on the corrected output until all verifications are passed or the maximum number of iterations is reached. If all verifications are passed, output the corrected result; otherwise, return to S1041 to relocate the illusion source and repeat the process of S1041-S1044 until all verifications are passed; if all verifications are still not passed after reaching the preset maximum number of iterations, mark the analysis results generated by the large language model as low confidence.

[0068] As a concrete example, suppose the model output contains the following hallucination information: 1. Dereferencing a null pointer: Location: Line 4, memcpy call; Triggering condition: When malloc returns NULL; The verification engine discovered an issue during the logical consistency verification phase: while malloc can indeed return NULL, the model's claim that a null pointer dereference occurs on line 4 is inaccurate because memcpy on line 4 dereferences the input parameter (the target buffer is buffer), rather than directly dereferencing buffer. Further analysis of the internal behavior of memcpy is needed.

[0069] Therefore, the system constructs the following correction prompt words: <correction>; <error_type> Logical reasoning error< / error_type> ; <error_content> It is claimed that line 4 contains a null pointer dereference.< / error_content> ; <error_reason> In `memcpy(buffer, input, size)`, `buffer` is used as the target address. If `buffer` is NULL, `memcpy` will write data to NULL, which is indeed incorrect. However, it should be clearly stated that it is "writing to a null pointer" and not "dereferencing".< / error_reason> ; <correct_reference> The memcpy function signature is: void *memcpy(void *dest, const void*src, size_t n). Its behavior is undefined when dest (the first parameter of the memcpy function) is NULL.< / correct_reference> ; <correction_instruction> Please re-analyze the potential problems when malloc returns NULL, and accurately describe the problem type.< / correction_instruction>< / correction> .

[0070] in, <correction>(Correction): The root tag of the correction prompt word indicates that this is a prompt word used to guide the large language model to correct erroneous output; <error_type> (Error Type): Indicates the category of error in the model output, such as "Logical Reasoning Error", "Identifier Error", "Type Error", etc.; <error_content> (Error content): Specifically indicates the content segment in the model output that contains errors, i.e. the original text that needs to be corrected; <error_reason> (Reason for error): Explain why the output is incorrect, and describe the specific reasons and technical details of the error; <correct_reference> (Correct Reference Information): Provide correct information related to the error as a reference, such as standard function signatures, correct type definitions, and actual code behavior; <correction_instruction> (Correction Instruction): Provides clear correction instructions to guide the large language model to re-analyze in the correct direction and generate corrected output.

[0071] After receiving correction prompts, the large language model regenerates more accurate analysis results: 2. Null pointer write (undefined behavior): Location: Line 4, memcpy call; Triggering condition: When malloc returns NULL and size > 0; Recommended fix: Add a NULL check before using the buffer.

[0072] This embodiment achieves automatic correction of hallucinations through a feedback-guided iterative correction strategy, thereby improving the system's adaptability and output reliability.

[0073] S105: Aggregate the validated analysis results to generate the final program analysis report.

[0074] The program analysis report includes information such as analysis conclusions, confidence scores, and supporting evidence chains.

[0075] The analysis conclusion refers to the final judgment result given by the large language model to the code to be analyzed. In the vulnerability detection scenario, it includes: the type of vulnerability detected, the location information of the vulnerability, the triggering conditions of the vulnerability, and remediation suggestions.

[0076] The supporting chain of evidence refers to the reasoning process and basis that supports the conclusion of the analysis, and consists of the following information: (1) The code snippets referenced by the model and their locations; (2) The reasoning steps of the model; (3) Records that have passed verification.

[0077] The supporting chain of evidence is automatically generated by recording the reasoning process of the model and associating it with the verification results.

[0078] Confidence scoring assesses the likelihood that the analysis conclusions match the actual situation in the code. In existing technologies, the confidence score of large language model outputs is usually calculated based on the model's own probability distribution. However, this method only reflects the model's confidence in its own output and cannot truly measure the consistency between the output content and objective facts.

[0079] The confidence scoring mechanism in this embodiment uses external validation results rather than the model's internal probability to calculate confidence. This confidence scoring based on objective validation can directly reflect the degree of agreement between the analysis results and the actual code situation. Different validation dimensions are weighted to reflect the relative importance of various validations, thereby providing users with interpretable reliability indicators.

[0080] Specifically, in this embodiment, the confidence score uses a weighted calculation method based on the verification pass rate, and the calculation formula is as follows: ; in, For the first i The status of the verification is indicated by 1 for successful verification and 0 for failed verification. For the first i The weights of the verification items are pre-set according to the importance of the verification type, i=1,2,3,4,5; which respectively refer to the aforementioned five verification items.

[0081] As a specific example, the generated vulnerability detection report is as follows: === Memory Security Vulnerability Detection Report === File: src / memory_handler.c (source code path); Function: process_data(function_name); Detected vulnerabilities: 1. [High Risk] Buffer overflow (Confidence: 95%); 2. [Medium Risk] Memory Leak (Confidence Level: 90%); 3. [Medium Risk] Null pointer write (Confidence: 85%); Verification status: All 5 consistency verifications have passed.

[0082] This embodiment provides quantifiable reliability indicators for the analysis results through a confidence scoring mechanism, which facilitates user evaluation and decision-making.

[0083] This embodiment proposes a hallucination suppression method for program analysis agents based on structured cue words. It effectively captures key features of code by designing multi-dimensional structured cue words, employs XML markup technology to organize lexical, syntactic, and semantic information of the code in an orderly manner, and establishes an intermediate result verification mechanism based on a program analysis toolchain. This fully utilizes the verifiability of program analysis results for hallucination detection and correction, thereby effectively solving the hallucination problem of large language models in program analysis tasks. The overall solution is specifically designed for program analysis scenarios, fully leveraging the structured features of code and the verification capabilities of program analysis tools, and exhibits better performance in program analysis tasks compared to general hallucination suppression methods.

[0084] Example 2 In one or more embodiments, a program analysis intelligent agent hallucination suppression system based on structured cue words is disclosed, combined with Figure 5 Specifically, it includes: (1) Feature extraction module, configured to extract structured feature information of the code to be analyzed; specifically including: (1-1) Lexical Analysis Submodule: Based on the compiler front-end, it is responsible for converting the source code into a lexical unit stream and extracting information such as identifiers, keywords, operators, and literals; (1-2) Syntax Analysis Submodule: Responsible for constructing the abstract syntax tree and extracting the definitions and reference relationships of program entities such as functions, classes, and variables; (1-3) Semantic Analysis Submodule: Responsible for performing semantic-level analysis such as type checking, data flow analysis, and control flow analysis, and generating intermediate representations such as symbol tables, call graphs, and dependency graphs; (1-4) Context Extraction Submodule: Responsible for extracting natural language information such as comments and docstrings in the code and performing semantic parsing.

[0085] (2) The prompt word construction module is configured to construct multi-dimensional structured prompt words based on extracted structured features using XML format markup technology; specifically including: (2-1) Template Management Submodule: Maintains a library of prompt word templates for different analysis tasks (vulnerability detection, code optimization, defect localization, etc.); (2-2) XML generation submodule: responsible for marking and assembling task instructions, code content, structural features, context information and validation constraints according to XML format; respectively obtaining task instruction blocks, code content blocks, structural feature blocks, context enhancement blocks and validation constraint blocks; (2-3) Prompt word optimization submodule: responsible for optimizing the length of generated prompt words, deduplicating information, and standardizing the format.

[0086] (3) Large Language Model Interface Module, responsible for interacting with the large language model, specifically including: (3-1) API Adapter: Supports unified access to multiple large language model APIs (such as OpenAI GPT series, Anthropic Claude series, open source LLaMA series, etc.); (3-2) Request Builder: Responsible for encapsulating structured prompts into API request format; (3-3) Response parser: responsible for parsing the response content returned by the model and extracting the analysis results.

[0087] (4) The analysis and verification module is configured to input structured prompt words into a large language model for analysis and to perform multi-stage intermediate result verification on the analysis results of the large language model; specifically including: (4-1) Symbol table verifier: Verifies the validity of identifiers in the model output based on the compiler's symbol table; (4-2) Type checker: Verifies the correctness of type information inferred by the type system model; (4-3) Path verifier: Verifies the reachability of execution paths described by the control flow graph model; (4-4) Data Flow Validator: Verifies the data transfer relationships described by the model based on the data flow analysis results; (4-5) Logical reasoning verifier: Verify the logical self-consistency of the model reasoning chain based on formal methods.

[0088] (5) The hallucination correction module is configured to locate the hallucination source based on the verification results and construct correction prompts to guide the large language model to perform self-correction; specifically including: (5-1) Error locator: Locates the specific location where the hallucination occurred based on the verification failure information; (5-2) Correction prompt word generator: Generates correction prompt words based on error type and correct reference information; (5-3) Iterative controller: controls the number of correction iterations to avoid infinite loops.

[0089] (6) The results aggregation module is configured to aggregate the validated analysis results and generate the final program analysis report, specifically including: (6-1) Results Merger: Merges and removes duplicates from the results of multiple rounds of analysis; (6-2) Confidence calculator: Calculates the confidence level of each result based on the validation pass rate and the consistency of the model output; (6-3) Report Generator: Generates structured analysis reports according to predefined formats.

[0090] It should be noted that the specific working method of the above modules has been described in detail in Embodiment 1, and will not be repeated here.

[0091] Example 3 In one or more embodiments, a terminal device is disclosed, comprising a processor and a memory, the processor being used to implement instructions; the memory being used to store multiple instructions adapted to be loaded by the processor and executed by the processor for the program analysis intelligent agent hallucination suppression method based on structured cue words in Embodiment 1.

[0092] It should be understood that in this embodiment, the processor can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0093] Memory may include read-only memory and random access memory, and provides instructions and data to the processor. A portion of memory may also include non-volatile random access memory. For example, memory may also store information about the device type.

[0094] In the implementation process, each step of the above method can be completed by the integrated logic circuits in the processor hardware or by software instructions.

[0095] Example 4 In one or more embodiments, a computer-readable storage medium is disclosed, wherein a plurality of instructions are stored, the instructions being adapted to be loaded by a processor of a terminal device and executed by the program analysis agent hallucination suppression method based on structured cue words in Embodiment 1.

[0096] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.< / correction>

Claims

1. A method for suppressing hallucinations in a program analysis agent based on structured cue words, characterized in that, include: Extract the structured feature information of the code to be analyzed, including lexical features, syntactic features, semantic features, and developer intent features; Based on the extracted structured features, multi-dimensional structured prompt words are constructed; The structured prompts include: task instruction blocks, code content blocks, structural feature blocks, context enhancement blocks, and validation constraint blocks; The structured prompt words are input into a large language model for analysis, and the analysis results of the large language model are verified through multiple stages of intermediate results. Based on the verification results, the source of the hallucination is located, and correction prompts are constructed to guide the large language model to perform self-correction. The validated analysis results are aggregated to generate the final program analysis report.

2. The method for suppressing hallucinations in a program analysis agent based on structured cue words as described in claim 1, characterized in that, The lexical features include identifiers, keywords, operators, and literals; The syntactic features include function definitions, class definitions, control flow structures, and expression structures; The semantic features include variable definition-usage chains, function call relationships, and dependency graphs; The developer intent features include code comments, docstrings, and natural language descriptions of naming conventions.

3. The method for suppressing hallucinations in a program analysis agent based on structured cue words as described in claim 1, characterized in that, The task instruction block includes the analysis task type identifier, target description, output format requirements, and constraints. The code content block includes the original text of the code to be analyzed, and is annotated with code metadata; The structural feature blocks include the structured analysis results of the AST summary, control flow graph summary, and data flow graph summary; The context enhancement block includes relevant code snippets, API documentation, and historical analysis records. The verification constraint block includes multiple verification rules.

4. The method for suppressing hallucinations in a program analysis agent based on structured cue words as described in claim 1, characterized in that, The analysis results of the large language model are validated through multiple stages of intermediate results, including: Syntax consistency verification is used to verify whether the code elements involved in the output of a large language model exist in the original code; Type consistency verification is used to verify whether the type information inferred by the large language model is consistent with the results of static type analysis. Control flow consistency verification is used to verify whether the execution path described by the large language model exists in the control flow graph; Data flow consistency verification is used to verify whether the data transmission relationships described by the large language model are consistent with the data flow analysis results. Logical consistency verification is used to verify whether the logical relationships in the reasoning chain of a large language model are self-consistent.

5. The method for suppressing hallucinations in a program analysis agent based on structured cue words as described in claim 4, characterized in that, Based on the specific type of verification failure, the information fragment that caused the hallucination is located, and the hallucination information fragment, the reason for the verification failure, and the correct reference information are encapsulated into correction prompt words. The correction prompt words are then input into the large language model to guide the large language model to correct the erroneous output.

6. The method for suppressing hallucinations in a program analysis agent based on structured cue words as described in claim 4, characterized in that, If all validations pass, the analysis results are aggregated to generate a program analysis report. The program analysis report includes at least the analysis conclusions, confidence scores, and supporting evidence chain information, wherein the confidence scores are calculated based on a weighted average of the validation pass rates.

7. The method for suppressing hallucinations in a program analysis agent based on structured cue words as described in claim 6, characterized in that, The confidence score is calculated as follows: ； in, For the first i The status of the verification is indicated by 1 for successful verification and 0 for failed verification. For the first i The weights of each validation item are pre-set based on the importance of the validation type.

8. A hallucination suppression system for a program analysis intelligent agent based on structured cue words, characterized in that, include: The feature extraction module is configured to extract structured feature information of the code to be analyzed, including lexical features, syntactic features, semantic features, and developer intent features. The prompt word construction module is configured to construct multi-dimensional structured prompt words based on extracted structured features; The structured prompts include: task instruction blocks, code content blocks, structural feature blocks, context enhancement blocks, and validation constraint blocks; The analysis and verification module is configured to input structured prompt words into a large language model for analysis and to perform multi-stage intermediate result verification on the analysis results of the large language model. The hallucination correction module is configured to locate the hallucination source based on the verification results and construct correction prompts to guide the large language model to perform self-correction. The results aggregation module is configured to aggregate validated analysis results to generate a final program analysis report.

9. A terminal device comprising a processor and a memory, the processor for implementing instructions; the memory for storing multiple instructions, characterized in that, The instructions are adapted to be loaded by a processor and executed as described in any one of claims 1-7, which are methods for suppressing hallucinations in intelligent agents based on structured cue words.

10. A computer-readable storage medium storing a plurality of instructions, characterized in that, The instructions are adapted to be loaded by the processor of a terminal device and executed as described in any one of claims 1-7, the method for suppressing hallucinations in intelligent agents based on structured cue words.

Images