Code illusion detection and positioning method based on hidden layer vector mapping

Abstract

The invention belongs to the technical field of software system security, and discloses a hidden layer vector mapping-based code illusion detection and positioning method, which comprises the following steps of: preprocessing a code illusion detection data set, and analyzing a code sample in the preprocessed code illusion detection data set to obtain an original abstract syntax tree; encoding the original abstract syntax tree through the trained code generation model to generate hidden representation, and judging whether illusion exists in the code sample or not based on the hidden representation; mapping the hidden representation of the original abstract syntax tree to a syntax subspace through a probe network to obtain a vector tuple, and reconstructing the vector tuple into a predicted abstract syntax tree; and comparing differences between the predicted abstract syntax tree and the original abstract syntax tree, and identifying and locating code lines related to code illusion. The method is suitable for accurately positioning the specific illusion code line while detecting the illusion, the positioning accuracy is improved, and powerful technical support is provided for the field of code security.

Description

Technical Field

[0001] This invention belongs to the field of software system security technology and relates to a method for detecting and locating code illusions based on hidden layer vector mapping. Background Technology

[0002] In recent years, with the rapid development of artificial intelligence technology, Large Language Models (LLMs) have shown enormous potential in code generation. AI models such as Copilot and Cursor have become important tools for developers, automatically generating source code according to needs and greatly improving development efficiency. However, code generated by AI models often exhibits a "code illusion" phenomenon, failing to meet real-world requirements. Code illusion refers to the phenomenon where code generated by AI models may resemble normal code in syntax and style, but hides potential problems. This not only impairs the reliability and usability of the code but also reduces people's trust in AI tools. Code illusion is the most common problem in AI model-generated code; previous research indicates that the average illusion rate of mainstream LLMs in code generation is as high as 20% to 60%.

[0003] Current research approaches for large-scale model code generation illusion detection include:

[0004] 1) Agarwal et al. proposed a research method for detecting and analyzing code illusions in large language models based on execution verification and a classification system. Code illusion, as defined in this paper, refers to the phenomenon where LLM-generated code is syntactically correct or semantically reasonable but ultimately fails to execute as expected or meet specified requirements, indicating functional defects. The paper obtains a data stream of illusion code through execution verification and syntactic-semantic parsing of LLM-generated code, and constructs a benchmark based on the correspondence between programming problems and test cases in the HumanEval and MBPP datasets. The experimental design employs a multi-stage verification method: first, 1,137 Python code samples are generated using GPT-3.5 to construct the CodeMirage benchmark dataset, covering five categories: syntactic illusion, semantic illusion, functional illusion, security illusion, and performance illusion; then, a one-shot strategy is used to test the illusion detection capabilities of CodeLLaMA, GPT-3.5, and GPT-4, ensuring detection accuracy through a triple verification mechanism of execution verification, static analysis, and manual annotation; finally, a horizontal comparative analysis is conducted with the CodeBERT baseline model. Experimental results show that GPT-4 has the highest accuracy in hallucination detection on the HumanEval dataset and performs comparably to the CodeBERT baseline on the MBPP dataset. At the same time, significant differences were found in the distribution of different types of hallucinations, providing an important reference for subsequent improvements in code generation quality.

[0005] 2) Jiang et al. proposed a benchmark study on large-scale code illusion prediction based on execution verification and machine learning. They obtained the illusion code data stream by executing and verifying LLM-generated code and extracting multi-dimensional features. A benchmark was constructed based on the correspondence between problem descriptions and test cases in code generation (CG) and automated program repair (APR) tasks. Different schemes then executed different processes. The experiment used stratified sampling of 13,234 illusion instances to ensure uniform distribution across LLMs and task types. Traditional machine learning algorithms such as logistic regression, random forest, and support vector machines were used for training. Simultaneously, multi-dimensional feature vectors were input into a multilayer perceptron and a deep neural network, trained using the Adam optimizer (lr=0.001), ReLU activation function, Dropout (0.3) regularization, and cross-entropy loss function. Cross-validation and Bayesian optimization were used to ensure model performance. Experimental results show that the prediction accuracy of traditional machine learning and neural network methods reaches 22.03-33.15% on 5 datasets and 11 different LLMs' code illusion instances. The effectiveness of the methods is verified by statistical significance test (p<0.05). Meanwhile, confusion matrix analysis and feature importance study reveal the complexity of code illusion prediction and the challenge of accurately locating LLMs' illusions.

[0006] 3) Tian et al. obtained the illusion code data stream through execution verification and statistical inductive analysis of LLM-generated code. They constructed a detection framework based on the correspondence between problem descriptions and test cases in the code generation dataset, and then executed different processes for different schemes. The experimental design employed the CodeHalu dynamic detection algorithm to perform statistical quantitative analysis on 17 mainstream LLMs on the complex APPS dataset, identifying and verifying 18 illusion state patterns that violate human expectations. Simultaneously, code samples were collected, and illusion detection was performed using execution verification combined with a two-stage heuristic identification method. Frequently occurring error types, syntax breaks, and unexpected execution results were identified through statistical induction, constructing an illusion type knowledge base. The detected code illusions were classified into four major categories according to their phenomenon nature and origin: Mapping illusion, Naming illusion, Resource illusion, and Logic illusion. These categories were further subdivided into eight subcategories, and the independence and effectiveness of the classification were verified through cross-task occurrence rate analysis. Experimental results show that the constructed CodeHaluEval benchmark contains 8,883 samples from 699 different tasks. The systematic evaluation of 17 popular LLMs reveals significant differences in their code generation accuracy and reliability, confirms the independence of the code illusion classification system, and provides detailed insights and theoretical basis for further improving the code generation capabilities of LLMs.

[0007] The aforementioned research on code generation illusion detection in large-scale models mainly focuses on the classification and recognition of illusions. It can only determine whether there are illusion problems in the code at an overall level, but cannot precisely locate the specific location of the illusion. This forces developers to manually review the code line by line to find the illusion location, and makes subsequent automated repair difficult, failing to meet the practical engineering needs of large-scale code review scenarios. Therefore, this paper proposes a code illusion detection and localization method based on hidden layer vector mapping. This method can accurately locate the specific line of code causing the illusion while detecting it, providing a reference for subsequent automated repair and manual review. Summary of the Invention

[0008] The purpose of this invention is to provide a method for detecting and locating code illusions based on hidden layer vector mapping, so as to detect code illusions and accurately locate specific lines of illusion code.

[0009] To achieve the above objectives, the technical solution adopted by this invention is as follows:

[0010] A method for detecting and locating code illusions based on hidden layer vector mapping includes the following steps:

[0011] S1. Data preprocessing: Obtain code samples to form a code illusion detection dataset, preprocess the code samples in the code illusion detection dataset, and use a syntax parser to parse the code samples in the preprocessed code illusion detection dataset to obtain the original abstract syntax tree O-AST.

[0012] S2. Code Illusion Detection: The original abstract syntax tree (O-AST) is encoded by the trained code generation model to generate a hidden representation containing semantic information. Based on the hidden representation, it is determined whether there is an illusion in the code sample.

[0013] S3. Probe AST Mapping and Reconstruction: The hidden representation of the original abstract syntax tree O-AST is mapped to the syntax subspace through a probe network to obtain vector tuples, and the vector tuples are reconstructed into the predicted abstract syntax tree P-AST.

[0014] S4. Code Illusion Localization: Compare the differences between the predicted abstract syntax tree (P-AST) and the original abstract syntax tree (O-AST) to identify and locate the lines of code associated with code illusion.

[0015] As a limitation, in step S1, each code sample in the code illusion detection dataset contains source code, label, and illusion line index fields.

[0016] As a further limitation, in step S1, the process of using a syntax parser to parse the code samples in the preprocessed code illusion detection dataset to obtain the original abstract syntax tree (O-AST) includes:

[0017] A syntax parser is used to parse the code samples in the preprocessed code illusion detection dataset into an abstract syntax tree (AST), and the AST is checked to see if it is a valid tree structure, while invalid tree structures are filtered out.

[0018] Check if the Abstract Syntax Tree (AST) contains erroneous nodes and filter out erroneous nodes;

[0019] The abstract syntax tree (AST) is segmented using a lexical analyzer, converted into token sequences, the number of token sequences is counted, and ASTs exceeding the maximum length limit are filtered out.

[0020] Extract all token sequences and their positions in the source code from the filtered Abstract Syntax Tree (AST) to obtain the original Abstract Syntax Tree (O-AST);

[0021] The location information includes the start byte, the end byte, and the line number.

[0022] As a second limitation, step S2 includes the following process:

[0023] The preprocessed code illusion detection dataset is divided into a training set, a validation set, and a test set according to a preset ratio of 8:1:1.

[0024] A code generation model based on the Transformer architecture is built. The code generation model is fine-tuned and trained using a training set, validated using a validation set, and evaluated using a test set to obtain the trained code generation model.

[0025] The trained code generation model encodes the Abstract Syntax Tree (AST) to generate a hidden representation containing semantic information.

[0026] Based on the hidden representation, a linear classifier is used to classify the hidden representation to determine whether there is hallucination in the code sample.

[0027] As a further limitation, the code generation model based on the Transformer architecture consists of 12 stacked encoder layers.

[0028] Each encoder layer integrates and refines the extracted features through a multi-head self-attention mechanism and a feedforward network. The formula for calculating the attention weight at each position in the input sequence of each encoder layer is as follows:

[0029] ;

[0030] in, For query vector, For key vectors, For value vectors, It is the dimension of the key vector;

[0031] The calculation method for each attention head in a multi-head self-attention mechanism is as follows:

[0032] ;

[0033] in, For the first The query projection weight matrix of each attention head For the first The key projection weight matrix of each attention head. For the first The values ​​of each attention head are projected onto the weight matrix. The first in the multi-head self-attention mechanism One's attention, For attention operations;

[0034] The final output of the multi-head self-attention mechanism is:

[0035] ;

[0036] in, The first in the multi-head self-attention mechanism The output of each attention head For the total number of attention heads, This is a learnable output projection weight matrix used to integrate the stitched information. For splicing operations;

[0037] In each encoder layer, the feedforward network applies a nonlinear transformation. The calculation method for the feedforward network is as follows:

[0038] ;

[0039] in, For the input vector, , This is the weight matrix. , For bias vectors, This is the ReLU activation function.

[0040] As a further clarification, the code generation model based on the Transformer architecture adopts CodeBERT, UnixCoder, CodeT5, GraphCodeBERT, BiLSTM, or MLP.

[0041] As a limitation, step S3 includes the following process:

[0042] The hidden representation of the token sequence in the original O-AST is mapped to the syntax subspace by a learnable linear projection matrix through the Probe network.

[0043] In the syntactic subspace, the original abstract syntax tree (O-AST) is converted into a binary tree structure;

[0044] Traverse the binary tree structure, calculate the tree distance between each pair of adjacent token sequences in the binary tree structure, and obtain the distance sequence d, the parent node type sequence c, and the unary node type sequence u. Then convert the binary tree structure into a vector tuple t = (d, c, u).

[0045] Using the hidden representation of the original abstract syntax tree O-AST as input and the target vector tuple as target, we train a linear projection matrix to learn the mapping from the hidden representation space to the syntax subspace.

[0046] The hidden representation is processed using the trained linear projection matrix to obtain the predicted vector tuples;

[0047] The binary tree structure is recursively reconstructed based on the predicted vector tuples, and the reconstructed binary tree structure is then used to reconstruct the predicted abstract syntax tree P-AST.

[0048] As a further limitation, the conversion of the original abstract syntax tree (O-AST) into a binary tree structure in the syntactic subspace includes:

[0049] For nodes with more than two children in the original O-AST, convert them into binary form by introducing empty nodes; for nodes with only one child, merge the parent and child node types.

[0050] As a further refinement, the binary tree structure will be recursively reconstructed based on the predicted vector tuples. When the reconstructed binary tree structure is used to rebuild the predicted abstract syntax tree (P-AST), the formula for calculating the loss function is as follows:

[0051] ;

[0052] ;

[0053] ;

[0054] ;

[0055] in, For the total loss function, For depth-sorting loss, Loss of common ancestry classification The classification loss for merging nodes, For balance coefficient, For constraint terms, For the first The depth of each token in the syntax tree For the first The depth of each token in the syntax tree The vector representation of the label. The model predicts the first The structural information of each token sequence The model predicts the first The structural information of each token sequence For the first The hidden representation of a sequence of tokens. For the first The hidden representation of a sequence of tokens. After transformation by projection matrix B Coordinate representation in the syntactic subspace S After transformation by projection matrix B Coordinate representation in the syntactic subspace S Public Ancestors Tag The corresponding label vector, , The set of all possible common ancestor tags. To merge node labels The corresponding label vector, , The set of all possible merged node labels;

[0056] The model predicts the first Structure information of each token sequence The calculation formula is:

[0057] ;

[0058] Constraints The calculation formula is:

[0059] ;

[0060] in, It is the Frobenius norm. It is the corresponding identity matrix.

[0061] As a third limitation, step S4 includes the following process:

[0062] Node type and structural information are extracted from the predicted abstract syntax tree (P-AST) and the original abstract syntax tree (O-AST) to form the predicted structure set and the original structure set, respectively.

[0063] Assign base scores to code tags associated with syntactic structures in the predicted structure set and syntactic structures in the original structure set, respectively, and assign weighted scores higher than the base scores to code tags associated with syntactic structures belonging to the control flow type.

[0064] Aggregate the scores of all code tags belonging to the same line of code to obtain the line-level score for that line of code;

[0065] Lines of code are sorted based on line-level scores, and the line with the highest score is identified as the line associated with code illusion.

[0066] The present invention, by adopting the above-described technical solution, achieves the following technical advancements compared to existing technologies:

[0067] (1) The present invention encodes the original abstract syntax tree O-AST by a trained code generation model to generate a hidden representation containing semantic information, thereby improving the detection accuracy;

[0068] (2) This invention maps the hidden representation of the original abstract syntax tree O-AST to the syntax subspace through a probe network, which solves the error problem caused by the difference in node types in the traditional abstract syntax tree reconstruction and provides a reliable foundation for subsequent illusion localization;

[0069] (3) By comparing the difference between the predicted abstract syntax tree P-AST and the original abstract syntax tree O-AST, the present invention achieves accurate identification of key code lines and improves the accuracy of localization;

[0070] (4) This invention not only improves the accuracy of detection and positioning, but also enhances the interpretability and scalability of the model, providing strong technical support for the field of code security.

[0071] In summary, this invention is applicable to accurately locating specific lines of code that trigger hallucinations while detecting them, thus improving the accuracy of the location and providing strong technical support for the field of code security. Attached Figure Description

[0072] Figure 1 The diagram shown is a flowchart of a method according to an embodiment of the present invention;

[0073] Figure 2 The diagram shown is an overall framework diagram of the trained code generation model and probe network in an embodiment of the present invention.

[0074] Figure 3 The diagram shown is a bidirectional transformation diagram between the original abstract syntax tree O-AST and the vector tuple t = (d, c, u) in an embodiment of the present invention. Detailed Implementation

[0075] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0076] Example: A method for code illusion detection and localization based on hidden layer vector mapping

[0077] like Figure 1 and Figure 2 As shown, this embodiment is a method for code illusion detection and localization based on hidden layer vector mapping, including the following steps:

[0078] S1. Data preprocessing: Obtain code samples to form a code illusion detection dataset, preprocess the code samples in the code illusion detection dataset, and use a syntax parser to parse the code samples in the preprocessed code illusion detection dataset to obtain the original abstract syntax tree O-AST.

[0079] In this step, each code sample in the code illusion detection dataset contains source code, label, and illusion line index fields.

[0080] To ensure data quality, the code samples undergo multi-layer filtering. A syntax parser is used to parse the code samples in the preprocessed code illusion detection dataset to obtain the original abstract syntax tree (O-AST). The process includes:

[0081] (1) Use a syntax parser to parse the code samples in the preprocessed code illusion detection dataset into an abstract syntax tree (AST), check whether the AST is a valid tree structure, and filter out invalid tree structures;

[0082] (2) Check whether the abstract syntax tree (AST) contains erroneous nodes and filter out erroneous nodes;

[0083] (3) Use the lexical analyzer tokenizer to segment the abstract syntax tree AST, convert it into a token sequence, count the number of token sequences, and filter out ASTs that exceed the maximum length limit (default 512); among them, the lexical analyzer tokenizer can be the RobertaTokenizer of CodeBERT.

[0084] (4) Extract all token sequences in the filtered Abstract Syntax Tree (AST) and their position information in the source code to obtain the original Abstract Syntax Tree (O-AST); the position information includes the start byte, end byte and line number.

[0085] S2. Code Illusion Detection: The original abstract syntax tree (O-AST) is encoded by the trained code generation model to generate a hidden representation containing semantic information. Based on the hidden representation, it is determined whether there is an illusion in the code sample.

[0086] The process in this step includes:

[0087] S21. Divide the preprocessed code illusion detection dataset into training, validation, and test sets according to a preset ratio of 8:1:1, and save them as JSONL files respectively.

[0088] S22. Build a code generation model based on the Transformer architecture, fine-tune the code generation model through the training set, validate the code generation model through the validation set, and evaluate the code generation model using the test set to obtain the trained code generation model.

[0089] In this step, the code generation model based on the Transformer architecture was pre-trained on a large-scale training set to learn general representations such as the syntactic structure and semantic relationships of code samples. The code generation model based on the Transformer architecture consists of 12 stacked encoder layers and adopts a hierarchical feature extraction strategy. Each encoder layer integrates and refines the features extracted from the input through a multi-head self-attention mechanism and a feedforward network to form a multi-level code representation. At the same time, the multi-head self-attention mechanism captures long-distance dependencies by calculating the correlation between each token sequence and other token sequences.

[0090] The formula for calculating the attention weight at each position in the input sequence of each encoder layer is:

[0091] ;

[0092] in, For query vector; The key vector; It is a value vector; It is the dimension of the key vector, used to scale the point integral, allowing the model to dynamically focus on other token sequences most relevant to the current token sequence, thereby constructing rich contextual representations; at the same time, in order to enhance the expressive power of the model, the encoder layer uses multiple independent attention heads in parallel. This multi-head self-attention mechanism allows the model to learn different subspace representations and capture various aspects of the input;

[0093] The calculation method for each attention head in a multi-head self-attention mechanism is as follows:

[0094] ;

[0095] in, For the first The query projection weight matrix of each attention head For the first The key projection weight matrix of each attention head. For the first The values ​​of each attention head are projected onto the weight matrix. The first in the multi-head self-attention mechanism One's attention, For attention operations;

[0096] The final output of the multi-head self-attention mechanism is:

[0097] ;

[0098] in, The first in the multi-head self-attention mechanism The output of each attention head For the total number of attention heads, This is a learnable output projection weight matrix used to integrate the stitched information. For splicing operations;

[0099] This process effectively integrates features from different subspaces into the hidden representation, allowing the model to focus on different types of code relationships simultaneously, such as syntactic dependencies, semantic associations, and structural hierarchies.

[0100] Meanwhile, in each encoder layer, the feedforward network (FFN) applies a nonlinear transformation to further refine the feature representation. The calculation method of the feedforward network (FFN) is as follows:

[0101] ;

[0102] in, For the input vector, , This is the weight matrix. , For bias vectors, It is the ReLU activation function;

[0103] FFN maps features to a higher-dimensional representation space through two layers of linear transformation and the ReLU activation function, enhancing the model's non-linear expressive power. This refined hidden representation allows the model to focus more on identifying illusion problems such as inconsistencies and logical errors in code samples, encodes the key features required for code illusion detection, and provides input for the probe network. By analyzing the syntactic structure and semantic information in these hidden representations, potential factors affecting code illusion detection can be identified, thereby achieving accurate code illusion localization.

[0104] In this step, the code generation model based on the Transformer architecture uses CodeBERT, UnixCoder, CodeT5, GraphCodeBERT, BiLSTM, or MLP;

[0105] S23. Encode the Abstract Syntax Tree (AST) using the trained code generation model to generate a hidden representation containing semantic information;

[0106] S24. Based on the hidden representation, a linear classifier is used to classify the hidden representation to determine whether there is a hallucination in the code sample; if there is a hallucination, proceed to the next step; if there is no hallucination, the code sample is temporarily saved as the correct sample.

[0107] In this step, the trained code generation model encodes the abstract syntax tree (AST) to generate a hidden representation containing rich semantic information. Through fine-tuning training on a labeled code illusion detection dataset, the trained code generation model learns to recognize the differences between normal code and code containing illusions. Then, an additional linear classifier is used to classify the hidden representation to determine whether there are illusions in the code samples, in order to generate the final detection results. This stage not only provides basic illusion detection capabilities, but more importantly, it provides rich hidden representation features for subsequent illusion localization analysis.

[0108] S3. Probe AST Mapping and Reconstruction: The hidden representation of the original abstract syntax tree O-AST is mapped to the syntax subspace through a probe network to obtain vector tuples, and the vector tuples are reconstructed into the predicted abstract syntax tree P-AST.

[0109] In this step, the hidden representation is used express, It encodes basic information for hallucination recognition and detection; in order to reveal To investigate the encoded content and how it affects detection results, a probe network is introduced. This method helps to examine the features captured by the model and gain a deeper understanding of the reasoning behind the model's predictions; [hypothesis] For all hidden representations A set, in which each , The hidden representation contains rich information, but its content is complex because it includes some irrelevant information. To prevent this irrelevant information from interfering with the results, a probe network is used to focus specifically on the syntactic information in the hidden representation. It is assumed that there exists a syntactic subspace of the hidden representation information of an original abstract syntax tree (O-AST). The target of the detection is through a learnable linear projection matrix. Hidden representation set Projected onto the syntax subspace :

[0110] ;

[0111] Step S3 includes the following process:

[0112] S31. The hidden representation of the token sequence in the original abstract syntax tree O-AST is mapped to the syntax subspace by using a learnable linear projection matrix through the Probe network.

[0113] S32. In the syntax subspace, the original abstract syntax tree O-AST is converted into a binary tree structure;

[0114] S33. Traverse the binary tree structure, calculate the tree distance between each pair of adjacent token sequences in the binary tree structure, and obtain the distance sequence d, the parent node type sequence c, and the unary node type sequence u. Convert the binary tree structure into a vector tuple t = (d, c, u).

[0115] like Figure 3 The diagram shows the bidirectional transformation between the original O-AST (Obscissa-Abstract Syntax Tree) and the vector tuple t = (d, c, u). Rounded rectangles represent non-terminal nodes, and circles represent terminal nodes. This transformation is performed in two steps:

[0116] Step 1: Convert the original O-AST (Original Abstract Syntax Tree) into a binary tree structure. To convert the original O-AST into a binary tree structure, unary and binary nodes need to be binary-coded. The following three requirements must be met during the binary-coding process:

[0117] 1) If a node is n-ary and has more than two child nodes, insert a special empty node to binary-ify it; e.g. Figure 3 As shown, the 3-ary non-terminal node if_statement is converted into a binary structure by adding an empty node.

[0118] 2) For a unary node (with only one child node), merge it with its child node to form a new node; this rule applies to both terminal and non-terminal nodes.

[0119] 3) To convert the binary tree structure back to the original abstract syntax tree, empty nodes need to be removed. Then reconnect the tree structure; in addition, the previously merged unary nodes need to be expanded again.

[0120] The second step is to convert the binary tree structure into a vector tuple t = (d, c, u), thus completing the two-step transformation from the original abstract syntax tree O-AST to the vector tuple representation. At the same time, the vector tuple t = (d, c, u) can also be used to perform the reverse transformation to obtain the corresponding abstract syntax tree AST.

[0121] 1) The distance sequence d encodes structural information; the distance sequence d records the structural relationships between adjacent terminal symbols in the binary tree structure; each d z The value represents the distance from the root node to two adjacent terminal symbols (W). z W z+1 The path length of the lowest common ancestor of the two terminal symbols is the value that reflects the hierarchical relationship between the two terminal symbols in the syntax tree. For example, d=(2,1) means that the common ancestor of the first and second terminal symbols is at level 2, and the common ancestor of the second and third terminal symbols is at level 1.

[0122] 2) The parent node type sequence c encodes the label information; it stores the syntax label of the lowest common ancestor of each pair of adjacent terminal symbols. These labels reflect the syntactic structure type of the code. The dimension of the parent node type sequence c is the same as that of the distance sequence d, because each pair of adjacent terminal symbols corresponds to a common ancestor; for example: c = (if_statement, The first pair of terminal symbols has a common ancestor of the if_statement node, and the second pair is an empty node.

[0123] 3) The unary node type sequence u also encodes label information; the unary node type sequence u records whether each terminal symbol node is merged with a non-terminal symbol node. If a terminal symbol node is merged with a non-terminal symbol node, the unary node type sequence u records the merged label; otherwise, it is recorded as... <empty>The dimension is the number of terminal symbols, n;

[0124] S34. Using the hidden representation of the original O-AST as input and the target vector tuple as the target, train the linear projection matrix. Learn the mapping from the hidden representation space to the syntactic subspace;

[0125] S35. The hidden representation is processed using the trained linear projection matrix to obtain the predicted vector tuples;

[0126] During the learning process, the learned linear projection matrix is ​​first used for parsing and hidden representation. The corresponding input code is used to obtain the original abstract syntax tree (O-AST), and then a two-step transformation is applied to generate the target vector tuple. Then, using the hidden representation of the original O-AST as input and the target vector tuple as the target, a linear projection matrix is ​​trained. The hidden representation is processed using the trained linear projection matrix to extract AST structure information and obtain the predicted vector tuples.

[0127] S36. Recursively reconstruct the binary tree structure based on the predicted vector tuples. Based on the reconstructed binary tree structure, reconstruct the predicted abstract syntax tree P-AST to complete the process of extracting syntactic information from the hidden representation.

[0128] The binary tree structure is recursively reconstructed based on the predicted vector tuples. When the reconstructed binary tree structure is used to rebuild the predicted abstract syntax tree (P-AST), the formula for calculating the loss function is as follows:

[0129] ;

[0130] ;

[0131] ;

[0132] ;

[0133] in, For the total loss function, The depth-ordering loss is used to maintain the correct relative order of token depths. The common ancestor classification loss is used to predict the type of the common ancestor node between adjacent tokens. The classification loss for merging nodes. Used to predict the type of merging node corresponding to a token. For balance coefficient, For constraint terms, For the first The depth of each token in the syntax tree For the first The depth of each token in the syntax tree The vector representation of the label. The model predicts the first The structural information of each token sequence The model predicts the first The structural information of each token sequence For the first The hidden representation of a sequence of tokens. For the first The hidden representation of a sequence of tokens. After transformation by projection matrix B Coordinate representation in the syntactic subspace S After transformation by projection matrix B Coordinate representation in the syntactic subspace S Public Ancestors Tag The corresponding label vector, , The set of all possible common ancestor tags. To merge node labels The corresponding label vector, , The set of all possible merged node labels;

[0134] The model predicts the first Structure information of each token sequence The calculation formula is:

[0135] ;

[0136] To prevent overfitting of the model, constraint terms... The calculation formula is:

[0137] ;

[0138] in, It is the Frobenius norm. It is the corresponding identity matrix.

[0139] S4. Code Illusion Localization: Compare the differences between the predicted abstract syntax tree (P-AST) and the original abstract syntax tree (O-AST) to identify and locate the lines of code associated with code illusion.

[0140] The Abstract Syntax Tree (P-AST) is an abstract syntax tree extracted and reconstructed from the hidden representation, representing the encoded syntactic information and directly influencing the decision-making process of the code illusion detection model. During fine-tuning, features relevant to code illusion are enhanced, while features irrelevant to code illusion are suppressed. The Original Abstract Syntax Tree (O-AST) is a true abstract syntax tree directly constructed from the source code using parsers such as tree-sitter, representing the actual syntactic structure of the code. Based on this understanding, it is assumed that changes in the syntactic features embedded in the hidden representation will lead to corresponding changes in the extracted predicted P-AST, and these changes are closely related to the decisions of the code illusion model. In this step, code illusion localization is achieved by analyzing the differences between the predicted P-AST and the original O-AST. The comparison between the predicted P-AST and the original O-AST is presented with pseudocode for the comparison and scoring process. By comparing the differences between the predicted P-AST and the original O-AST, the system can identify deviations in the model's syntactic understanding, thereby detecting code illusion and achieving accurate line-level localization.

[0141] The process in this step includes:

[0142] S41. Extract the node type and structure information from the predicted abstract syntax tree P-AST and the original abstract syntax tree O-AST respectively to form the predicted structure set and the original structure set.

[0143] In this step, to compare the predicted abstract syntax tree P-AST and the original abstract syntax tree O-AST, tree component representation is used to evaluate the degree of overlap between the predicted abstract syntax tree P-AST and the original abstract syntax tree O-AST. The tree component representation is defined as:

[0144] ;

[0145] in, The tokenID used for the input code captures the root node and all leaf nodes of the subtree, but ignores the internal structure. To obtain a more complete representation of each subtree, a preorder traversal is performed, as shown below:

[0146] ;

[0147] Unlike tree component representations that only record the root and leaves, preorder traversal representations more comprehensively preserve the hierarchical structure of subtrees; this structured representation better captures the key syntactic patterns used for comparison.

[0148] S42. Assign basic scores to code tags associated with syntax structures in the predicted structure set and syntax structures in the original structure set, respectively, and assign weighted scores higher than the basic scores to code tags associated with syntax structures belonging to the control flow type.

[0149] In this step, the syntax structures in the predicted structure set and the syntax structures in the original structure set are traversed. Each syntax structure in the predicted structure set is compared with each syntax structure in the original structure set. If the prediction is correct, a base score of 1 point is given. For syntax structures that belong to the control flow type (such as if statement, loop statement, exception handling, etc.), a weighted score of 1.5 points is given.

[0150] S43. Aggregate the scores of all code tags belonging to the same line of code to obtain the line-level score of that line of code;

[0151] In this step, the token sequence-level score is mapped to the line-level score using relevant functions. For strict alignment, the scores of the corresponding token sequences for each line are added together to obtain the line-level score. For alignment failures, the system will attempt to re-segment and map the token sequences, ultimately generating lines_score containing the score for each line of code. Lines with higher scores are considered more relevant to the code illusion, thus completing the code illusion localization from the token level to the line level.

[0152] S44. Sort the lines of code based on line-level scores and locate the lines of code with the highest scores as the lines related to code illusion;

[0153] In this step, the scoring process is shown in Table 1:

[0154]

[0155] First, lines 1-3 perform the initialization preparation phase. Line 1 extracts all syntactic structures from the predicted abstract syntax tree (P-AST) and stores them in `structuresP`. Line 2 extracts all syntactic structures from the original abstract syntax tree (O-AST) and stores them in `structuresO`. Line 3 initializes a zero vector array `SCORE_VECTOR` of the same length as the number of token sequences. Starting from line 4, the syntactic structures in the predicted abstract syntax tree (P-AST) are traversed, and each structure in the P-AST is checked one by one. Line 5 performs structure matching verification, checking whether the current predicted structure exists in the original abstract syntax tree (O-AST) structure set `structuresO`. Line 6 obtains the root node type of the matching structure, preparing for weight allocation. Lines 7-11 begin processing the control flow-related weight allocation. Line 7 determines whether the root node type belongs to the control flow structure. Line 8 assigns a higher control weight `CONTROL_WEIGHT` (1.5 points) to the control flow structure. Line 10 assigns a basic weight `BASE_WEIGHT` (1.0 point) to the non-control flow structure. Line 12 extracts all token indices associated with the current structure to determine the specific token sequence position to be scored. Lines 13-15 perform weighted scoring on the related token sequences. Line 13 iterates through each associated token index of the current structure, and line 14 accumulates the corresponding weight values ​​to the corresponding position of SCORE_VECTOR. Accumulation is used because a single token sequence may belong to multiple syntax structures simultaneously. If a structure does not have a match in the original O-AST, it is skipped without any scoring.

[0156] Finally, the scores obtained from the marking are summarized to calculate the score for each line of code. Specifically, for each line of code, the scores of all the markings in that line are added together, and the cumulative score is calculated. Then, the line of code with the highest score is identified as the line associated with the code illusion.

[0157] It should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art can still modify the technical solutions described in the above embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.< / empty>

Claims

1. A method for detecting and locating code illusions based on hidden layer vector mapping, characterized in that, Includes the following steps: S1. Data preprocessing: Obtain code samples to form a code illusion detection dataset, preprocess the code samples in the code illusion detection dataset, and use a syntax parser to parse the code samples in the preprocessed code illusion detection dataset to obtain the original abstract syntax tree O-AST. S2. Code Illusion Detection: The original abstract syntax tree (O-AST) is encoded by the trained code generation model to generate a hidden representation containing semantic information. Based on the hidden representation, it is determined whether there is an illusion in the code sample. S3. Probe AST Mapping and Reconstruction: The hidden representation of the original abstract syntax tree O-AST is mapped to the syntax subspace through a probe network to obtain vector tuples, and the vector tuples are reconstructed into the predicted abstract syntax tree P-AST. S4. Code Illusion Localization: Compare the differences between the predicted abstract syntax tree (P-AST) and the original abstract syntax tree (O-AST) to identify and locate the lines of code associated with code illusion.

2. The method for code illusion detection and localization based on hidden layer vector mapping according to claim 1, characterized in that, In step S1, each code sample in the code illusion detection dataset contains source code, label, and illusion line index fields.

3. The method for code illusion detection and localization based on hidden layer vector mapping according to claim 2, characterized in that, In step S1, the process of using a syntax parser to parse the code samples in the preprocessed code illusion detection dataset to obtain the original abstract syntax tree (O-AST) includes: A syntax parser is used to parse the code samples in the preprocessed code illusion detection dataset into an abstract syntax tree (AST), and the AST is checked to see if it is a valid tree structure, while invalid tree structures are filtered out. Check if the Abstract Syntax Tree (AST) contains erroneous nodes and filter out erroneous nodes; The abstract syntax tree (AST) is segmented using a lexical analyzer, converted into token sequences, the number of token sequences is counted, and ASTs exceeding the maximum length limit are filtered out. Extract all token sequences and their positions in the source code from the filtered Abstract Syntax Tree (AST) to obtain the original Abstract Syntax Tree (O-AST); The location information includes the start byte, the end byte, and the line number.

4. The method for code illusion detection and localization based on hidden layer vector mapping according to claim 1, characterized in that, Step S2 includes the following process: The preprocessed code illusion detection dataset is divided into a training set, a validation set, and a test set according to a preset ratio of 8:1:

1. A code generation model based on the Transformer architecture is built. The code generation model is fine-tuned and trained using the training set, validated using the validation set, and evaluated using the test set to obtain the trained code generation model. The trained code generation model encodes the Abstract Syntax Tree (AST) to generate a hidden representation containing semantic information. Based on the hidden representation, a linear classifier is used to classify the hidden representation to determine whether there is hallucination in the code sample.

5. The method for code illusion detection and localization based on hidden layer vector mapping according to claim 4, characterized in that, The code generation model based on the Transformer architecture consists of 12 stacked encoder layers; Each encoder layer integrates and refines the extracted features through a multi-head self-attention mechanism and a feedforward network. The formula for calculating the attention weight at each position in the input sequence of each encoder layer is as follows: ； in, For query vector, For key vectors, For value vectors, It is the dimension of the key vector; The calculation method for each attention head in a multi-head self-attention mechanism is as follows: ； in, For the first The query projection weight matrix of each attention head For the first The key projection weight matrix of each attention head. For the first The values ​​of each attention head are projected onto the weight matrix. The first in the multi-head self-attention mechanism One's attention, For attention operations; The final output of the multi-head self-attention mechanism is: ； in, The first in the multi-head self-attention mechanism The output of each attention head For the total number of attention heads, This is a learnable output projection weight matrix used to integrate the stitched information. For splicing operations; In each encoder layer, the feedforward network applies a nonlinear transformation. The calculation method for the feedforward network is as follows: ； in, For the input vector, , This is the weight matrix. , For bias vectors, This is the ReLU activation function.

6. The method for code illusion detection and localization based on hidden layer vector mapping according to claim 5, characterized in that, The code generation model based on the Transformer architecture uses CodeBERT, UnixCoder, CodeT5, GraphCodeBERT, BiLSTM, or MLP.

7. The method for code illusion detection and localization based on hidden layer vector mapping according to claim 3, characterized in that, Step S3 includes the following process: The hidden representation of the token sequence in the original O-AST is mapped to the syntax subspace by a learnable linear projection matrix through the Probe network. In the syntactic subspace, the original abstract syntax tree (O-AST) is converted into a binary tree structure; Traverse the binary tree structure, calculate the tree distance between each pair of adjacent token sequences in the binary tree structure, and obtain the distance sequence d, the parent node type sequence c, and the unary node type sequence u. Then convert the binary tree structure into a vector tuple t = (d, c, u). Using the hidden representation of the original abstract syntax tree O-AST as input and the target vector tuple as target, we train a linear projection matrix to learn the mapping from the hidden representation space to the syntax subspace. The hidden representation is processed using the trained linear projection matrix to obtain the predicted vector tuples; The binary tree structure is recursively reconstructed based on the predicted vector tuples, and the predicted abstract syntax tree P-AST is then reconstructed based on the reconstructed binary tree structure.

8. The method for code illusion detection and localization based on hidden layer vector mapping according to claim 7, characterized in that, The process of converting the original abstract syntax tree (O-AST) into a binary tree structure in the syntactic subspace includes: For nodes with more than two children in the original O-AST, convert them into binary form by introducing empty nodes; for nodes with only one child, merge the parent and child node types.

9. The method for code illusion detection and localization based on hidden layer vector mapping according to claim 8, characterized in that, The binary tree structure is recursively reconstructed based on the predicted vector tuples. When the reconstructed binary tree structure is used to rebuild the predicted abstract syntax tree (P-AST), the formula for calculating the loss function is as follows: ； ； ； ； in, For the total loss function, For depth-sorting loss, Loss of common ancestry classification The classification loss for merging nodes, For balance coefficient, For constraint terms, For the first The depth of each token in the syntax tree For the first The depth of each token in the syntax tree The vector representation of the label. The model predicts the first The structural information of each token sequence The model predicts the first The structural information of each token sequence For the first The hidden representation of a sequence of tokens. For the first The hidden representation of a sequence of tokens. After transformation by projection matrix B Coordinate representation in the syntactic subspace S After transformation by projection matrix B Coordinate representation in the syntactic subspace S For public ancestry tags The corresponding label vector, , The set of all possible common ancestor tags. To merge node labels The corresponding label vector, , The set of all possible merged node labels; The model predicts the first Structure information of each token sequence The calculation formula is: ； Constraints The calculation formula is: ； in, It is the Frobenius norm. It is the corresponding identity matrix.

10. The method for code illusion detection and localization based on hidden layer vector mapping according to claim 1, characterized in that, Step S4 includes the following process: Node type and structural information are extracted from the predicted abstract syntax tree (P-AST) and the original abstract syntax tree (O-AST) to form the predicted structure set and the original structure set, respectively. Assign base scores to code tags associated with syntactic structures in the predicted structure set and syntactic structures in the original structure set, respectively, and assign weighted scores higher than the base scores to code tags associated with syntactic structures belonging to the control flow type. Aggregate the scores of all code tags belonging to the same line of code to obtain the line-level score for that line of code; Lines of code are sorted based on line-level scores, and the line with the highest score is identified as the line associated with code illusion.

Images