Code sequence compression method, device and equipment combining semantic analysis with attention

Abstract

This invention provides a code sequence compression method, apparatus, and device that combines semantic analysis and attention, relating to the field of large language model processing technology. The method includes: performing lexical and syntactic analysis on a source code sequence to generate a syntax tree corresponding to the source code sequence; generating an original syntax topology distribution network based on the dependencies between nodes in the syntax tree; determining the comprehensive maintenance value of each node in the original syntax topology distribution network; and removing redundant nodes from the original syntax topology distribution network based on a comprehensive maintenance value threshold and the comprehensive maintenance value of each node in the original syntax topology distribution network, resulting in an updated syntax topology distribution network and a compressed code sequence. The method of this application avoids the problems of traditional compression methods such as accidentally deleting key nodes, disrupting program connectivity, and causing semantic understanding bias in the model, providing high-quality input with standardized structure, semantic coherence, and reliable reasoning for large language models.

Description

Technical Field

[0001] This invention relates to the field of large language model processing technology, and in particular to a code sequence compression method, apparatus, and device that combines semantic analysis and attention. Background Technology

[0002] In scenarios involving processing lengthy sequences of source code to fit the limited context windows of large language models, existing techniques generally draw upon input compression methods from the field of natural language processing. These methods are typically based on information entropy, word frequency statistics, or simple pruning strategies based on model attention weights, aiming to reduce the number of input tokens while preserving semantically key segments. Their underlying paradigm treats code as a continuous stream of text, primarily judging the redundancy of tokens from a statistical or shallow semantic level. However, the core assumptions of these methods lie in the discreteness of vocabulary and the locality of semantics, which fundamentally conflict with the rigorous syntactic structure, nested logic, and strong cross-contextual dependencies inherent in source code. Summary of the Invention

[0003] This invention provides a code sequence compression method, apparatus, and device that combines semantic analysis and attention. By constructing a global syntactic topology distribution network that can fully represent the syntax and execution dependencies of the source code, it integrates local attention features with the betweenness centrality of nodes in the topology network to obtain a comprehensive maintenance value for evaluating the importance of nodes. This value serves as the core basis for determining redundant nodes, achieving accurate identification and effective removal of redundant nodes. While significantly shortening the code input length and adapting to the limited context window of large language models, it fully preserves the core syntactic structure, data dependencies, and control flow logic. This fundamentally avoids the problems of traditional compression methods such as accidentally deleting key nodes, disrupting program connectivity, and causing semantic understanding bias in the model. It achieves intelligent code compression that balances compression efficiency and logical integrity, providing high-quality input with standardized structure, semantic coherence, and reliable reasoning for large language models.

[0004] This invention provides a code sequence compression method that combines semantic analysis and attention, comprising the following steps: Lexical and syntactic analysis are performed on the source code sequence to generate a syntax tree corresponding to the source code sequence; Based on the dependencies between nodes in the syntax tree, an original syntax topology distribution network is generated; Based on the preset window sliding along the source code sequence, the comprehensive maintenance value of each node in the original syntax topology distribution network is determined according to the local attention characteristics of the code subsequences in each sliding window and the network betweenness centrality of the nodes associated with the code subsequences in each sliding window in the original syntax topology distribution network. Based on the comprehensive maintenance degree value of each node in the original syntax topology distribution network, the threshold value of the comprehensive maintenance degree value is determined; Based on the comprehensive retention threshold and the comprehensive retention value of each node in the original syntax topology distribution network, redundant nodes in the original syntax topology distribution network are removed to obtain the updated syntax topology distribution network and the compressed code sequence.

[0005] The present invention also provides a code sequence compression device that combines semantic analysis and attention, comprising the following modules: The analysis module is used to perform lexical and syntactic analysis on the source code sequence and generate a syntax tree corresponding to the source code sequence. The generation module is used to generate the original syntax topology distribution network based on the dependencies between nodes in the syntax tree; The determination module is used to slide along the source code sequence based on a preset window, and determine the comprehensive maintenance value of each node in the original syntax topology distribution network according to the local attention characteristics of the code subsequences in each sliding window and the network betweenness centrality of the nodes associated with the code subsequences in each sliding window in the original syntax topology distribution network. The processing module is used to determine the threshold value of the comprehensive maintenance degree based on the comprehensive maintenance degree value of each node in the original syntax topology distribution network; The compression module is used to remove redundant nodes in the original syntax topology distribution network based on the comprehensive maintenance degree threshold and the comprehensive maintenance degree value of each node in the original syntax topology distribution network, so as to obtain the updated syntax topology distribution network and the compressed code sequence.

[0006] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the code sequence compression method combining semantic analysis and attention as described above.

[0007] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the code sequence compression method combining semantic analysis and attention as described above.

[0008] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the code sequence compression method combining semantic analysis and attention as described above.

[0009] This invention provides a code sequence compression method, apparatus, and device that combines semantic analysis and attention. By constructing a global syntactic topology distribution network that can fully represent the syntax and execution dependencies of the source code, it integrates local attention features with the betweenness centrality of nodes in the topology network to obtain a comprehensive maintenance value for evaluating the importance of nodes. This value serves as the core basis for determining redundant nodes, achieving accurate identification and effective removal of redundant nodes. While significantly shortening the code input length and adapting to the limited context window of large language models, it fully preserves the core syntactic structure, data dependencies, and control flow logic. This fundamentally avoids the problems of traditional compression methods such as accidentally deleting key nodes, disrupting program connectivity, and causing semantic understanding bias in the model. It achieves intelligent code compression that balances compression efficiency and logical integrity, providing high-quality input with standardized structure, semantic coherence, and reliable reasoning for large language models. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0011] Figure 1 This is one of the flowcharts of the code sequence compression method that combines semantic analysis and attention provided by the present invention.

[0012] Figure 2 This is the second flowchart of the code sequence compression method that combines semantic analysis and attention provided by this invention.

[0013] Figure 3 This is a schematic diagram of the structure of the code sequence compression device that combines semantic analysis and attention provided by the present invention.

[0014] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0016] The following is combined Figures 1 to 4 The present invention describes a code sequence compression method, apparatus, and device that combines semantic analysis and attention.

[0017] To facilitate a clearer understanding of the technical solutions of the various embodiments of this application, some technical content related to the various embodiments of this application will be introduced first.

[0018] In scenarios involving processing lengthy sequences of source code to fit the limited context windows of large language models, existing techniques generally draw upon input compression methods from the field of natural language processing. These methods are typically based on information entropy, word frequency statistics, or simple pruning strategies based on model attention weights, aiming to reduce the number of input tokens while preserving semantically key segments. Their underlying paradigm treats code as a continuous stream of text, primarily judging the redundancy of tokens from a statistical or shallow semantic level. However, the core assumptions of these methods lie in the discreteness of vocabulary and the locality of semantics, which fundamentally conflict with the rigorous syntactic structure, nested logic, and strong cross-contextual dependencies inherent in source code.

[0019] When directly applying such methods to process code, the problems and shortcomings are as follows: due to a lack of understanding of the global execution logic topology of the program, "greedy" compression based on local statistical features or short-range semantics is prone to mistakenly deleting syntactic nodes (such as interface definitions, control flow conditions, or parameter passing points) that play a key relay or bridging role in the global dependency graph. Although this stripping shortens the sequence in form, it destroys the inherent syntactic integrity of the code and the connectivity of data / control flow, resulting in logical breaks, which in turn leads to deviations in the large language model's understanding of the program's semantics or interruptions in execution flow reasoning. For example, removing a seemingly redundant intermediate function call that is responsible for passing key context parameters will prevent the model from correctly associating subsequent deep-level logic.

[0020] Figure 1 This is one of the flowcharts illustrating the code sequence compression method combining semantic analysis and attention provided by this invention, such as... Figure 1 As shown, the method includes the following: Step 101: Perform lexical and syntactic analysis on the source code sequence to generate a syntax tree corresponding to the source code sequence.

[0021] Specifically, after obtaining the source code to be compressed, this application performs lexical and syntactic analysis on the source code sequence using a deep syntax parser to generate a corresponding Abstract Syntax Tree (AST). This transforms the linear code text into a structured tree representation, accurately capturing the syntactic nesting and hierarchical organization relationships of the code. The nodes in the AST... (The set of abstract syntax tree nodes) corresponds to the syntax units in the source code, including but not limited to function declarations, variable definitions, control flow statements, expressions, etc.

[0022] Step 102: Generate the original syntax topology distribution network based on the dependencies between nodes in the syntax tree.

[0023] Specifically, after generating the syntax tree corresponding to the source code, this application constructs a data dependency graph and a control dependency graph, and analyzes the data dependencies and control dependencies of the nodes in the syntax tree. Among these, data dependency analysis tracks the relationships between variable definitions and usage; if a node... The variable values ​​used depend on the node The assignment operation in the middle establishes a path from arrive Directed edges; control dependency analysis is used to determine whether the execution of one node is controlled by the conditional judgment result of another node.

[0024] Optionally, this application uses the node set of the AST. With the set of directed edges obtained through program dependency analysis By merging them, a primitive syntax topology distribution network is formed. , This is a set of network nodes, corresponding to key syntactic units; Let be a set of directed edges, each edge Indicates from node To the node There are some kind of syntactic or execution-level dependencies, which in turn provide a comprehensive and computable structured context for subsequent steps.

[0025] In other words, this application abandons the "greedy" pruning strategy that relies solely on local attention or statistical features, and instead constructs and relies on the original syntactic topology distribution network that reflects the code syntax and execution dependencies. Based on this network, truly redundant nodes that contribute little to global logical connectivity can be accurately identified. This mechanism fundamentally avoids the risk of accidentally deleting critical relay or bridging code nodes in traditional methods. Its beneficial effect is that the compression process is no longer a blind length reduction, but an intelligent slimming based on an understanding of the program structure. It ensures that while the length is significantly reduced, the core execution path and dependency network are completely preserved, providing a structurally correct and high-quality input base for subsequent model understanding. It achieves accurate and adaptive redundant code stripping under global syntactic topology constraints, ensuring the core logical integrity of the compressed sequence from the source.

[0026] Step 103: Based on the preset window sliding along the source code sequence, determine the comprehensive maintenance value of each node in the original syntax topology distribution network according to the local attention characteristics of the code subsequences in each sliding window and the network betweenness centrality of the nodes associated with the code subsequences in each sliding window in the original syntax topology distribution network.

[0027] Specifically, this application uses a preset sliding window to scan the source code in segments, extracts the local attention features of the code subsequences in each window, and associates the code subsequences with the corresponding nodes in the original syntax topology distribution network. By fusing the local attention features with the network betweenness centrality of the associated nodes, the comprehensive maintenance value of each node is obtained. This effectively constructs a unified evaluation index that can simultaneously reflect the local semantic saliency and global structural importance of a node, providing an objective and calculable standard for the determination of redundant nodes.

[0028] Exemplary, local attention features This primarily reflects the semantic attention of a large language model to the local context of a node. However, this data-based perception may overlook code nodes that play a crucial structural pivot role in the global execution logic but have sparse or conventional local text patterns (such as interface definitions and control flow branch points). If node selection is based solely on this, it is easy to mistakenly delete these "structural anchors" that maintain the overall topological connectivity. Therefore, this application focuses on how to construct a unified evaluation index that can simultaneously reflect the local semantic saliency and global structural importance of a node. This index needs to effectively integrate forward perception from the large language model and backward structural knowledge from program analysis to correct potential biases from a single perspective, providing a decision-making basis that is both semantically sensitive and structurally robust for subsequent adaptive stripping. This requires the fusion mechanism to be mathematically interpretable and parameter-adjustable to adapt to different coding styles or task requirements.

[0029] For example, the specific implementation process is as follows: For the network Each node in Read the associated local attention activation features from the aligned feature library, denoted as This feature value is typically a normalized scalar, for example, obtained from the attention vector during the alignment process. Norm calculation or mean pooling is used to obtain the norm, and its range is constrained to [a certain value]. The range represents the interval, with higher values ​​indicating stronger model attention to that node within its local context. Additionally, it's necessary to calculate the value for each node. Network betweenness centrality Betweenness centrality is a classic graph theory centrality metric used to measure how well a node serves as a bridge between other pairs of nodes in a graph. It is calculated as follows: for all pairs of nodes in the graph... Calculate the number of shortest paths between them, and the nodes that these shortest paths pass through. The sum of the ratios of the two quantities is the quantity. Betweenness centrality. This quantifies the frequency with which the node is "traversed" and "dependent on" in the global syntax dependency and execution call chain. (Calculation) Afterwards, normalization is usually required, such as dividing by... (For undirected graphs) or perform max-min normalization to keep the values ​​within a stable range. After obtaining the above two features, calculate the nodes. Comprehensive maintenance value :

[0030] In this formula, Represents a node The overall retention value quantifies the priority of the node to be retained during the compression process; the higher the value, the greater the necessity of retention. This refers to the local attention activation features extracted and aligned from the large language model, representing the local semantic salience of the node. For nodes exist In this context, the network betweenness centrality represents the global structural importance of a node. This is the modality fusion weighting factor, an adjustable hyperparameter used to control the relative weights of local attention features and global structural features in the final evaluation. When the value is close to 1, decision-making relies more on model perception; when... As the value approaches zero, decision-making becomes more reliant on topological structure. Extensive experimental data suggests that, to achieve a balance between semantic understanding and structural preservation, the empirical range for this value is typically set at... . It is the topological smoothing factor, also a hyperparameter, mainly used to adjust the network betweenness centrality. For exponential functions Input sensitivity. Introduction The function is designed to amplify the contribution of nodes with high betweenness centrality, while This can suppress the explosive growth of output values ​​caused by excessively high centroid of individual nodes, ensuring the numerical stability of the fusion process. The recommended empirical range is... This weighted fusion formula calculates a scalar score for each node in the network that integrates local semantic popularity and global topological hub value, i.e., a comprehensive retention value. This completed the key transformation from heterogeneous features to a unified decision-making basis, providing an objective and calculable standard for the determination of redundant nodes.

[0031] Step 104: Determine the threshold value of the comprehensive maintenance degree based on the comprehensive maintenance degree value of each node in the original syntax topology distribution network.

[0032] Specifically, after determining the comprehensive retention value of each node in the original grammatical topology distribution network, this application can statistically analyze the distribution characteristics of the comprehensive retention value of all nodes in the original grammatical topology distribution network, such as the mean and variance, and dynamically calculate an adaptive comprehensive retention value threshold to accurately distinguish between the core nodes that should be retained and the redundant nodes that can be removed in the original grammatical topology distribution network.

[0033] Step 105: Based on the comprehensive retention value threshold and the comprehensive retention value of each node in the original syntax topology distribution network, remove redundant nodes in the original syntax topology distribution network to obtain the updated syntax topology distribution network and the compressed code sequence.

[0034] Specifically, after determining the threshold value for comprehensive retention, this application can use the threshold value as a criterion to mark nodes below the threshold as redundant nodes and remove them from the original syntax topology distribution network. Simultaneously, the corresponding code segments are removed to obtain the updated syntax topology distribution network and compressed code sequence. This significantly shortens the input length, adapts to the context window of a large language model, and accurately preserves the core syntax units and logical dependencies. It effectively solves the problems of traditional code compression methods that easily destroy program structure and lead to model comprehension bias, thereby improving the efficiency and accuracy of code compression.

[0035] The method described in the above embodiments constructs a global syntactic topology distribution network that can fully represent the syntax and execution dependencies of the source code. It integrates local attention features with the betweenness centrality of nodes in the topology network to obtain a comprehensive maintenance value for evaluating the importance of nodes. This value serves as the core basis for determining redundant nodes, enabling accurate identification and effective removal of redundant nodes. While significantly shortening the code input length and adapting to the limited context window of large language models, it fully preserves the core syntactic structure, data dependencies, and control flow logic. This fundamentally avoids the problems of traditional compression methods such as accidentally deleting key nodes, disrupting program connectivity, and causing semantic understanding bias in the model. It achieves intelligent code compression that balances compression efficiency and logical integrity, providing high-quality input with standardized structure, semantic coherence, and reliable reasoning for large language models.

[0036] In some embodiments, the code sequence compression method combining semantic analysis and attention further includes: Based on a preset window sliding along the source code sequence, the local attention features and semantic summaries of the code subsequences within each sliding window are determined; Based on the semantic summary of the code subsequences within each sliding window and the feature similarity of each node in the original syntax topology distribution network, the nodes associated with the code subsequences within each sliding window in the original syntax topology distribution network are determined.

[0037] Specifically, in this embodiment, a preset window is used to slide along the source code sequence. Local features are extracted from the code sub-sequence within each window to obtain a condensed local attention activation matrix representing the mutual attention between words within that window. Simultaneously, the contextual feature vectors of words near the center of the window are extracted as the semantic summary of that window. Optionally, the semantic summary of the code sub-sequence extracted within the sliding window can be compared with the pre-stored features of each node in the original grammatical topology distribution network. Based on the similarity, the code sub-sequence is accurately matched and associated with the corresponding grammatical nodes in the topology network, thereby establishing a reliable mapping relationship from the text sequence to the structural network. This efficiently achieves the alignment and fusion of local semantic features and global structure, effectively improving the accuracy and reliability of redundant node identification and ensuring the logical integrity of the code compression process.

[0038] For example, this application completes the original syntax topology distribution network. After its construction, the focus shifted to how to effectively align the "local semantic awareness" of the large language model with the network's "global structural knowledge" of code sequences under limited computing resources. Optionally, the large language model processes sequences through a self-attention mechanism, generating attention maps that contain an understanding of local syntax and latent semantic relationships in the code. However, this understanding is data-driven, black-box, and insensitive to long-range structural dependencies. Directly performing global attention computation on the entire sequence to extract features would result in computational complexity... ( For long code files, the length of the sequence is unacceptable. Therefore, an efficient local scanning and global alignment mechanism must be designed.

[0039] The core analysis of this application lies in the fact that, although globally dense computation is not possible, strategic local sampling can be performed, and pre-built [data / features] can be utilized. As a prior knowledge graph, the sampled local features are "anchored" to specific nodes in the network. This is essentially a cross-modal feature retrieval and alignment process, the goal of which is to establish a mapping relationship between the statistical, continuous vector space representation of the large language model and the symbolic, discrete program analysis graph, thereby providing interpretable and computable input for subsequent intelligent decision-making based on network topology.

[0040] For example, the specific implementation process is as follows: Define a sliding window of a fixed size. Its size Based on experience, it usually corresponds to a syntactically complete block of code, such as a function body or a complex control flow block. The range of experience values ​​can be set to Each token. This window With step size (For example ) along the original source code sequence The process involves sliding the window. For each window position, its corresponding word subsequence is extracted. For each word subsequence The input is fed into the encoder layer of a pre-trained large language model (e.g., CodeLlama, StarCoder, etc.), but instead of generating a complete sequence, it is only forward propagated to an intermediate layer (e.g., the penultimate layer). The mean of the attention weight matrices of all attention heads in that layer is extracted or some kind of aggregation is performed (e.g., taking the maximum value), resulting in a condensed local attention activation matrix that represents the mutual attention between words within that window. Simultaneously, the contextual feature vectors of words near the center of the window are extracted. This serves as a semantic summary of the window.

[0041] Next, initiate the feature alignment process. The window needs to be... and its characteristics Mapped to On one or more nodes in the code. First, find the location using the code location mapping table. All morphemes in the original The corresponding syntax nodes. Select these nodes in... The corresponding one with the highest betweenness centrality Nodes with either in-degree or out-degree values ​​can be used as "anchor nodes". . Typically, this is a key hub within the global logic of the code window (such as a function entry point or loop condition checkpoint). Then, feature association is performed. A semantic summary of the calculated window is then generated. and middle The pre-stored features of its first-order neighbor nodes (which can be obtained from...) Similarity between nodes (obtained through node type, variable name embedding, etc.) . This could be a cosine similarity or the output of a small neural network matcher. Ultimately, the window's local attention activates the features. (can be done by...) (The one-dimensional vector obtained by pooling) is formally associated with The node with the highest similarity Above, recorded as .

[0042] This process makes The nodes not only contain syntactic and dependency information, but also additionally attach local semantic attention features interpreted from the perspective of a large language model. This completes the feature alignment from local scanning to the global network, laying the data foundation for subsequent calculation of the "salience" of each node to the model.

[0043] The method described above, through sliding window local scanning and feature extraction, accurately aligns local semantic perception with the global structural knowledge of the original grammatical topology distribution network while controlling computational complexity. It anchors the local attention features of code subsequences to the corresponding nodes of the topology network, enabling the topology network nodes to simultaneously carry grammatical dependency information and semantic attention information from the model's perspective. This provides an accurate and interpretable feature foundation for subsequent calculations of node comprehensive maintenance and redundant node identification, ensuring both the efficiency of long code processing and significantly improving the accuracy of feature matching and topology alignment, effectively guaranteeing the logical integrity and reliability of code compression.

[0044] In some embodiments, determining a threshold value for the comprehensive maintenance degree based on the comprehensive maintenance degree value of each node in the original syntax topology distribution network includes: Based on the comprehensive retention value of each node in the original grammatical topology distribution network, determine the mean and variance of the comprehensive retention value; The threshold for the overall retention rate is determined based on the mean and variance of the overall retention rate value.

[0045] Specifically, in this embodiment, by statistically analyzing the mean and variance of the comprehensive maintenance degree values ​​of all nodes in the original syntax topology distribution network, and dynamically calculating an adaptive comprehensive maintenance degree value threshold based on the mean and variance of the comprehensive maintenance degree values ​​of the nodes, it can effectively avoid the problem of insufficient compression due to setting the static threshold too high, or the problem of key nodes being mistakenly deleted due to setting it too low. It can achieve accurate differentiation between core nodes and redundant nodes in the original syntax topology distribution network, significantly improve the rationality and reliability of redundant node removal, and enable code compression to fully preserve the program logic connectivity while achieving efficient simplification.

[0046] For example, this application completes the comprehensive maintenance degree value of all nodes. After the calculation, the next core decision problem is: how to set a reasonable and adaptive threshold to accurately distinguish between core nodes that should be retained and redundant nodes that can be removed.

[0047] Traditional static thresholding methods have inherent limitations because they cannot adapt to the drastic fluctuations in the distribution of node importance within different source code files. For complex code with densely packed critical nodes, setting the static threshold too high can lead to over-retention and poor compression; while for simple code with sparse critical nodes, setting the threshold too low may mistakenly delete important nodes, causing semantic breaks. Therefore, it is necessary to design a constraint mechanism that can dynamically adjust based on the inherent statistical characteristics of each batch of processed data.

[0048] This application focuses on how to start from the current batch node. From the distribution characteristics, a robust discrimination boundary is automatically derived. An ideal constraint It should be able to automatically adapt to the central tendency and dispersion of the data distribution. If Setting it too aggressively (too low) will retain too many low-value nodes, weakening the compression effect; if Setting the constraints too conservatively (too high) will strip away nodes that should be retained, disrupting program logic. Therefore, the calculation of constraints must be based on the actual statistics of the current data, enabling it to intelligently "tighten" or "relax" the screening criteria, thereby achieving a context-adaptive, data-driven node importance filtering.

[0049] For example, the specific implementation process is as follows: First, collect the original syntax topology distribution network in the current processing batch. All nodes Comprehensive maintenance value This forms a set of numbers. ,in This represents the total number of nodes in this batch. Next, the statistical characteristics of this set are calculated, including the mean. and standard deviation Mean This reflects the average importance of nodes in this batch, while the standard deviation reflects the average importance. This quantifies the dispersion of importance values. Subsequently, the adaptive lower limit constraint is dynamically calculated. :

[0050] In this formula, The threshold value represents the adaptive extreme value lower limit constraint and serves as a dynamic boundary for determining whether a node is redundant. It represents the average value of the overall maintenance degree of all nodes in the current batch, and it serves as a baseline to define the central position of importance. The standard deviation represents the range of variation of importance values ​​around the mean. It is the standard deviation scaling factor, a key hyperparameter that controls how strict the constraint boundary is relative to the data dispersion. The larger the value, The lower the threshold, the more lenient the screening criteria, and the more nodes will be retained. The smaller the value, The higher the value, the stricter the screening criteria, and the more nodes are stripped away. This applies to the standard deviation scaling factor. Its empirical value range is set as This range is set based on extensive experimental verification: when When constraints are too strict, it is easy to mistakenly delete valuable nodes that are in the long tail distribution in batches with large distribution dispersion; when At this point, the constraints are too lenient, leading to a significant decrease in compression efficiency. Therefore, taking values ​​within this range can achieve a better balance between compression ratio and logic preservation.

[0051] The method described above, by dynamically calculating an adaptive threshold based on the mean and standard deviation of the comprehensive maintenance value, can accurately distinguish between core nodes and redundant nodes in the original grammatical topology distribution network. This avoids problems such as insufficient compression or accidental deletion of key nodes caused by static thresholds. While efficiently simplifying the code sequence, it fully preserves the core grammatical structure, data dependencies, and control flow connectivity of the program, greatly improving the rationality and reliability of redundant node removal and providing high-quality compressed input for model inference.

[0052] In some embodiments, removing redundant nodes from the original syntax topology distribution network includes: Nodes in the original syntax topology distribution network whose comprehensive maintenance value is less than the comprehensive maintenance value threshold are identified as redundant nodes. Remove redundant nodes from the original syntax topology distribution network.

[0053] Specifically, after determining the threshold value for the comprehensive maintenance degree, this application identifies nodes in the original syntax topology distribution network whose comprehensive maintenance degree value is lower than the threshold value as redundant nodes, removes them from the original syntax topology distribution network, and simultaneously deletes the code segments corresponding to these nodes, ultimately obtaining the updated syntax topology distribution network and the compressed and simplified code sequence.

[0054] For example, this application calculates the threshold value of the comprehensive maintenance degree. Then, traverse the original syntax topology distribution network. Each node in Then, perform the judgment and stripping operation. The specific process of the stripping operation is as follows: for any node... If its overall maintenance value satisfy If so, the node is determined to be redundant. It is then removed from memory. Network node set The nodes are removed from the original source code text sequence, and all tokens covered by that node are simultaneously deleted from the original source code text sequence. After traversing and filtering all nodes, the remaining nodes that have not been stripped and their corresponding source code tokens are reorganized to form a set that is significantly simplified in terms of sequence length and structural complexity, namely the initial convergent text sequence set described in this invention. This set serves as the input for subsequent topology bridging and reconstruction operations, and it contains only core code logic units that are determined to have high local semantic salience or global structural hub value.

[0055] The method described above accurately identifies and removes redundant nodes with low structural value and low semantic attention by using an adaptive comprehensive retention threshold. While significantly shortening the code sequence length and reducing the complexity of the model input, it fully preserves the core syntactic units, data dependencies, and control flow hub nodes. This effectively avoids the problems of accidental deletion of key nodes and destruction of logical connectivity that are prone to occur in traditional compression methods, ensuring that large language models can still stably and accurately complete code understanding and reasoning under compressed input.

[0056] In some embodiments, after removing redundant nodes from the original syntax topology distribution network based on the comprehensive retention value threshold and the comprehensive retention value of each node in the original syntax topology distribution network, and obtaining the updated syntax topology distribution network and the compressed code sequence, the method further includes: Based on the updated syntax topology distribution network and the compressed code sequence, determine the source and target nodes of logical chain breaks in the updated syntax topology distribution network; Based on the position information of the source node and the target node in the original syntax topology distribution network, as well as the parameters or context features passed between the source node and the target node, the compensation identifier is determined and injected into the compressed code sequence.

[0057] Specifically, this application, after removing redundant nodes from the original syntax topology distribution network based on an adaptive comprehensive maintenance degree threshold to obtain an updated syntax topology distribution network and a compressed code sequence, further identifies the locations of dependency breaks caused by the removal of redundant nodes, determines the source and target nodes of the logical chain break, and generates compensation identifiers for repairing dependencies based on the position information of the source and target nodes in the original syntax topology distribution network, as well as the parameters and context features passed between the source and target nodes. The compensation identifiers are then injected into the compressed code sequence, thereby reconstructing the broken execution dependencies and data transmission links, effectively eliminating the semantic breaks caused by code compression.

[0058] For example, this application, after completing the stripping of redundant nodes and generating a set of initial convergent text sequences, Then, a new key problem arises: while the stripping operation achieves compression in terms of sequence length, it may also destroy the original syntax topology distribution network. Direct connectivity of dependencies. For example, a stripped node. It may have been a source node. The role of directly dependent relay nodes is to pass data or control flow to deeper target nodes. Remove back, and A logical break will occur between them, resulting in... Direct reasoning can lead to incorrect semantic understanding or execution path deviations due to missing dependencies. This phenomenon is particularly common when simplifying code blocks, eliminating intermediate variables, or encapsulating inlined calls.

[0059] To address this issue, this application focuses on how to dynamically compensate for broken logical links caused by missing nodes without restoring redundant nodes that have already been removed. The ideal solution is not simply to revert to the deletion history, but rather to utilize... The preserved complete global topological knowledge allows for the re-establishment of a "virtual channel" from the source node of the broken link to its final logical destination. This requires embedding a lightweight navigation metadata that can be understood and executed by a Large Language Model (LLM) into the compressed text sequence. This guides the model to cross the semantic gap caused by compression and directly connect to deep nodes that originally had indirect dependencies.

[0060] For example, the specific implementation process is as follows: First, the set of initial convergent text sequences... The simplified network structure is then traversed and analyzed to identify all logical broken links. The logic for broken link identification is defined as follows: In the original syntax topology distribution network... If a dependency path exists ,in It has been identified as a redundant node and from Removed from, and and All are retained in If it is in the middle, then it is determined that it is in In the context, There is a point pointing to The broken chain. The source node is referred to as the broken link. That is in Guided by this, a deeper target node was found. A broken link was identified. Afterwards, a cross-level pointer metadata compensation identifier needs to be generated for it. The core is encoding two aspects of information: topological position jump differences and necessary parameter propagation residuals. First, from... Extract and Normalized topological location identifiers in the original source code sequence and These identifiers can be the center point of the term index range, or composite coordinates encoded according to the depth and breadth of the AST. Secondly, analysis... Reached via the stripped node The original path is used to extract key parameters or contextual features propagated along that path. These features are abstracted into parameter propagation residual features. For example, if what is being stripped is a function call wrapper, It may contain high-level semantic information such as the name of the called function, the order of the parameter list, and the type signature. Dimensions Requires the hidden layer dimension of LLM Alignment or adaptation is typically achieved through a small projection layer. This is then processed via a non-linear state-structured mapping mechanism. The position difference and residual features are fused and encoded to generate a set of topological displacement bias features. That is, the final compensation identifier :

[0061] In this formula, It represents the relative displacement vector from the source node to the target deep node in the original topological space. This is a nonlinear state-structured mapping processing mechanism, which can be a lightweight neural network (such as a multilayer perceptron, MLP). Its function is to map discrete position jumps and semantic residuals into a continuous, dense, and highly compressed set of topological displacement bias features. Treating this as "pointer metadata" means that its physical meaning is to tell the model: "Starting from here, it is necessary to combine..." The described context spans a semantic distance of Only by eliminating logical gaps can we reach the correct subsequent execution point.

[0062] Finally, the injection operation is performed. For each identified broken chain, the injection is performed on its source node. Following the corresponding lexical position (or via a special marker), the generated cross-level pointer metadata compensation identifier will be... As a special, non-natural language lexical unit, it is injected into After performing the above identification, generation, and injection process on all broken links, the original... It is transformed into a new set of sequences containing core code text and embedded navigation metadata, namely the topology bridging fusion sequence set described in this invention. This collection not only preserves the core logic after compression, but also implicitly repairs the key dependency links that were broken due to compression through structured metadata, providing crucial structural guidance for the correct inference of subsequent models on compressed inputs.

[0063] It should be noted that this application, by injecting structured cross-level pointer metadata, losslessly repairs the logical chain break caused by physical stripping, maintaining the coherence of long-range causal reasoning. The generation and injection of the inventive technical feature "cross-level pointer metadata compensation identifier" is a direct and innovative solution to the side effects (logical chain break) that may be caused by the first inventive point. After redundant nodes (often relay nodes) are stripped, it can actively identify the source node and target deep node of the chain break, and use the nonlinear state structured mapping processing mechanism F_encode to generate a vector containing the relative displacement vector from the source node to the target deep node in the original topology space. and parameter transfer residual characteristics Topological displacement bias feature set This identifier is not simple text, but rather encapsulates "navigation information" across the stripped regions. Its beneficial effect lies in its ability to reconstruct the broken call chains within the physically compressed sequence with extremely low data overhead, bridging the previously fragmented causal logic. This allows the large language model to understand and reason across code levels based on this injected metadata during subsequent processing, thus enabling the construction of a coherent "strongly connected inference execution flow topology set" on the compressed sequence, resolving the problem of model inference interruption caused by broken chains.

[0064] The method described above accurately locates logical breaks based on the global structural knowledge of the original grammatical topology distribution network. It generates lightweight compensation identifiers by encoding topological location information and parameter transmission residuals and injects them into the compressed sequence. This reconstructs the broken data dependencies and control flow links, effectively eliminating semantic breaks caused by compression. It provides clear structural navigation guidance for large language models and significantly improves the accuracy of code understanding and reasoning coherence of the model under compressed input.

[0065] In some embodiments, after injecting the compensation identifier into the compressed code sequence, the method further includes: Determine the self-attention mask distribution corresponding to the compressed code sequence; The compensation identifier in the compressed code sequence is parsed, and the mask elements corresponding to the source and target nodes in the self-attention mask distribution are modified according to the position information of the source and target nodes to obtain the modified self-attention mask distribution. The modified self-attention mask distribution is input into the large language model.

[0066] Specifically, after injecting the compensation identifier into the compressed code sequence, this application can determine the corresponding self-attention mask distribution based on the compressed code sequence with the injected compensation identifier. Then, the compensation identifier in the compressed code sequence is parsed to extract the position information of the source node and the target node. Based on the position information, the mask elements corresponding to the source node and the target node in the self-attention mask distribution are modified in a targeted manner to obtain the modified self-attention mask distribution. The modified self-attention mask distribution is then input into the large language model to participate in the self-attention calculation, which effectively enhances the model's attention to the repair logic link after the break, avoids the model ignoring the topology guidance information carried by the compensation identifier, and effectively prevents the model from experiencing inference interruption and semantic understanding deviation.

[0067] For example, this application successfully constructs and injects a set of topology bridging fusion sequences with cross-level pointer metadata compensation identifiers. Then, a key parsing and mapping problem arises: As a hybrid sequence containing natural language lexical units and special embedded metadata, the compensation identifier... These are entirely new and unseen symbols for subsequent large language models. The model itself does not have the capability to directly understand them. The ability to encapsulate structured navigation semantics internally. If the physical location of these markers cannot be accurately located from the sequence, and the target node information encoded within them cannot be converted into index coordinates that the model's self-attention mechanism can directly manipulate, then the carefully constructed bridging logic in the preceding steps will completely fail, and the model will still face logical disconnections.

[0068] Therefore, the core problem addressed by this application is: how to design an automated mechanism for merging topology bridging sequence sets. Perform efficient scanning and parsing to accurately identify the metadata compensation identifier of each embedded cross-level pointer. The absolute index position in the current sequence is determined, and the index position of the target deep node associated with that identifier in the original global topology is decoded simultaneously. This process is essentially a combination of "metadata parsing" and "coordinate reverse mapping." It acts as a translator and locator connecting the structured navigation instructions of non-natural language with the underlying attention calculation matrix of the model. It is a necessary prerequisite for transforming high-level semantic bridging intentions into low-level executable intervention operations.

[0069] The specific implementation process is as follows: First, the topology bridging fusion sequence set is processed. Perform a linear scan. Because during the injection phase, each compensation identifier... Each token is inserted as an independent special lexical with a unique feature pattern, and can be matched based on a predefined lexical ID or feature vector matching algorithm (such as cosine similarity exceeding a threshold). experience points To identify them. Assume that the first... A compensation identifier, recorded in The lexical index position is This location This is defined as the "source index position" of the bridging instruction. Next, this identifier needs to be decoded. The associated "target node index location". This identifier encapsulates a set of topological displacement offset features during generation. The target node is encoded in the original syntax topology distribution network. Topological location identifier in Call a function that is similar to... Corresponding lightweight decoder The decoder receives As input, the normalized topological position coordinates of the target node are reconstructed through an inverse nonlinear mapping. :

[0070] in Usually a pair A small neural network trained symmetrically ensures that information loss during the encoding-decoding process is minimized. Normalized topological coordinates of the target node are obtained. Then, it needs to be mapped back to the current state. Target node index position in the sequence This mapping is achieved by querying a pre-defined "global topological coordinates-lexicon index" mapping table. To achieve this. Because yes The stripped subset retains each node that is recorded in both the original network and the current sequence. Search and The nearest node coordinates that exist in the current sequence Then get the node in The corresponding starting word index is used as the target node index position. This location This is defined as the "index position of the associated target node". Ultimately, for For each compensation identifier identified, an index position pair is output. This position precisely indicates two key locations that need to be forcibly associated in subsequent self-attention calculations: the source index position ( ) and its associated target node index location ( The set of all position pairs. As the core output, where m represents the m-th identifier, it provides the absolutely necessary coordinate input for precise and targeted forced intervention of the self-attention mask distribution, ensuring that the bridging logic can be accurately "welded" into the model's inference path.

[0071] The method described in the above embodiments, by parsing and mapping the compensation identifier in reverse coordinates, can automatically and accurately locate the source node and target node index positions corresponding to the logical chain break, providing a reliable coordinate basis for the directional intervention of the self-attention mask. This allows for precise modification of the mask elements at the corresponding positions of the source node and target node in the self-attention mask distribution, obtaining a self-attention mask distribution adapted to the compressed code sequence. This enhances the model's attention to the repaired logical chain, ensures that the model effectively identifies the topology guidance information carried by the compensation identifier, and avoids inference interruption and semantic understanding deviation.

[0072] In some embodiments, modifying the mask elements corresponding to the source node and the target node in the self-attention mask distribution includes: Apply a positive bias intervention to the mask elements; the positive bias intervention is used to enhance the attention of the large language model to the logical link from the source node to the target node.

[0073] Specifically, this application applies a positive bias to the mask elements corresponding to the source node and the target node in the self-attention mask distribution, thereby increasing the mask value at the corresponding position and forcibly enhancing the model's attention weight to the repaired logical link. This enables the model to fully focus on the topological relationship indicated by the compensation identifier, avoiding the model ignoring the compensation identifier as a newly added unknown symbol, and ensuring that the key logical link is effectively identified and utilized.

[0074] For example, this application accurately identifies the source index location of all cross-level pointer metadata compensation identifiers. and the index location of its associated target node And form a set of position pairs Then, the final and most critical engineering challenge lies in achieving the goal. This is due to the injected compensation markers. As special symbols unseen by the model, their representation in the embedding space is a random or zero-valued initialization state that has not been pre-trained. When the model's self-attention mechanism is computed, due to the probability normalization property of the Softmax function, features at these unknown locations are inherently at a disadvantage in the query-key dot product interaction. Their corresponding attention weights are easily diluted to extremely low levels close to zero, causing all preceding bridging efforts to fail. Therefore, a mechanism must be established to forcibly correct this "ignoring" behavior of the model, compelling it to focus on the features at the key-key dot product. The indicated logical link.

[0075] This application focuses on how to precisely and forcefully increase the model's attention to specific position pairs without changing the model weights or performing secondary training, solely through rule-based intervention in a specific stage of the self-attention computation process. This requires the intervention to occur at the source of the attention weight distribution generation—the "logarithmic its" stage, after the dot product of the query vector and the key vector but before Softmax normalization. This is achieved through specific coordinates at this stage. Applying a very large positive bias value can directly bypass the model's conventional attention calculation logic based on content similarity, artificially creating a weight "high ground," thereby forcing the model to focus on the position. When receiving information, one must highly concentrate their attentional resources on the location. This forces a logical jump.

[0076] For example, the specific implementation process is as follows: Obtain the original self-attention mask distribution of the current sequence to be processed. In causal language modeling scenarios, Typically, it's a lower triangular matrix (including the diagonal), where the mask value for areas of interest is 0, and the mask value for occluded areas is a very large negative number (e.g., ...). For from a set Each position extracted from the middle ,in , For the original self-attention mask distribution In coordinates element at Force modification:

[0077] In this intervention formula, These are the modified mask distribution elements. and These represent the source index position and the target index position, respectively. It is a forced attention energy control gain factor, whose function is to... A large positive bias signal is injected directly at the coordinates to overwhelm the original attention logits (its) calculated by the model based on content. To ensure the absolute effectiveness of the intervention, The empirical value must be large enough, and its value should not be less than [a certain value]. . It is a distance attenuation control parameter, which adjusts the intervention intensity as a function of the absolute distance between the source and target positions. The rate of increase and decrease. The function ensures that the attenuated gain value is smoothly constrained within a certain range. Within the interval, and then with Multiplying them together yields the final bias. A larger value indicates slower decay and stronger support for interventions involving long-distance jumps; a smaller value indicates faster decay and more focused interventions on short-distance jumps. Based on engineering practice, its value range is set as follows: . It is a minimum constant to prevent zero overflow, and is usually taken as... Used to prevent when The numerical calculation error was caused by the denominator being zero. The solution is to iterate through and modify all position pairs. Afterwards, the original self-attention mask distribution Updated to a new mask matrix with a forced positive bias intervention. .this The core feature of a matrix is ​​that, within a specific context defined by all bridging instructions... On the coordinate system, its value increased by several orders of magnitude compared to the original value. When this... These large positive values ​​are used in subsequent self-attention calculations and are directly passed to the input of the Softmax function, making the calculation position... When distributing attention, it is assigned to the position. The weight probabilities are amplified dramatically and irresistibly, thereby achieving the fundamental goal of "enhancing the model's attention to bridging logic." This intervention is local and precise, affecting only specified coordinate pairs, while the attention calculations at other positions in the sequence fully follow the model's original mechanism. Thus, while forcibly establishing key bridging, the model's original semantic understanding capabilities are preserved to the greatest extent.

[0078] Optionally, complete the distribution of the original self-attention mask. Forced intervention and generation Subsequently, strong constraints were pre-set for the model's attention weight allocation. However, only a new mask matrix with a mandatory positive bias intervention was available. It cannot directly generate any inference output; it is merely a static intervention template used to guide the probability distribution. This application focuses on how to seamlessly and forcefully integrate the complex "instruction set" and "navigation map" constructed by this series of preliminary steps (from structure awareness, node stripping, bridging injection to mask intervention) into the core forward computation process of the model, ultimately driving the model to generate expected and logically coherent inference results. This requires the implementation process to precisely... We embed a standard attention computation framework and explain how it ultimately leads to a structurally complete output. The core issue is that the generative capability of large language models is rooted in their token-based autoregressive prediction mechanism, and each prediction step relies on a weighted summation (i.e., attention) of all positions in the input sequence. The matrix is ​​used for specific coordinates in this weighted summarization process. The "priority" of weight allocation was artificially and significantly increased.

[0079] Therefore, the specific implementation process involves performing standard attention calculations, but will As a key input, this priority is transformed into the contextual information that the model actually relies on when generating the next lexical. Ultimately, driven by this "corrected" attention horizon, the complete output sequence generated by the model naturally follows the reasoning path guided by cross-level pointer metadata, forming a strongly connected logical whole.

[0080] For example, the specific implementation process is as follows: The topology bridging fusion sequence set is... The input is fed into a large language model. The model first transforms the word sequence into an initial feature vector sequence through its embedding layer. Then, in the self-attention computation module of the target Transformer decoder layer, the reconstructed attention computation is performed:

[0081] In this core calculation formula, This represents the attention weight distribution matrix finally calculated after intervention by this methodology. It is a query feature set, which is obtained by linearly transforming the hidden state features of each position in the current sequence. Its function is to actively initiate "queries" to other positions in the sequence. It is the transpose of the key feature set, which is obtained by another linear transformation and transpose of the hidden state features of all positions in the sequence. Its function is to provide an "identifier" for each position that can be queried and matched. It is usually a fixed hyperparameter (e.g., 4096 or 8192). The scaling operation is to prevent the dot product result from becoming too large, which would cause the Softmax gradient to vanish. This is the new mask matrix with a forced positive bias applied. The calculation process begins with the dot product of the query and the key. This yields the original attention logarithm `its` based on content similarity; subsequently, a new mask matrix with a mandatory positive bias intervention is applied. This is added directly to the original attention logarithm (its) matrix. This addition operation is crucial: for all identified specific coordinates... ,because A huge forced attention energy control gain factor was imposed. This makes it possible to be in position In the corresponding row, the first The column values ​​were significantly inflated, far exceeding other values ​​calculated based on content. Then, the values ​​were applied row-by-row to the superimposed matrix. The function is normalized. Because... The exponential amplification effect, at position In the process, that cause The intervention that has a huge absolute value Column elements will be assigned extremely high probability weights close to 1, while the weights in other positions will be compressed to close to 0. This perfectly achieves the goal of "driving the model to infer based on the guidance of the cross-level pointer metadata compensation identifier." Ultimately, this... Weighted log-valued feature set Perform a weighted summation to generate the output of the current attention head, and continue with subsequent forward propagation and autoregressive generation.

[0082] Through the above process, when the model generates each subsequent word, its "attention focus" is... The structured bridging logic it carries powerfully shapes the system. Long-range dependencies that might have broken due to redundant node stripping are re-established through forced high-weight attention. The final complete sequence or inference result output by the model—the so-called "strongly connected inference execution flow topology set"—is not an explicit graph structure, but rather refers to a high-quality semantic output whose internal logical dependencies (reflected in the model's attention flow and feature aggregation) maintain a high degree of consistency with the connectivity of the initial grammatical topology network, without any broken links. This output set is functionally equivalent to the ideal result the model should obtain after processing the uncompressed original long sequence, but achieves a significant improvement in computational efficiency due to input compression, thus fully realizing all the design goals of this method.

[0083] It should be noted that this application ensures that the model gives sufficient attention to the injected bridging metadata through mandatory self-attention mask bias intervention, achieving logical penetration guidance without model retraining. Another inventive technical feature is the distribution of the original self-attention mask. Mandatory and targeted modifications ( This is crucial to ensuring that the aforementioned bridging logic can be effectively utilized by the model. This is due to the injected topological displacement bias feature set. These are unknown features beyond the model's pre-trained knowledge, and the model's native attention mechanism will instinctively suppress their weights. This invention addresses this by assigning weights to specific coordinates within the mask matrix. i,j Apply a huge forced attention energy control gain factor. The dominant positive bias logically implies that this operation will force the model to significantly increase its attention to this artificial bridging path when calculating attention. Its direct benefit is that, without any model parameter fine-tuning or retraining, it successfully guides the model's internal computational process through input-level intervention alone, thereby altering its attention weight distribution matrix. It strictly followed the logical navigation path defined by the injected metadata. This ultimately enabled the model to accurately understand the logical connections across the original blank areas in the compressed sequence, overcoming the final hurdle from "logic repair" to "model acceptance".

[0084] The method described above applies precise positive bias intervention to the elements corresponding to the self-attention mask. Without modifying model parameters or retraining, it forcibly increases the model's attention weight to the logical links between the source and target nodes indicated by the compensation identifier. This fundamentally prevents the model from ignoring key topological information due to the compensation identifier being an unknown symbol, effectively reconstructs broken dependencies during compression, and maintains strong connectivity between code execution flow and semantic logic. While significantly improving input compression efficiency, it also enables the model to output high-quality, unbroken inference results consistent with those processed from the original long code.

[0085] For example, such as Figure 2 As shown, this application provides a code sequence compression method that combines semantic analysis and attention. The specific process is as follows: First, lexical and syntactic analysis is performed on the source code sequence to generate the corresponding syntax tree, and an original syntax topology distribution network is constructed based on the node dependencies of the syntax tree. Then, local attention features of the code subsequences are extracted using a sliding window, and the comprehensive maintenance value of each node is calculated by combining the betweenness centrality of the node network. Next, an adaptive threshold is determined based on the statistical characteristics of the comprehensive maintenance value, and nodes below the threshold are identified as redundant nodes and removed, resulting in an updated syntax topology distribution network and a compressed code sequence. Then, logical breaks in the compressed network are identified, and compensation labels are generated based on the position information and transmission characteristics of the source and target nodes and injected into the compressed code sequence. Finally, the self-attention mask distribution corresponding to the sequence is determined, the compensation labels are parsed, and positive bias intervention is applied to the mask elements corresponding to the source and target nodes. The modified mask distribution is input into the large language model to enhance the model's attention to the repaired logical links and ensure the semantic coherence and execution flow integrity of the code reasoning.

[0086] The following describes the code sequence compression apparatus combining semantic analysis and attention provided by the present invention. The semantic analysis and attention-based code sequence compression apparatus described below can be referred to in correspondence with the semantic analysis and attention-based code sequence compression method described above. For example, such as... Figure 3As shown, the code sequence compression device that combines semantic analysis and attention includes: Analysis module 310 is used to perform lexical and syntactic analysis on the source code sequence and generate a syntax tree corresponding to the source code sequence; The generation module 320 is used to generate the original syntax topology distribution network based on the dependencies between nodes in the syntax tree; The determination module 330 is used to slide along the source code sequence based on a preset window, and determine the comprehensive maintenance value of each node in the original syntax topology distribution network according to the local attention characteristics of the code subsequences in each sliding window and the network betweenness centrality of the nodes associated with the code subsequences in each sliding window in the original syntax topology distribution network. Processing module 340 is used to determine the threshold value of comprehensive maintenance degree based on the comprehensive maintenance degree value of each node in the original syntax topology distribution network; Compression module 350 is used to remove redundant nodes in the original syntax topology distribution network based on the comprehensive maintenance degree threshold and the comprehensive maintenance degree value of each node in the original syntax topology distribution network, so as to obtain the updated syntax topology distribution network and the compressed code sequence.

[0087] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440. The processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions from the memory 430 to execute a code sequence compression method that combines semantic analysis and attention.

[0088] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to perform the code sequence compression method that combines semantic analysis and attention provided by the above methods.

[0089] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a code sequence compression method combining semantic analysis and attention provided by the above methods.

[0090] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0091] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A code sequence compression method combining semantic analysis and attention, characterized in that, include: Lexical and syntactic analysis are performed on the source code sequence to generate a syntax tree corresponding to the source code sequence; Based on the dependencies between nodes in the syntax tree, an original syntax topology distribution network is generated; Based on the preset window sliding along the source code sequence, the comprehensive maintenance value of each node in the original syntax topology distribution network is determined according to the local attention characteristics of the code subsequences in each sliding window and the network betweenness centrality of the nodes associated with the code subsequences in each sliding window in the original syntax topology distribution network. Based on the comprehensive maintenance degree value of each node in the original syntax topology distribution network, the threshold value of the comprehensive maintenance degree value is determined; Based on the comprehensive retention threshold and the comprehensive retention value of each node in the original syntax topology distribution network, redundant nodes in the original syntax topology distribution network are removed to obtain the updated syntax topology distribution network and the compressed code sequence.

2. The code sequence compression method combining semantic analysis and attention according to claim 1, characterized in that, The method further includes: Based on a preset window sliding along the source code sequence, the local attention features and semantic summaries of the code sub-sequences within each sliding window are determined; Based on the semantic summary of the code subsequences within each sliding window and the feature similarity of each node in the original syntax topology distribution network, the nodes associated with the code subsequences within each sliding window in the original syntax topology distribution network are determined.

3. The code sequence compression method combining semantic analysis and attention according to claim 1, characterized in that, The step of determining the threshold value of the comprehensive retention degree based on the comprehensive retention degree value of each node in the original syntax topology distribution network includes: Based on the comprehensive retention value of each node in the original syntax topology distribution network, determine the mean and variance of the comprehensive retention value; The threshold value for the comprehensive maintenance degree is determined based on the mean and variance of the comprehensive maintenance degree value.

4. The code sequence compression method combining semantic analysis and attention according to claim 3, characterized in that, The process of removing redundant nodes from the original syntax topology distribution network includes: Nodes in the original syntax topology distribution network whose comprehensive maintenance value is less than the comprehensive maintenance value threshold are identified as redundant nodes. Remove the redundant nodes from the original syntax topology distribution network.

5. The code sequence compression method combining semantic analysis and attention according to any one of claims 1-4, characterized in that, After removing redundant nodes from the original syntax topology distribution network based on the comprehensive retention value threshold and the comprehensive retention value of each node in the original syntax topology distribution network, and obtaining the updated syntax topology distribution network and the compressed code sequence, the method further includes: Based on the updated syntax topology distribution network and the compressed code sequence, determine the source and target nodes of logical chain breaks in the updated syntax topology distribution network; Based on the location information of the source node and the target node in the original syntax topology distribution network, as well as the parameters or context features transmitted between the source node and the target node, a compensation identifier is determined and injected into the compressed code sequence.

6. The code sequence compression method combining semantic analysis and attention according to claim 5, characterized in that, After injecting the compensation identifier into the compressed code sequence, the method further includes: Determine the self-attention mask distribution corresponding to the compressed code sequence; The compensation identifier in the compressed code sequence is parsed, and the mask elements corresponding to the source node and the target node in the self-attention mask distribution are modified according to the position information of the source node and the target node to obtain the modified self-attention mask distribution. The modified self-attention mask distribution is input into the large language model.

7. The code sequence compression method combining semantic analysis and attention according to claim 6, characterized in that, The modification of the mask elements corresponding to the source and target nodes in the self-attention mask distribution includes: A positive bias intervention is applied to the mask elements; the positive bias intervention is used to enhance the attention of the large language model to the logical link from the source node to the target node.

8. A code sequence compression device combining semantic analysis and attention, characterized in that, include: The analysis module is used to perform lexical and syntactic analysis on the source code sequence and generate a syntax tree corresponding to the source code sequence. The generation module is used to generate the original syntax topology distribution network based on the dependencies between nodes in the syntax tree; The determination module is used to slide along the source code sequence based on a preset window, and determine the comprehensive maintenance value of each node in the original syntax topology distribution network according to the local attention characteristics of the code subsequences in each sliding window and the network betweenness centrality of the nodes associated with the code subsequences in each sliding window in the original syntax topology distribution network. The processing module is used to determine the threshold value of the comprehensive maintenance degree based on the comprehensive maintenance degree value of each node in the original syntax topology distribution network; The compression module is used to remove redundant nodes in the original syntax topology distribution network based on the comprehensive maintenance degree threshold and the comprehensive maintenance degree value of each node in the original syntax topology distribution network, so as to obtain the updated syntax topology distribution network and the compressed code sequence.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the code sequence compression method that combines semantic analysis and attention as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the code sequence compression method that combines semantic analysis and attention as described in any one of claims 1 to 7.

Images