Quick code repair method for lightweight compressed context generation

Abstract

The invention belongs to the technical field of software engineering, and particularly relates to a quick code repairing method for lightweight compressed context generation. According to the method, efficient repair is realized by constructing a'retrieval-compression-generation 'collaborative architecture; comprising the following steps: constructing a cross-file code retrieval database, designing a lightweight semantic compression encoder (SCE), and compressing retrieved cross-file code blocks into short tokens with high semantic density; semantic compatibility with a large language model (LLM) is realized through a two-stage training strategy; wherein in the semantic alignment stage, the SCE is trained so that the compression token can be accurately analyzed by the LLM, and in the task alignment stage, the LLM is finely adjusted so that the compression token can be utilized to complete repair. Experiments show that on the premise that key logic dependence is reserved, the reasoning efficiency of the model can be improved, the code repairing effect is guaranteed, the method is suitable for cross-file dependence repairing scenes of a large code warehouse and can be integrated into integrated development environment plug-ins, automatic code auditing tools and the like, and the software development and maintenance efficiency is remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of software engineering technology, specifically relating to a method for rapid code repair based on a lightweight compression context. Background Technology

[0002] As software systems grow exponentially in scale and code complexity continues to rise, code repair, a crucial aspect of ensuring software quality, is facing increasingly severe challenges. Traditional manual repair methods heavily rely on developers' deep understanding of code logic, which is not only time-consuming and labor-intensive but also prone to introducing new errors due to human oversight, making it difficult to meet the dual demands of efficiency and reliability in modern software engineering.

[0003] In recent years, automated code repair technology based on large language models has become a research hotspot. This technology generates repair schemes by learning from large-scale code corpora, significantly reducing the cost of manual intervention. However, existing technologies still have significant limitations when dealing with complex code repair tasks involving cross-file dependencies. These limitations manifest in the following three aspects:

[0004] First, the inherent limitations of retrieval-enhanced generation frameworks. To address the insufficient cross-file context awareness of large language models, existing solutions often employ retrieval-enhanced generation frameworks, which retrieve relevant code blocks from the code repository as auxiliary information input to the model. However, this approach suffers from three major problems: First, the retrieved code blocks often contain a large amount of information irrelevant to the repair task, such as redundant comments, formatting statements, and historical debugging traces, making it difficult for the model to focus on key logic and reducing repair accuracy. Second, long contextual inputs cause an exponential increase in the model's computational load; for example, the length of an input sequence containing multiple code blocks far exceeds the conventional processing range, leading to a significant increase in inference time. Third, when processing extremely long sequences, the model pays significantly less attention to key information in the middle, such as function parameter definitions and variable type declarations, which can easily cause logical breaks or syntax errors in the repair results.

[0005] Second, existing context compression methods lack adaptability. For long context problems, current mainstream compression technologies are primarily designed for natural language and do not fully consider the special characteristics of code. Code contains strict syntactic structures and logical dependencies, such as function call chains and variable scope. Existing compression strategies based on text redundancy easily disrupt these structures, leading to the loss of crucial repair information. Simultaneously, the compression results are not aligned with the model's semantic space, making it difficult for the model to parse the deep logic of the compressed information, thus reducing the repair effect. Furthermore, some compression methods require the introduction of additional modules, increasing the hardware threshold for model deployment and making them difficult to adapt to resource-constrained development environments.

[0006] Third, the unique challenges of code repair tasks. Code repair requires precise matching of cross-file dependencies, such as the association between function calls and class definitions between modules, demanding extremely high integrity and accuracy of the context. Existing technologies either cause dependency breaks due to excessive compression or suffer from inefficiency due to retaining too much redundant information, making it difficult to strike a balance between efficiency and accuracy. Summary of the Invention

[0007] The purpose of this invention is to provide a fast code repair method using lightweight compressed context generation, which significantly reduces inference costs while preserving key cross-file dependency information, enabling efficient and accurate repair of large code repositories.

[0008] The fast code repair method for lightweight compressed context generation provided by this invention addresses the problems of context redundancy, high inference cost, and semantic mismatch in cross-file code repair of the existing Retrieval-augmented Generation (RAG) [1] framework. It achieves efficient repair by constructing a collaborative architecture of "retrieval-compression-generation". The method includes: constructing a cross-file code retrieval database, designing a lightweight semantic compression encoder (SCE) to compress the retrieved cross-file code blocks into short tokens with high semantic density, and achieving semantic compatibility with large language models (LLM) through a two-stage training strategy: semantic alignment and task alignment. In the semantic alignment stage, the SCE is trained so that the compressed tokens can be accurately parsed by the LLM, and in the task alignment stage, the LLM is fine-tuned to use the compressed tokens to complete the repair. The specific steps are as follows:

[0009] (I) Construction of Cross-File Code Retrieval Database

[0010] To accurately obtain the associated context of the code to be repaired, it is necessary to retrieve relevant code blocks from large code repositories and build a retrieval system that supports efficient matching. The specific process is as follows:

[0011] (1) Structured code block segmentation:

[0012] The core principle of automated chunking of the original code repository is to ensure the integrity of the code syntax and logic. For multi-line structures such as function definitions and class declarations, they are automatically expanded to complete syntactic units (e.g., starting from `def func()` and ending with the corresponding closing symbol `}`). For linearly executable code segments, the original code block is divided into basic code chunks every N lines (N can be adjusted according to actual needs, such as every 10 or 20 lines). After segmentation, a syntax parser (such as Python's AST (Abstract Syntax Tree) module) filters out low-quality code chunks: code chunks with more than 50% blank lines or containing only comments or log output are removed, while valid code chunks containing key logic such as variable declarations, function calls, parameter passing, and class inheritance relationships are retained, laying a high-quality data foundation for subsequent retrieval and compression.

[0013] (2) Semantic vector encoding and index optimization:

[0014] A pre-trained model specifically designed for code (such as UniXcoder) is used to semantically encode the filtered code blocks. After converting the code blocks into token sequences, they are input into the pre-trained model to obtain the vector of the last hidden state, which is then averaged as the semantic representation of the code block. This encoding method can capture the deep logic of the code (such as the dependency relationship of "function A calls function B"), rather than just staying at the surface text matching. The vectors of all code blocks and their original content are stored in a vector database (such as FAISS), and an IVF (Inverted File) index is built to accelerate retrieval: the vector space is divided into K clusters through a clustering algorithm, and each cluster maintains a local index, which reduces the retrieval time complexity from O(n) to O(logn), ensuring millisecond-level response in a library of millions of code blocks and meeting the requirements for real-time repair.

[0015] (II) Design of Lightweight Semantic Compressor (SCE)

[0016] To address the inefficiency of inference caused by long context inputs, a lightweight semantic compression encoder (SCE) is designed. This encoder compresses retrieved cross-file code blocks into short tokens with high semantic density while preserving all information of the retrieved document, thus avoiding information loss caused by explicit compression. The SCE consists of two parts: an encoder and a projector. The specific structure and functions are as follows:

[0017] Encoder: Used for code semantic feature extraction and compression

[0018] A lightweight code pre-trained model (such as CodeBERT-base[2], with approximately 110M parameters) is used as the basic encoder. This model is pre-trained on a large-scale code corpus and can accurately identify the grammatical rules (such as variable scope and function nesting relationships) and logical dependencies (such as call chains and return value types) of the code, which is its core advantage over traditional text encoders. The retrieved original code block is used as the input of the encoder. The encoder extracts deep semantic features through the pre-trained model and finally outputs a feature matrix with the shape of [seq_len, hidden_state]. In order to achieve compression, attention-weighted average pooling is performed on the feature matrix: according to the importance of each token in the code (such as function names and parameter lists having higher attention weights), the variable-length sequence is fused into a feature vector of fixed length (such as 96 tokens).

[0019] Projector: Used for semantic space alignment and adaptation

[0020] The projector is a three-layer linear neural network whose core function is to map the feature vector output by the encoder to the semantic space of the generator, solving the problem of "incompatibility between compressed tokens and LLM semantics". For example, the input dimension of the projector is 768 (consistent with the encoder output), the intermediate layer dimension is 2048, the ReLU activation function is used to enhance the non-linear expressive power, and the final output dimension perfectly matches the hidden layer dimension of the target LLM. For example, it maps the 768-dimensional semantic vector to the 4096-dimensional CodeLlama7B. Through this mapping operation, the semantic distribution of the compressed token is highly aligned with the semantic space formed by the LLM during pre-training, enabling the LLM to directly "understand" the deep logic in the compressed information and avoid repair errors caused by "semantic gaps". For example, when the compressed token contains the information that "the parameters of the function add are two integers", the LLM can accurately understand and generate a call statement that conforms to this rule in the repair.

[0021] The total number of parameters in SCE is only a few million, which is far less than the number of parameters in the generator (hundreds of millions). This can significantly reduce the inference cost and is far lower than heavyweight compression methods such as ICAE[5]. It can run efficiently on consumer-grade GPUs (such as NVIDIA RTX 3090), which greatly reduces the deployment threshold.

[0022] (III) A two-stage training strategy is adopted: semantic alignment and task alignment;

[0023] To enable SCE and LLM to collaborate on code repair tasks, a two-stage training strategy is designed to optimize compression quality and repair capability respectively, ensuring that compression tokens can be understood by LLM and effectively assist the repair process:

[0024] Phase 1: Semantic Alignment Training – Enabling LLM to “Understand” Compressed Tokens;

[0025] The core objective of this stage is to train the SCE to generate compressed tokens that can be accurately parsed by the LLM, ensuring that the compressed information can be restored by the LLM to the key logic of the original code block. The specific process is as follows:

[0026] (1) Training data construction: randomly select code blocks from the retrieval library, generate compressed tokens using the current SCE, and then reconstruct the original code blocks based on the user's instructions using the compressed semantic vectors. The compressed semantic vectors are then merged with the original code blocks to form the input part of our model.

[0027] (2) Training Process: Input the above prompt text into the LLM, requiring the LLM to generate content consistent with the original code block based on the compression token. During training, freeze the LLM parameters (to avoid destroying its pre-trained knowledge) and optimize only the encoder and projector parameters of the SCE. By continuously adjusting the SCE parameters, minimize the difference between the content generated by the LLM and the original code block; for example, if the original code block is "def add(a:int,b:int)>int:return a+b", then the LLM is required to be able to recover the core logic of the "integer addition function" from the compression token, including parameter types and return values.

[0028] (3) Convergence Criterion: When the LLM achieves an accuracy of over 90% in reconstructing the original code block (Exact Match), the semantic alignment training is considered complete. This stage ensures that the compressed semantic vectors retain most of the information of the code block, and that large models can reconstruct the code based on this, laying the foundation for subsequent repair tasks.

[0029] Phase Two: Task Alignment Training – Enabling LLM to “Make Good Use” of Compression Tokens

[0030] The goal of this phase is to fine-tune the LLM so that it can combine the compression token with the context of the code to be repaired to generate the correct repair results. The specific process is as follows:

[0031] (1) Training data construction: Collect code fix samples containing cross-file dependencies (such as "incorrect parameter type when module A calls function of module B" or "references an undefined variable from another module"), and then inject the retrieved code blocks into the context of the model. The difference is that the input to the large model is a learnable semantic vector, rather than a visible code token.

[0032] (2) Training process: Freeze the parameters of SCE (to keep the compression quality stable) and fine-tune LLM using low-rank adaptation technique [6]. Specifically, additional parameters are introduced into the query and key matrices in the attention module, and low-rank matrix classification is introduced so that the training parameters do not exceed 1% of the model. Through this lightweight fine-tuning, LLM learns to use cross-file dependency information in the compression token for repair, while avoiding overfitting that leads to a decrease in generalization ability. For example, when the code to be repaired reports an error due to "insufficient number of parameters when calling the function", LLM can locate the definition of the function in other modules (such as "def func(a,b):...") through the compression token and generate a repair result with supplementary parameters.

[0033] (3) Convergence Criterion: Training is stopped when the LLM's repair accuracy (Exact Match) on the validation set no longer improves for three consecutive rounds. This stage ensures that the LLM can efficiently utilize the information in the compression token to improve the accuracy of cross-file code repair.

[0034] (iv) End-to-end code fast repair

[0035] Furthermore, based on the aforementioned retrieval, compression, and training mechanisms, an end-to-end process from "error code input" to "fix code output" is implemented, as follows:

[0036] (1) Code parsing and retrieval triggering to be fixed

[0037] The code to be fixed (such as syntax errors marked in the IDE or code segments where test cases failed) is parsed: A parser is used to locate the error position, identify the error type (such as "undefined variable," "parameter mismatch," "function call error," etc.), and extract contextual information from N lines (e.g., lines 10, 20, etc.) before and after the error position to ensure semantic coherence. The same embedding model used when encoding the codebase is employed to encode the current contextual information to ensure consistency in the retrieved semantic space.

[0038] (2) Retrieval and filtering of related code blocks

[0039] Calculate the cosine similarity between the query vector and all code block vectors in the retrieval database, and select the code block with the highest similarity.

[0040] (3) Code block compression and token generation

[0041] The retrieved k code blocks are input into the trained SCE. After compression by the encoder and mapping by the projector, k manually set adaptive code tokens are generated. The tokens are sorted according to the original retrieval similarity (the token with higher similarity is placed first) to avoid the LLM ignoring key information due to attentional distraction. For example, the function definition tokens that are directly related to the error will be placed first to ensure that the LLM pays priority.

[0042] (4) Fix code generation and optimization

[0043] The input sequence is concatenated using a template: the user's code repair instructions, the code to be repaired, and the code blocks extracted as semantic vectors are concatenated together using special tokens and input into the fine-tuned LLM. To ensure the model's generation efficiency and the final repair effect, the generation parameters are set as follows: max_new_tokens is the length of the code to be repaired + 10 (to ensure complete coverage of the repair content), and temperature = 0.7 (to balance creativity and accuracy). After generation, the grammatical correctness of the repair results is verified using a syntax checker (such as `pylint` or `flake8`). If syntax errors are found, the model can be iterated again using a larger model, or some parameters of the retrieval tool can be modified to increase the number of retrievals or the completeness of semantic information to optimize the results.

[0044] The main technical features and functional advantages of this invention are:

[0045] (1) Lightweight and efficient: The lightweight semantic compression encoder (SCE) compresses code blocks across files. The number of SCE parameters is negligible compared to large models, resulting in low deployment costs. The context length can be adaptively compressed, and the inference time is significantly reduced compared to traditional repair solutions with cross-file dependencies, approaching the efficiency of single-file processing, which greatly improves the user experience. It can run efficiently on consumer-grade GPUs (such as NVIDIA RTX 3090), greatly reducing inference costs and adapting to the rapid iteration needs of modern software engineering.

[0046] (2) High semantic fidelity: Based on the code pre-trained model, SCE can accurately retain key information on cross-file dependencies (such as function call chains, parameter types, variable scope, etc.). Combined with a two-stage training strategy, it enables the Large Language Model (LLM) to deeply understand compressed semantics and adapt to repair tasks. The repair accuracy is 10%-30% higher than that of the traditional Retrieval Augmentation Generation (RAG) framework, effectively reducing repair errors caused by contextual redundancy or semantic loss.

[0047] (3) Strong universality: The SCE parameter scale is only about 1.5% of that of LLM. The lightweight design reduces the hardware deployment threshold and can be integrated into integrated development environment (IDE) plugins, automated code auditing tools, etc. Through two-stage training, it can adapt to a variety of LLMs (such as CodeLlama, Deepseek-Coder[4] etc.) and programming languages ​​(Python, C#, TypeScript etc.), meet the code repair scenarios of different technology stacks, and improve the universality of the solution.

[0048] Experiments show that this invention can accelerate the inference efficiency of the model while preserving key logical dependencies and ensuring the effectiveness of code repair. It is suitable for cross-file dependency repair scenarios in large code repositories and can be integrated into integrated development environment (IDE) plugins, automated code auditing tools, etc., significantly improving software development and maintenance efficiency. Attached Figure Description

[0049] Figure 1 This is a flowchart of the method for quickly repairing code generated by the lightweight compression context of the present invention. Detailed Implementation

[0050] The code to be fixed is located in the project's `order_processing.py` file, which defines a function `calculate_total` to process orders. This function needs to call the `compute_fee` function in the `payment_utils.py` module to calculate the transaction fee. However, because the `compute_fee` function added a parameter (`discount_rate`) in subsequent iterations, and the call statement of the `calculate_total` function was not updated accordingly, a runtime error "TypeError: compute_fee() missing 1 required positional argument: 'discount_rate'" is thrown. The specific code snippet is as follows:

[0051] #order_processing.py (To be fixed)

[0052] from payment utils import compute_fee

[0053] def calculate_total(amount,tax_rate):

[0054] # Calculate transaction fees (This call is missing the newly added discount rate parameter)

[0055] fee=compute fee(amount, tax rate)

[0056] total=amount+fee+(amount*tax rate)

[0057] return total

[0058] Issue to be fixed: Add the discount_rate parameter to the compute_fee(amount,tax_rate) call to match the updated function definition in payment_utils.py.

[0059] (1) Error detection and context extraction:

[0060] Error capture: When the project is executed, a parameter mismatch error is triggered. The error tracking mechanism of the IDE (such as PyCharm) locates the 4th line of order_processing.py (fee=compute_fee(amount,tax_rate)) and marks the error type as "parameter number mismatch".

[0061] Context resolution: The system automatically resolves the context information at the error location, including:

[0062] (a) The complete definition of the function to be repaired, calculate_total (parameter list, function body logic);

[0063] (b) The incorrect call to the statement compute_fee(amount,tax_rate) and the code before and after it on the same line (such as the calculation logic of total);

[0064] (c) Import relationship (from payment_utils import compute_fee) to determine that the called function comes from payment_utils.py.

[0065] Search Request Generation: Generate search requests based on context information: "Find the latest definition of the compute_fee function in payment_utils.py and obtain its parameter list (especially the meaning and default value of the newly added discount_rate parameter)."

[0066] (2) Cross-file code block retrieval and filtering:

[0067] Search trigger: Convert the search request into a semantic vector (encode the "compute_fee function definition and parameter list" using the UniXcoder model) and perform the search in the vector database of the project code repository.

[0068] Related code block matching: The vector database returns 5 code blocks that are semantically similar to the retrieval requirements, from which:

[0069] (a) Code block 1: The latest definition of the compute_fee function in payment_utils.py (containing three parameters: amount, tax_rate, and discount_rate; the comments indicate that discount_rate is the "discount rate, default 0.05");

[0070] (b) Code block 2: Historical versions of payment_utils.py (containing only the amount and tax_rate parameters, which are now obsolete);

[0071] (c) Code block 3: Test cases in test_payment.py (when calling compute_fee, discount_rate = 0.03 is passed in);

[0072] (d) Code block 4: Project documentation (explaining that the value range of discount_rate is 0-0.2);

[0073] (e) Code block 5: A correct example of calling compute_fee in other modules (e.g., compute_fee(1000,0.08,0.05));

[0074] (3) Code block compression and semantic token generation:

[0075] SCE initialization: Load the pre-trained lightweight semantic compression encoder (SCE), which consists of a CodeBERT-base (encoder) and a 3-layer linear network (projector), and has been adapted to the target LLM (Deepseek-Coder-6.7B) through two-stage training.

[0076] Compression processing: Input the three filtered code blocks (code blocks 1, 3, and 5) into SCE sequentially:

[0077] Encoder processing: CodeBERT-base extracts deep semantic features for each code block. For example, for code block 1 (the latest definition of compute_fee), the extracted features include "function parameters are amount(float), tax_rate(float), discount_rate(float, default=0.05)" and "function is to calculate the handling fee = amount × tax rate × (1 - discount rate)".

[0078] Compression and Mapping: Each code block is compressed from the original 768 tokens to a feature vector of 96 tokens through average pooling, and then mapped to the semantic space (4096 dimensions) of Deepseek-Coder-6.7B by a projector, ensuring that LLM can directly parse the compressed information.

[0079] Effect Analysis

[0080] Efficiency improvement: The original 3 code blocks (2304 tokens in total) are compressed to only 288 tokens (96×3) after SCE compression, with a compression ratio of 8:1. The LLM inference time is reduced from 1.2 seconds (original RAG method) to 0.35 seconds, which is close to the efficiency of single file processing (0.3 seconds), and the time consumption is reduced by 70.8%.

[0081] Accuracy Guarantee: The compressed token fully preserves the parameter information (name, type, default value) and business logic (dynamic setting rules for discount rate) of the compute_fee function. The repair result passes all unit tests on the first try with 100% accuracy, which is 35% better than the traditional RAG method (which has a repair accuracy of only 65% ​​due to missing parameter information caused by long context).

[0082] Generalization verification: If the Python code in the case is replaced with Java (e.g., the parameters do not match when OrderProcessing.java calls PaymentUtils.computeFee), the same efficient fix can still be achieved by adjusting the projector parameters of SCE to adapt to the Java-specific LLM (e.g., CodeLlama-Java-7B), proving the adaptability of the present invention to multiple programming languages.

[0083] Technical features

[0084] Cross-file dependency resolution: By preserving parameter change information of functions across files through lightweight compression, it solves the problem of difficult dependency resolution caused by excessively long context in traditional methods;

[0085] Dynamic optimization mechanism: Combining multiple rounds of generation and verification, the repair results not only meet the requirements of grammatical correctness, but also adapt to business logic (such as dynamic setting of discount rate);

[0086] Lightweight deployment: SCE parameters are only 1.5% of LLM, can be integrated into local IDE plugins, do not rely on cloud computing power, and are adapted to the security requirements of enterprise private code repositories.

[0087] As can be seen from the above examples, this invention, through a collaborative process of "retrieval-compression-generation," significantly reduces inference costs while accurately preserving key information on cross-file dependencies, thereby significantly improving the efficiency and accuracy of code repair. It is particularly suitable for cross-module collaboration scenarios in large enterprise-level projects.

[0088] References:

[0089] [1]Gao Y, Xiong Y, Gao X, et al.Retrieval-augmented generation for large language models:A survey[J].arXiv preprint arXiv:2312.10997,2023,2(1).

[0090] [2]Feng Z,Guo D,Tang D,et al.Codebert:A pre-trained model for programming and natural languages[J].arXiv preprint arXiv:2002.08155,2020.

[0091] [3]Roziere B,Gehring J,Gloeckle F,et al.Code llama:Open foundation models for code[J].arXiv preprint arXiv:2308.12950,2023.

[0092] [4]Guo D, Zhu Q, Yang D, et al. DeepSeek-Coder: When the Large LanguageModel Meets Programming--The Rise of Code Intelligence[J]. arXiv preprintarXiv:2401.14196,2024.

[0093] [5]Ge T,Hu J,Wang L,et al.In-context autoencoder for contextcompression[J].arXiv preprint arXiv:2307.06945,2023.

[0094] [6]Hu EJ,Shen Y,Wallis P,et al.Lora:Low-rank adaptation of largelanguage models[J].ICLR,2022,1(2):3.

Claims

1. A method for rapid code repair using lightweight compression context generation, characterized in that, Efficient repair is achieved by constructing a collaborative "retrieval-compression-generation" architecture. This includes: building a cross-file code retrieval database; designing a lightweight semantic compression encoder (SCE) to compress retrieved cross-file code blocks into short tokens with high semantic density; and employing a two-stage training strategy—semantic alignment and task alignment—to achieve semantic compatibility with Large Language Models (LLMs). In the semantic alignment stage, the SCE is trained so that the compressed tokens can be accurately parsed by the LLM; in the task alignment stage, the LLM is fine-tuned to utilize the compressed tokens for repair. The specific steps are as follows: (I) Construction of Cross-File Code Retrieval Database To accurately obtain the associated context of the code to be repaired, relevant code blocks are retrieved from large code repositories, and a retrieval system supporting efficient matching is built. The specific process is as follows: (1) Structured code block segmentation: The original code repository is automatically segmented to ensure the integrity of the code syntax and logic. For multi-line structures such as function definitions and class declarations, they are automatically expanded to complete syntactic units. For linearly executed code segments, the original code block is divided into basic code blocks every N lines, where N can be adjusted according to actual needs. After segmentation, a syntax parser filters out low-quality code blocks: code blocks with more than 50% blank lines or containing only comments or log output are removed, while valid code blocks containing key logic such as variable declarations, function calls, parameter passing, and class inheritance relationships are retained, laying a high-quality data foundation for subsequent retrieval and compression. (2) Semantic vector encoding and index optimization: A pre-trained model is used to semantically encode the filtered code blocks. After converting the code blocks into token sequences, they are input into the pre-trained model to obtain the vector of the last hidden state and average it as the semantic representation of the code block. This encoding method can capture the deep logic of the code. The vectors of all code blocks and their original contents are stored in a vector database, and an IVF index is built to accelerate retrieval. The vector space is divided into K clusters by a clustering algorithm, and each cluster maintains a local index, which reduces the retrieval time complexity from O(n) to O(log n), ensuring millisecond-level response in a library of millions of code blocks and meeting the requirements for real-time repair. (II) Design of Lightweight Semantic Compressor (SCE) To address the inefficiency of inference caused by long context inputs, a lightweight semantic compression encoder (SCE) is designed. This encoder compresses retrieved cross-file code blocks into short tokens with high semantic density while fully preserving all information from the retrieved documents. This avoids the information loss caused by explicit compression. The SCE consists of two parts: an encoder and a projector, as detailed below: Encoder: Used for code semantic feature extraction and compression The encoder employs a lightweight code pre-trained model. This model, pre-trained on a large-scale code corpus, can accurately identify the grammatical rules and logical dependencies of the code. The retrieved original code blocks are used as input to the encoder, and deep semantic features are extracted through the pre-trained model. The final output is a feature matrix of shape [seq_len, hidden_state]. To achieve compression, attention-weighted average pooling is applied to this feature matrix: based on the importance of each token in the code, the variable-length sequence is fused into a fixed-length feature vector. Projector: Used for semantic space alignment and adaptation The projector is a 3-layer linear neural network. Its function is to map the feature vector output by the encoder to the semantic space of the generator, solving the problem of "incompatibility between compressed tokens and LLM semantics". The ReLU activation function is used to enhance the non-linear expressive power, and the final output dimension is completely matched with the hidden layer dimension of the target LLM. Through the mapping operation, the semantic distribution of the compressed token is highly aligned with the semantic space formed by the LLM during the pre-training process, enabling the LLM to directly "understand" the deep logic in the compressed information and avoid repair errors caused by "semantic gaps". (III) A two-stage training strategy is adopted: semantic alignment and task alignment; To enable SCE and LLM to collaborate on code repair tasks, a two-stage training strategy is designed to optimize compression quality and repair capability respectively, ensuring that compression tokens can be understood by LLM and effectively assist the repair process: Phase 1: Semantic Alignment Training – Enabling LLM to "Understand" Compressed Tokens; The goal is to train the SCE to generate compressed tokens that can be accurately parsed by the LLM, ensuring that the compressed information can be restored by the LLM to the key logic of the original code block; the specific process is as follows: (1) Training data construction: randomly select code blocks from the retrieval library, generate compressed tokens through the current SCE, and then use the user's instructions: restore the original code blocks based on the following compressed semantic vectors, and then merge the compressed semantic vectors with the original code blocks to form the input part of our model; (2) Training process: Input the above prompt text into LLM and ask LLM to generate content consistent with the original code block based on the compression token; freeze the parameters of LLM during training and optimize only the encoder and projector parameters of SCE; minimize the difference between the content generated by LLM and the original code block by continuously adjusting the SCE parameters. (3) Convergence criterion: When the LLM achieves a restoration accuracy of over 90% for the original code block, the semantic alignment training is considered complete. This stage ensures that the compressed semantic vector can retain most of the information of the code block, and the large model can restore the code based on this, laying the foundation for subsequent repair tasks. Phase Two: Task Alignment Training – Enabling LLM to "Make Good Use" of Compression Tokens The goal is to fine-tune the LLM so that it can combine the compression token with the context of the code to be repaired to generate the correct repair results; the specific process is as follows: (1) Training data construction: Collect code repair samples containing cross-file dependencies to construct the input sequence, and concatenate the repair code, reference sequence and repair result together; where "related reference" contains cross-file code block compression tokens related to the error type; (2) Training process: Freeze the parameters of SCE and fine-tune LLM using low-rank adaptation technique; Specifically, introduce additional parameters on the query and key matrices in the attention module and introduce low-rank matrix classification so that the training parameters do not exceed 1% of the model; Through lightweight fine-tuning, LLM learns to use cross-file dependency information in compressed tokens for repair, while avoiding overfitting that leads to a decrease in generalization ability. (3) Convergence criterion: Training is stopped when the LLM’s repair accuracy on the validation set no longer improves for three consecutive rounds; this stage ensures that the LLM can efficiently utilize the information in the compression token to improve the accuracy of cross-file code repair.

2. The method for rapid code repair using lightweight compression context generation according to claim 1, characterized in that, Furthermore, end-to-end code repair is performed quickly, and the specific process is as follows: (1) Code parsing and retrieval triggering to be fixed The code to be repaired is parsed: the parser locates the error location, identifies the error type, and extracts N lines of context information before and after the error location to ensure the semantic coherence of the context; the same embedding model as when encoding the code library is used to encode the current context information to ensure the consistency of the semantic space of the retrieval. (2) Retrieval and filtering of related code blocks Calculate the cosine similarity between the query vector and all code block vectors in the retrieval database, and select the Top-K code blocks with the highest similarity. (3) Code block compression and token generation The retrieved k code blocks are input into the trained SCE. After compression by the encoder and mapping by the projector, k manually set adaptive code tokens are generated. The tokens are sorted according to the original retrieval similarity, with the tokens with higher similarity appearing first, to avoid LLM ignoring key information due to attentional distraction. (4) Fix code generation and optimization The input sequence is concatenated according to the template: the user's repair instructions, the code to be repaired, the semantic vector of the retrieved code block, and the fine-tuned LLM; the generation parameters are set as follows: max_new_tokens = length of the code to be repaired + 10, temperature = 0.7, top_p = 0.95; after generation, the syntax correctness of the repair result is verified by a syntax checker. If there are syntax errors, iterative generation is performed again using a large model, or the parameters of the retrieval machine are adjusted to achieve a better value.

3. The method for rapid code repair using lightweight compression context generation according to claim 1, characterized in that, The semantic vector encoding uses a pre-trained model called UniXcoder.

4. The method for rapid code repair using lightweight compression context generation according to claim 1, characterized in that, The lightweight code pre-trained model is CodeBERT-base.

Images