A large language model prompt compression method for code generation

Abstract

The application provides a large language model prompt compression method for code generation, in order to solve the problem of performance decline of a large language model caused by too long input in a code generation task, the input of the large language model is compressed from three aspects, including program dependency environment preprocessing, code segment compression and natural language instruction compression. The application reduces the influence of redundant information on the large language model, provides accurate reference information for the large language model, helps the large language model to quickly and accurately understand instruction information, and reduces the cost consumption of the large language model.

Description

Technical Field

[0001] This invention relates to, and more particularly to, a method for compressing large language model prompts for code generation. Background Technology

[0002] In recent years, with the rapid development of deep learning, large-scale pre-trained language models (LLMs) have demonstrated powerful capabilities in tasks such as code generation and natural language processing. However, the practical application of large language models faces two key challenges: limited contextual input windows and high costs. In daily use, if the input length exceeds the input window length of the large language model, it will become unresponsive or its capabilities will significantly decrease. When the input length is within the tolerance range of the large language model, it can provide more accurate and reasonable answers due to the richness of contextual information. However, rich contextual information usually means more input content, which greatly increases the cost.

[0003] Specifically, existing technologies for handling code-related tasks typically require the complete program dependency environment or code snippets as input, which may cause the input length to exceed the model's context window limit. Furthermore, redundant information in the code (such as non-critical functions, repetitive code structures, etc.) not only increases the input length but also reduces the model's efficiency. In terms of natural language instruction compression, traditional text compression methods often rely on simple rules or statistical features, failing to effectively preserve semantically critical information, leading to a decline in the quality of the model's generated results.

[0004] Sivand, Reference: Rabin MRI, Hellendoorn VJ, Alipour M A. Understanding neural code intelligence through program simplification[C] / / Proceedings of the 29th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering. 2021:441-452. This technique uses incremental debugging to gradually simplify the input program, aiming to find the minimum input program that can maintain the model's prediction results. Specifically, the method first divides the original input program into multiple parts, generating a series of candidate programs. Then, by iteratively removing parts, it checks whether these candidate programs can still maintain the same model prediction results as the original input program. This process continues until the shortest input program, the so-called "1-minimal" program, is found, which satisfies the following condition: no part can be removed without losing the expected properties of the original input program (such as the prediction results). The core advantage of this method is that it can significantly reduce the complexity of the input program without affecting the model's prediction results, thereby providing a simpler and more understandable input example for the model's behavioral analysis.

[0005] However, this method has high time complexity, resulting in significant computational overhead and low efficiency for large-scale program inputs. It relies on incremental debugging to generate a large number of candidate programs, many of which are invalid (i.e., syntactically incorrect), requiring additional filtering and further increasing computational burden. The final simplified program obtained is not unique; different simplification paths may lead to different results. Finally, this technique only considers program simplification, neglecting external factors. In real-world programming scenarios, code generation tasks must consider not only the current program content but also the program's dependencies and the user's next instructions to the large language model. This method, focusing only on program simplification, has very limited applicability.

[0006] This invention addresses three challenging problems: (1) how to preprocess the program-dependent environment, reduce the impact of redundant information on large language models, and provide accurate reference information for large language models; (2) how to compress code fragments without breaking code syntax; and (3) how to compress natural language instructions given to the model without relying on statistical rules, so as to help large language models quickly and accurately understand instruction information and reduce the cost of large language models. Summary of the Invention

[0007] To address the aforementioned problems, this invention discloses a large language model-based code generation-oriented suggestion compression method, which mainly includes a program dependency environment preprocessing module, a code snippet compression module, and a natural language instruction compression module. The specific contents of each module are as follows:

[0008] (1) Program dependency environment preprocessing module:

[0009] The program dependency environment preprocessing module aims to avoid inputting the complete program dependency environment into the large language model. It retrieves and filters the functions that the model is most likely to need, providing the large language model with accurate reference information. Specifically, this module includes the following six steps:

[0010] Step 1: The program dependency environment preprocessing module first obtains the import statements of the code and saves each import statement separately into a list denoted as I = [i1, i2, ..., i...]. t ], where i t This indicates the t-th import statement.

[0011] Step 2: After the program's dependency environment preprocessing module obtains, segments, and saves the import statements, it analyzes the form of each import statement, which is mainly divided into two forms: from...import... and import.... Depending on the form of the import statement, corresponding processing methods are adopted.

[0012] Step 3: After obtaining the analysis results from Step 2, the program dependency environment preprocessing module constructs the path of the imported dependency module based on the content of the import statement and locates the __init__.py file in the module directory.

[0013] Step 4: After obtaining the results of Step 3, the program dependency environment preprocessing module uses static code analysis technology to read the __init__.py file in the module directory, and uses AST (Abstract Syntax Tree) technology to parse the contents of the __init__.py file and construct an abstract syntax tree of the file content.

[0014] Step 5: After obtaining the results from Step 4, the program dependency environment preprocessing module analyzes the obtained abstract syntax tree, extracts and saves the source code of the independent functions defined in the __init__.py file, and analyzes the __all__ list in the file to find functions defined outside the file. Combining the import statements in the __init__.py file, it iteratively performs steps one through five until all function code at the same level as the import statements in Step 1 is obtained, providing data support for the following steps.

[0015] Step Six: After obtaining the results from Step Five, the program dependency environment preprocessing module will further compress the program dependency environment, using low-cost, large-scale pre-trained language models such as DeepSeek-R1-Distill-Qwen-32B to summarize function signatures and function functions, retaining valid information.

[0016] After the above steps, we obtain the signatures and functional summaries of the functions that the model most likely needs, avoiding the need to directly input a large amount of source code that the program depends on into the model.

[0017] (2) Code snippet compression module:

[0018] The code snippet compression module aims to streamline Python code through static code analysis, removing redundant identifiers, symbols, and structures while retaining the core logic and functionality. This reduces code complexity, improves readability, and decreases the length of code input to large language models, thereby increasing the model's processing efficiency. Specifically, this module includes the following five steps:

[0019] Step 1: The code snippet compression module first receives the input Python code and uses the `ast` module from the Python standard library to parse the code into an abstract syntax tree (AST). An AST is a tree-like representation of the code's syntactic structure, clearly demonstrating its logical structure. During parsing, the module checks the code's syntax to ensure its validity. After parsing, the module saves the AST as intermediate data, providing a foundation for subsequent feature extraction and code compression.

[0020] Step Two: After obtaining the abstract syntax tree, the code snippet compression module traverses the syntax tree using a custom TokenVisitor class to extract key features from the code. The feature extraction process consists of two levels:

[0021] Lexical features: Extracting identifiers such as variable names and function names, as well as symbols such as operators and delimiters from the code. These features are the basic components of the code and reflect its surface structure.

[0022] Syntactic features: Extract control structures such as if, while, and for, function definitions, and function calls from the code. These features reflect the logical structure of the code and are the core of its functionality.

[0023] The extracted features are saved into the tokens_map dictionary by category to provide data support for subsequent code compression.

[0024] Step 3: The code snippet compression module formulates a code compression strategy based on the specified compression ratio and the set of identifiers that need to be retained. The compression strategy removes features from the code in the following priority order:

[0025] 1. Symbols: Prioritize removing operators, delimiters, and other symbols, as these have little impact on the logic of the code.

[0026] 2. Identifiers: Next, remove identifiers such as variable names and function names. These identifiers have a certain impact on the semantics of the code, but their token count can be reduced by replacing or renaming them.

[0027] 3. Control Structures: Next, remove control structures such as if, while, and for. These control structures have a significant impact on the logic of the code, but they can simplify the code in certain scenarios.

[0028] 4. Function calls: Remove function calls, which reflect the functional dependencies of the code, but can be omitted in some scenarios.

[0029] 5. Function Definitions: Remove function definitions. Function definition code is the core part of the code, but in some scenarios it can be simplified by summarizing or replacing it. The module will calculate the number of features to be removed based on the compression ratio and generate a specific compression plan.

[0030] Step 4: The code snippet compression module traverses the abstract syntax tree using a custom TokenRemover to remove or replace specified features. Specific operations include:

[0031] 1. Identifier replacement: Replace variable names, function names, and other identifiers with placeholders (_) to reduce the number of tokens in the code without causing syntax errors.

[0032] 2. Function renaming: Rename the function name in the function definition to a shorter token to reduce the token length without breaking the code's syntax structure.

[0033] 3. Retain key features: Retain specified identifiers to ensure that identifiers that the user expects the model to see are not deleted, and to ensure that the model can obtain the content that the user expects the model to understand.

[0034] During the traversal, the module dynamically modifies the abstract syntax tree, ultimately resulting in a simplified syntax tree.

[0035] Step 5: After feature removal and replacement, the code compression module uses the Python third-party dependency library astor to transform the simplified abstract syntax tree back into Python code. The astor library can convert the abstract syntax tree into a readable code string.

[0036] (3) Natural Language Command Compression Module

[0037] The natural language instruction compression module aims to compress input instructions using natural language preprocessing techniques and pre-trained language models. This process preserves the core information of the instructions while removing redundant content, helping the model to quickly and accurately understand the instructions and improving processing efficiency. Specifically, this module includes the following five steps:

[0038] Step 1: The natural language instruction compression module first receives the input instruction and preprocesses the instruction text using regular expressions and word segmentation tools. Preprocessing first removes redundant characters such as extra spaces and newlines to ensure text cleanliness. Next, the text is segmented into independent sentences based on punctuation at the end of each sentence for easier subsequent processing. Finally, natural language processing tools, such as Spacy and NLTK, are used to segment the sentences and extract semantic units.

[0039] Step 2: The natural language instruction compression module loads a small pre-trained language model and its corresponding word segmenter. The pre-trained language model is loaded locally and then onto the GPU device.

[0040] Step 3: After loading the model, the natural language instruction compression module first encodes the instruction text into a format acceptable to the model. Then, it calculates the probability distribution of each semantic unit through the model's forward propagation and uses self-information to calculate the formula I(x) = -log2P(x). t |x0,x1,…,x t-1 The self-information of each semantic unit is calculated and saved to provide data support for the subsequent formulation of compression strategies.

[0041] Step 4: The natural language instruction compression module formulates a compression strategy based on the specified compression ratio and compression level. First, it calculates a compression threshold based on the compression ratio and self-information distribution, which serves as a reference for subsequent token deletion. Then, it filters semantic units whose self-information is below the threshold based on the size of their self-information and marks them as content to be removed.

[0042] Step 5: The natural language instruction compression module deletes the content marked for removal in Step 4 according to the defined compression strategy. Then, the module concatenates the resulting semantic unit set into a complete instruction text, obtaining the simplified instruction.

[0043] The beneficial effects of this invention are:

[0044] 1. This invention addresses the performance degradation caused by excessively long inputs to large language models in code generation tasks by compressing the input of large language models at three levels: program dependency environment preprocessing, code fragment compression, and natural language instruction compression. The invention aims to remove non-critical information from the input content to achieve compression.

[0045] 2. Program dependency environment preprocessing can compress program dependency environment information, accurately locate the files where functions that the model may use are located, and summarize function functions in combination with low-cost large language models, thereby optimizing the problem of excessively long program dependency environment input.

[0046] 3. Code snippet compression can compress code content without breaking the code snippet syntax. It extracts tokens from the code by category using AST parsing tools, and reduces the length of the code snippet according to the importance of the tokens, thereby compressing the code snippet length.

[0047] 4. Natural language instruction compression can reduce the input instructions to a large language model. By calculating the self-information of semantic units in natural language instructions and deleting semantic units in ascending order of self-information, the length of natural language instructions can be compressed.

[0048] 5. This invention does not consider training the model or changing the model's parameters; it only considers compressing the model's input and aims to propose a method for compressing input content to help the model generate inferences. Attached Figure Description

[0049] Figure 1 A schematic diagram of a large language model-based compression method module for code generation;

[0050] Figure 2 Flowchart of the program dependency environment preprocessing module;

[0051] Figure 3 Flowchart of the code snippet compression module;

[0052] Figure 4 Flowchart of the Natural Language Command Compression Module. Detailed Implementation

[0053] The present invention will be further illustrated below with reference to the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. It should be noted that the terms "front," "rear," "left," "right," "up," and "down" used in the following description refer to directions in the accompanying drawings, and the terms "inner" and "outer" refer to directions toward or away from the geometric center of a specific component, respectively.

[0054] like Figure 1As shown in the figure, a specific implementation example of a large language model suggestion compression method for code generation in this embodiment is as follows:

[0055] like Figure 2 As shown, S1: Program dependency environment preprocessing, retrieving useful functions and summarizing function functionalities, employing static code analysis technology. This technology can automatically find module paths, parse module names, analyze import statements, and recursively parse module source code to achieve accurate retrieval and summarization of function functionalities in the program dependency environment, specifically including:

[0056] S101: The program dependency environment preprocessing module first obtains the import statements of the code and saves each import statement separately into a list denoted as I = [i1, i2, ..., i t ], where i t This indicates the t-th import statement. Taking importtorch.nn and from acclerate import Accelrator as examples, I = [import torch.nn, from acclerate import Accelrator].

[0057] S102: After obtaining the results of S101, the program dependency environment preprocessing module analyzes the form of each import statement, mainly divided into two forms: from...import... and import.... Depending on the form of the import statement, corresponding processing methods are adopted. `from acclerate import Accelerator` will be split into `['from','accelerate','import','Accelerator']`, and `import torch.nn` will be split into `['torch','nn']`.

[0058] S103: The program dependency environment preprocessing module will find the path of the imported dependency module based on the result of S102, and locate the __init__.py file in the module directory. In the two examples, the partial paths obtained are: ... / Lib / site-packages / accelerate / __init__.py and ... / Lib / site-packages / torch / nn / __init__.py.

[0059] S104: After obtaining the steps in S103, the program dependency environment preprocessing module will use static code analysis technology to read the __init__.py file in the module directory, use AST (Abstract Syntax Tree) technology to parse the contents of the __init__.py file, and construct the abstract syntax tree of the file content.

[0060] S105: The program dependency environment preprocessing module analyzes the abstract syntax tree obtained in S104. First, it extracts and stores the source code of independent functions defined in the __init__.py file. Then, it analyzes the __all__ list in the file, finds functions defined outside the file, and combines them with the import statements in the __init__.py file to iteratively process S101 to S105 until it obtains the function code at the same level as the import statements in S101, providing data support for the following steps.

[0061] S106: Finally, the program dependency environment preprocessing module further compresses and summarizes the data obtained in S105, using low-cost, large-scale pre-trained language models such as DeepSeek-R1-Distill-Qwen-32B to summarize function signatures and function functionalities, retaining valid information. The summarized form is as follows: def debug_launcher(function:Callable,args:Tuple=(),num_processes:int=2)->None:"""Launches a training function on multiple CPU processes for debugging purposes.""

[0062] After steps S101 to S106, a summary set of functions most likely to be needed by the model is finally obtained, avoiding the direct input of a large amount of source code from the program's dependency environment to the model.

[0063] like Figure 3 As shown, S2: the code snippet compression method, employs an Abstract Syntax Tree (AST)-based code compression approach. It constructs an AST by parsing Python code and classifies and marks tokens such as symbols, identifiers, structures, function signatures, and calls at both lexical and syntactic levels. Then, it gradually removes non-critical tokens from the code according to a preset removal order and compression ratio, while retaining specified critical tokens, ultimately generating a streamlined code. Specifically, this includes:

[0064] S201: The code snippet compression module first receives the input Python code and uses the `ast` module from the Python standard library to parse the code into an abstract syntax tree (AST). An AST is a tree-like representation of the code's syntactic structure, clearly demonstrating its logical structure. During parsing, the module checks the code's syntax to ensure its validity. After parsing, the module saves the AST as intermediate data, providing a foundation for subsequent feature extraction and code compression.

[0065] S202: After obtaining the results from S201, the syntax tree is traversed using a custom TokenVisitor to extract key features from the code. The feature extraction process consists of two levels:

[0066] Lexical features: Extracting identifiers such as variable names and function names, as well as symbols such as operators and delimiters from the code. These features are the basic components of the code and reflect its surface structure.

[0067] Syntactic features: Extract control structures such as if, while, and for, function definitions, and function calls from the code. These features reflect the logical structure of the code and are the core of its functionality.

[0068] The extracted features are saved into the tokens_map dictionary by category to provide data support for subsequent code compression.

[0069] S203: The code snippet compression module formulates a code compression strategy based on the specified compression ratio and a subset of identifiers to be retained. The compression strategy removes features from the code in the following priority order:

[0070] 1. Symbols: Prioritize removing operators, delimiters, and other symbols, as these have little impact on the logic of the code.

[0071] 2. Identifiers: Next, remove identifiers such as variable names and function names. These identifiers have a certain impact on the semantics of the code, but their token count can be reduced by replacing or renaming them.

[0072] 3. Control Structures: Next, remove control structures such as if, while, and for. These control structures have a significant impact on the logic of the code, but they can simplify the code in certain scenarios.

[0073] 4. Function calls: Remove function calls, which reflect the functional dependencies of the code, but can be omitted in some scenarios.

[0074] 5. Function Definitions: Remove function definitions. Function definition code is the core part of the code, but in some scenarios it can be simplified by summarizing or replacing it. The module will calculate the number of features to be removed based on the compression ratio and generate a specific compression plan.

[0075] S204: The code snippet compression module traverses the abstract syntax tree using a custom TokenRemover to remove or replace specified features. Specific operations include:

[0076] 1. Identifier replacement: Replace variable names, function names, and other identifiers with placeholders (_) to reduce the number of tokens in the code without causing syntax errors.

[0077] 2. Function renaming: Rename the function name in the function definition to a shorter token to reduce the token length without breaking the code's syntax structure.

[0078] 3. Retain key features: Retain specified identifiers to ensure that identifiers that the user expects the model to see are not deleted, and to ensure that the model can obtain the content that the user expects the model to understand.

[0079] During the traversal, the module dynamically modifies the abstract syntax tree, ultimately resulting in a simplified syntax tree.

[0080] S205: After completing feature removal and replacement, the code compression module uses the Python third-party dependency library astor to transform the simplified abstract syntax tree back into Python code. The astor library can convert the abstract syntax tree into a readable code string.

[0081] like Figure 4 As shown, S3: The Natural Language Instruction Compression Module aims to compress input instructions using natural language preprocessing techniques and pre-trained language models, preserving the core information of the instructions while removing redundant content. This helps the model quickly and accurately understand instruction information and improves model processing efficiency. Specifically, it includes:

[0082] S301: The Natural Language Instruction Compression Module first receives the input instruction and preprocesses the instruction text using regular expressions and word segmentation tools. Preprocessing first removes redundant characters such as spaces and newlines to ensure text cleanliness. Next, the text is segmented into independent sentences based on punctuation at the end of each sentence for easier subsequent processing. Finally, natural language processing tools, such as Spacy and NLTK, are used to segment the sentences and extract semantic units. For example, the text reads: "In natural language processing tasks, when the original instructions provided by the user contain repetitive content, redundant expressions, or irrelevant information, it is necessary to utilize natural language preprocessing techniques and pre-trained language models to calculate the self-information values ​​of semantic units, formulate reasonable compression strategies, retain core instruction elements, delete invalid information, and ultimately generate concise and accurate compressed instructions." The regular expression `re.split(r'[。!?]',...)` was used to split the text by punctuation marks, resulting in six independent sentences: "In natural language processing tasks," "When the original instructions provided by the user contain repetitive content," "When there is redundant expression or irrelevant information," "Natural language preprocessing techniques and pre-trained language models are needed," "By calculating the self-information value of semantic units," and "Developing a reasonable compression strategy to retain core instruction elements, delete invalid information, and ultimately generate concise and accurate compressed instructions."

[0083] S302: The Natural Language Instruction Compression Module loads a small pre-trained language model and its corresponding word segmenter. The pre-trained language model is loaded locally and then onto the GPU device.

[0084] S303: After loading the model, the natural language instruction compression module first encodes the instruction text into a format acceptable to the model, such as [{"input_ids":[...],"attention_mask":[...]}]. Then, it calculates the probability distribution of each semantic unit through the model's forward propagation and uses the self-information calculation formula:

[0085] I(x) = -log2P(x) t |x0,x1,…,x t-1 )

[0086] Calculate and save the self-information of each semantic unit to provide data support for subsequent compression strategy formulation, where I(x) represents the current token self-information, and P(x) represents the self-information of P(x). t |x0,x1,…,x t-1 () indicates that the model outputs x given the known information. t The probability of a sentence or phrase. The self-information of a sentence or phrase is represented as the sum of the self-information of all tokens in the sentence or phrase. The self-information values ​​of some semantic units are shown below:

[0087] "need" 0.82 "redundancy" 0.91 "Formulate" 0.78

[0088] S304: The natural language instruction compression module formulates a compression strategy based on the specified compression ratio and compression level. First, it calculates a compression threshold based on the compression ratio and self-information distribution, serving as a reference for subsequent token deletion. Semantic units with self-information below the threshold are filtered based on their self-information size and marked as content to be removed. The compression ratio is set to 0.5, and the compression strategy is formulated, with some deletions categorized into units: "when provided by the user," "and pre-trained," and "finally generated."

[0089] S305: After obtaining the compression strategy defined in S304, the natural language instruction compression module will delete the marked content to be removed. It will retain the following information: [In natural language processing tasks, when instructions are redundant, it is necessary to utilize preprocessing techniques and models to calculate self-information values, formulate reasonable strategies, retain core elements, delete invalid information, and generate concise and accurate instructions]. The sequence will be assembled into complete content: In natural language processing tasks, when instructions are redundant, it is necessary to utilize preprocessing techniques and models to calculate self-information values, formulate reasonable strategies, retain core elements, delete invalid information, and generate concise and accurate instructions.

[0090] This invention proposes a large-scale model-based suggestion compression method for code generation, which can help users in the field reduce the cost of using large-scale pre-trained language models. The advantage of this invention lies in compressing the input content in code generation tasks from three perspectives: project dependency environment, code snippets, and natural language instructions. Corresponding compression methods are designed for the features of each perspective, ensuring the integrity of key information while removing redundant information to the maximum extent, thus achieving concise and optimized suggestion content.

[0091] The technical means disclosed in this invention are not limited to those disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features.

Claims

1. A method for compressing large language model prompts for code generation, characterized in that: Includes the following steps: Step 1: Design a program dependency environment preprocessing module. This module uses static code analysis to automatically parse module names, locate module paths, analyze import statements, and recursively parse module source code. It then retrieves functions from the dependent modules and summarizes their functionalities. Specifically, Step 1 includes: Step 11: Obtain the import statements of the code and save each import statement separately into a list; Step 12: The program dependency environment preprocessing module analyzes the form of each import statement, which includes two forms: from ... import ... and import ...; it then parses the names of the dependent modules based on the form of the import statement. Step 13: After obtaining the analysis results from Step 12, the program dependency environment preprocessing module constructs the path of the imported dependency module based on the content of the import statement and locates the __init__.py file under the dependency module path; Step 14: After obtaining the results of Step 13, the program dependency environment preprocessing module uses AST (Abstract Syntax Tree) technology to parse the contents of the __init__.py file and construct the abstract syntax tree of the __init__.py file; Step 15: After obtaining the results of Step 14, the program dependency environment preprocessing module extracts independent functions from the __init__.py file, analyzes the external functions in the __all__ list, and combines the import statements in the __init__.py file to iteratively perform steps 11 to 15 until all functions at the same level as the import statements in the code in Step 11 are obtained. Step 16: After obtaining the results of Step 15, the program dependency environment preprocessing module uses a pre-trained language model to summarize the function signature and function functionality; Step 2: Design a code snippet compression module, which adopts a code compression method based on abstract syntax trees. It classifies and marks tokens in the code based on two levels: lexical and syntactic, and removes non-critical tokens in the code according to a preset removal order and compression ratio. Step 3: Design a natural language instruction compression module to remove redundant content from instructions using natural language preprocessing techniques and pre-trained language models.

2. The method for compressing large language model prompts for code generation according to claim 1, characterized in that: Step 2 specifically includes: Step 21: The code snippet compression module uses AST technology to parse the input Python code and checks the syntax of the code during the parsing process; Step 22: After obtaining the abstract syntax tree, the code snippet compression module traverses the abstract syntax tree and extracts key features from the code; Step 23: The code snippet compression module formulates a code compression strategy based on the specified compression ratio and the set of identifiers that need to be retained; Step 24: The code snippet compression module traverses the abstract syntax tree, removing or replacing specified features; Step 25: After completing feature removal and replacement, the code compression module uses the astor tool to transform the simplified abstract syntax tree into Python code.

3. The method for compressing large language model prompts for code generation according to claim 2, characterized in that: The compression strategy in step 23 removes features from the code in the following priority order:

1. Symbols; 2. Identifiers; 3. Control structures; 4. Function calls; 5. Function definitions.

4. The method for compressing large language model prompts for code generation according to claim 2, characterized in that: Step 24 includes the following specific operations:

241. Identifier substitution; 242. Renaming functions; 243. Retain key features.

5. The method for compressing large language model prompts for code generation according to claim 1, characterized in that: Step 3 specifically includes: Step 31: The natural language instruction compression module preprocesses the instruction text, removes redundant characters, segments independent sentences, and extracts semantic units using the spacy and nltk tools; Step 32: The natural language instruction compression module loads the pre-trained language model and the corresponding word segmenter; Step 33: The natural language instruction compression module calculates the probability distribution of semantic units through the forward propagation of the pre-trained language model and calculates the self-information of the semantic units; Step 34: The natural language instruction compression module calculates the compression threshold based on the specified compression ratio and self-information distribution, and marks semantic units with self-information below the threshold as content that needs to be removed; Step 35: The natural language instruction compression module gradually removes the semantic units marked in step 34 according to the compression ratio; finally, the natural language instruction compression module concatenates the remaining semantic units into simplified instructions.

Images