Method and apparatus for adding context of large language model
By performing three-level classification and differentiation on the execution results of the Large Language Model (LLM), the problems of context window exhaustion, response latency, and semantic pollution are solved, achieving efficient context management and accurate understanding of LLM.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- UNIONTECH SOFTWARE TECH CO LTD
- Filing Date
- 2026-04-23
- Publication Date
- 2026-05-29
AI Technical Summary
The problems of context window exhaustion, increased response latency, semantic pollution and soaring costs in existing large language models (LLMs) are mainly due to the large size, complex structure, high noise and high reusability of the execution results, which leads to information loss, response latency and terminal sequence interference in understanding.
By employing a three-level classification strategy for the execution results, based on the data volume and the proportion of terminal sequences, the results are divided into complete retention type, compressed type, and discard type, and then differentiated processing is performed, including retention, compression, or discarding. Terminal sequences are removed, and compression is performed using a head-and-tail retention strategy and an error-priority strategy to extract summary information.
It effectively saves context space, reduces response latency, improves the semantic understanding accuracy of LLM, reduces costs, retains key information, and solves the problems of context window exhaustion and semantic pollution.
Smart Images
Figure CN122114186A_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of artificial intelligence, and in particular to a method and apparatus for adding context to a large language model. Background Technology
[0002] In the AI-native shell (AI-Shell), the Large Language Model (LLM) needs to decide on subsequent operations based on the execution results of traditional commands. Currently, a typical multi-step task chain may include the following commands: 1) Build tasks (such as `make build` or `npm run build`), which can generate thousands of lines of compilation logs; 2) Test tasks (such as `pytest` or `pm test`), which can output detailed test reports; 3) Log analysis (such as `docker logs container`), which can generate tens of megabytes of runtime logs; 4) Code inspection (such as `eslint` or `pylint`), which can output a large number of code quality issues. The outputs of these commands have the following characteristics: large volume, complex structure, high noise, and high reusability.
[0003] How to reasonably add the execution result (or output) of a command to the context of an LLM to better meet user needs is a technical problem that those skilled in the art have been working hard to study. Summary of the Invention
[0004] This disclosure provides a method and apparatus for adding context to a large language model, so as to at least solve the problem of context window exhaustion in related technologies of LLM.
[0005] According to a first aspect of the present disclosure, a method for adding context to a large language model is provided, comprising: obtaining the execution result of a target command, wherein the target command is a command invoked during the inference process of the large language model; classifying the execution result according to the data volume of the execution result and the proportion of terminal sequences in the execution result to obtain a classification result of the execution result, wherein the classification result is one of complete retention, compressed, and discarded, wherein the execution result corresponding to the complete retention type will be completely retained, the execution result corresponding to the compressed type will be compressed, and the execution result corresponding to the discarded type will be discarded; performing processing on the execution result corresponding to the classification result of the execution result; and adding the processing result of the execution result to the context of the large language model.
[0006] Optionally, the execution results are classified based on the amount of data in the execution results and the proportion of terminal sequences in the execution results, resulting in classification results, including: if the amount of data in the execution results is less than or equal to a first threshold and the proportion of terminal sequences in the execution results is less than a third threshold, the classification result is determined to be complete and retained; if the amount of data in the execution results is greater than the first threshold and less than the second threshold, or if the proportion of terminal sequences in the execution results is greater than or equal to the third threshold and the amount of data in the execution results is less than or equal to the first threshold, or if the amount of data in the execution results is greater than or equal to the second threshold and the target command is not a command in a preset list, the classification result is determined to be compressed; and if the amount of data in the execution results is greater than or equal to the second threshold and the target command is a command in a preset list, the classification result is determined to be discarded, wherein the second threshold is greater than the first threshold.
[0007] Optionally, the execution result is processed in accordance with the classification result of the execution result, including: in response to the classification result of the execution result being complete and retained, the complete execution result is used as the processing result of the execution result; in response to the classification result of the execution result being compressed, the execution result is compressed and the compressed result is used as the processing result of the execution result; in response to the classification result of the execution result being discarded, the execution result is discarded and the discard information is used as the processing result of the execution result, wherein the discard information indicates that the execution result has been discarded.
[0008] Optionally, if the format of the execution result is not within the preset format set, the execution result is compressed, including: retaining a first predetermined number of lines in the header section and a second predetermined number of lines in the tail section of the execution result; deleting the remaining lines in the execution result other than the first predetermined number of lines and the second predetermined number of lines; and inserting a truncation mark between the first predetermined number of lines and the second predetermined number of lines.
[0009] Optionally, in response to the existence of rows containing erroneous keywords among the remaining rows, the rows containing erroneous keywords are retained.
[0010] Optionally, if the format of the execution result is within a preset set of formats, the execution result is compressed, including: determining a summary format corresponding to the format of the execution result; extracting summary information from the execution result based on the summary format; and obtaining a compressed result by representing the summary information in a summary format.
[0011] Optionally, if the classification result of the execution result is compressed or fully preserved, before performing the processing corresponding to the classification result of the execution result, the method further includes: removing the terminal sequence from the execution result, wherein the terminal sequence includes at least terminal control characters or ANSI escape sequences.
[0012] Optionally, removing terminal sequences from the execution result includes: in response to the terminal sequence being a terminal control character, removing the terminal control character from the execution result by simulating the dynamic process of the terminal control character in the terminal; and in response to the terminal sequence being an ANSI escape sequence, removing the ANSI escape sequence from the execution result using regular expressions.
[0013] According to a second aspect of the present disclosure, an apparatus for adding context to a large language model is provided, comprising: an acquisition unit configured to acquire the execution result of a target command, wherein the target command is a command invoked during the inference process of the large language model; a classification unit configured to classify the execution result according to the data volume of the execution result and the proportion of terminal sequences in the execution result, to obtain a classification result of the execution result, wherein the classification result is one of complete retention, compressed, and discarded, wherein the execution result corresponding to the complete retention type will be completely retained, the execution result corresponding to the compressed type will be compressed, and the execution result corresponding to the discarded type will be discarded; a processing unit configured to perform processing on the execution result corresponding to the classification result of the execution result; and an adding unit configured to add the processing result of the execution result to the context of the large language model.
[0014] Optionally, the classification unit is further configured to determine the classification result of the execution result as complete retention type in response to the data volume of the execution result being less than or equal to a first threshold and the proportion of terminal sequences in the execution result being less than a third threshold; to determine the classification result of the execution result as compressed type in response to the data volume of the execution result being greater than the first threshold and less than a second threshold, or the proportion of terminal sequences in the execution result being greater than or equal to the third threshold and the data volume of the execution result being less than or equal to the first threshold, or the data volume of the execution result being greater than or equal to the second threshold and the target command not belonging to the command in the preset list; and to determine the classification result of the execution result as discarded type in response to the data volume of the execution result being greater than or equal to the second threshold and the target command belonging to the command in the preset list, wherein the second threshold is greater than the first threshold.
[0015] Optionally, the processing unit is further configured to, in response to the classification result of the execution result being of the complete retention type, use the complete execution result as the processing result of the execution result; in response to the classification result of the execution result being of the compressed type, compress the execution result and use the compressed result as the processing result of the execution result; in response to the classification result of the execution result being of the discard type, discard the execution result and use the discard information as the processing result of the execution result, wherein the discard information indicates that the execution result has been discarded.
[0016] Optionally, if the format of the execution result is not within the preset format set, the processing unit is further configured to retain a first predetermined number of lines in the header section and a second predetermined number of lines in the tail section of the execution result; delete the remaining lines in the execution result other than the first predetermined number of lines and the second predetermined number of lines; and insert a truncation mark between the first predetermined number of lines and the second predetermined number of lines.
[0017] Optionally, the processing unit is also configured to retain the row containing the erroneous keyword in response to the presence of a row containing the erroneous keyword in the remaining rows.
[0018] Optionally, if the format of the execution result is within a preset format set, the processing unit is further configured to determine the summary format corresponding to the format of the execution result; extract summary information from the execution result based on the summary format; and obtain the compression result by representing the summary information in a summary format.
[0019] Optionally, if the classification result of the execution result is compressed or fully preserved, the processing unit is further configured to remove the terminal sequence in the execution result before performing the processing corresponding to the classification result of the execution result, wherein the terminal sequence includes at least terminal control characters or ANSI escape sequences.
[0020] Optionally, the processing unit is further configured to remove terminal control characters from the execution result by simulating the dynamic process of terminal control characters in the terminal in response to the terminal sequence being terminal control characters; and to remove ANSI escape sequences from the execution result by using regular expressions in response to the terminal sequence being an ANSI escape sequence.
[0021] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement a method for adding context of a large language model according to the present disclosure.
[0022] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by at least one processor, causes at least one processor to perform the above-described method for adding context to a large language model according to the present disclosure.
[0023] According to a fifth aspect of the present disclosure, a computer program product is provided, including computer instructions that, when executed by a processor, implement a method for adding context according to a large language model of the present disclosure.
[0024] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects: According to the method and apparatus for adding context to a large language model (LLM) disclosed herein, the execution results are categorized into complete retention, compressed, or discarded types based on the amount of data in the execution results and the proportion of terminal sequences in the execution results. The execution results are then processed accordingly based on the final classification result before being added to the LLM context. This allows for differentiated processing based on different classification results, rather than adding all command outputs completely to the LLM context, thereby significantly saving context space for retaining more task history and system knowledge. Therefore, this disclosure at least solves the problems of context window exhaustion, increased response latency, semantic pollution, and soaring costs in related technologies of LLM.
[0025] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description
[0026] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.
[0027] Figure 1 This is a flowchart illustrating a method for adding context to an LLM according to an exemplary embodiment; Figure 2 This is an overall architecture diagram of a system for adding context for an LLM, illustrated according to an exemplary embodiment. Figure 3 This is a system flowchart illustrating a method for adding the context of an LLM according to an exemplary embodiment; Figure 4 This is a block diagram illustrating an apparatus for adding context to an LLM according to an exemplary embodiment; Figure 5 This is a block diagram of an electronic device 500 according to an embodiment of the present disclosure. Detailed Implementation
[0028] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.
[0029] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following examples do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.
[0030] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. Another example is "performing at least one of step one and step two", which means the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.
[0031] Currently, the execution results after executing commands have the following characteristics: 1) Large size, that is, the compilation logs generated by a single build task can reach 10-100MB, the test task output contains thousands of assertion results, and the container logs may accumulate hundreds of megabytes; 2) Complex structure, that is, the execution results are a mixture of human-readable text, machine logs, JSON fragments, terminal sequences, dynamic progress bars, and color output; 3) High noise, that is, the execution results contain a large number of American National Standards Institute (ANSI) color codes (\x1b[31m), carriage return overlay (\r), backspace redraw (\b), cursor movement sequences, etc.; 4) High reusability value, that is, subsequent steps need to retrieve error information, count failed test cases, and compare differences before and after.
[0032] If the entire execution result is injected into the LLM context, the following problems will occur: 1) Context window exhaustion, that is, a large amount of execution result output occupies the context space, and core task information is squeezed out of the window; 2) Increased response latency, that is, serialization, transmission and processing of a large amount of text significantly increases latency; 3) Semantic pollution, that is, the terminal sequence in the execution result interferes with the understanding of the large language model, leading to incorrect judgments; 4) Cost surge, that is, the cost of API calls charged by token increases linearly with the output volume.
[0033] Currently, the following two methods are commonly used to overcome the above problems: 1) fixed-length truncation, and 2) full retention + external storage. A brief introduction to these two methods is as follows: 1) Fixed length cut This method is often used in early command-line AI assistants and simple log viewing tools. Specifically, by setting a fixed upper limit for the output length (such as 10,000 characters), the execution result is truncated when the length limit is exceeded, and only the characters at the beginning of the execution result within the upper limit are retained. Then, the truncated execution result is injected into the context of the LLM.
[0034] However, this method has the following main drawbacks: Information loss is uncontrollable, meaning that critical erroneous information may be located in the discarded parts, causing large language models to be unable to diagnose the problem; It lacks semantic awareness, meaning the truncation location is random and may cut off the entire error stack or log entry; Undifferentiated processing means that all commands are treated the same, and there is no way to optimize processing based on different execution result types; Terminal sequences are not processed; for example, ANSI escape sequences will still contaminate the text.
[0035] 2) Full data retention + external storage This method is commonly used in enterprise-level log analysis systems and persistent command history management scenarios. Specifically, the execution results of commands are completely saved to an external file or database, and only the metadata of the execution results (file path, size, timestamp) is sent to the LLM. Then, when the LLM needs the execution results, it reads the execution results from the external file or database through a tool call.
[0036] However, this method has the following main drawbacks: The number of interaction rounds increases, meaning that the LLM needs to first determine whether a read operation is necessary before calling the appropriate tool, thus adding an extra round of interaction. Context fragmentation, i.e., the execution result is separated from the command, makes it difficult for LLM to establish a connection; Storage management is complex, meaning this method needs to handle issues such as file cleanup, permissions, and concurrent access. The initial response delay occurs even when the amount of data in the execution result is small, as it still needs to go through a "save-judge-read" process, resulting in a response delay.
[0037] To address the aforementioned issues, this disclosure provides a method for adding LLM context. This method categorizes execution results into complete retention, compressed, or discarded types based on the amount of data in the execution results and the proportion of terminal sequences in the execution results. Based on the final classification result, the execution results are processed accordingly before being added to the LLM context. This allows for differentiated processing based on different classification results, rather than adding all command execution results completely to the LLM context, thereby significantly saving context space for retaining more task history and system knowledge.
[0038] The method and apparatus for adding the context of an LLM according to exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings.
[0039] Figure 1 This is a flowchart illustrating a method for adding an LLM context according to an exemplary embodiment, such as... Figure 1 As shown, the method for adding an LLM context includes the following steps: In step 101, the execution result of the target command is obtained, wherein the target command is the command called during the inference process of the large language model; In step 102, the execution results are classified according to the amount of data in the execution results and the proportion of terminal sequences in the execution results, and the classification results of the execution results are obtained. The classification results are one of the following: complete retention type, compressed type, and discard type. The execution results corresponding to the complete retention type will be completely retained, the execution results corresponding to the compressed type will be compressed, and the execution results corresponding to the discard type will be discarded. In step 103, the execution result is processed according to the classification result of the execution result; In step 104, the processing result of the execution result is added to the context of the large language model.
[0040] As an example, the execution result of the aforementioned target command can also be referred to as "the output or output result obtained by executing the target command." This disclosure does not specifically limit this terminology; as long as the corresponding terms convey the same meaning, they are all included within the scope of this disclosure. The aforementioned terminal sequences may include, but are not limited to, terminal control characters and ANSI escape sequences. For ease of explanation, the following description uses terminal control characters and ANSI escape sequences as examples.
[0041] As an example, execution results with large data volumes can be discarded directly, while execution results with smaller data volumes can be retained and added to the LLM context. Alternatively, execution results with exceptionally large data volumes can be discarded directly, while execution results with smaller data volumes can be retained and added to the LLM context. Execution results with moderate data volumes can be compressed before being added to the LLM context. Alternatively, execution results with a large proportion of terminal sequences can be compressed, and the compressed result can be added to the LLM context, while execution results with a small proportion of terminal sequences and small data volumes can be retained and added to the LLM context. This disclosure does not impose specific limitations on these methods. Furthermore, to avoid discarding the execution results of critical commands, a preset list (also known as a discard whitelist) can be used. Only the execution results of commands in this preset list can be discarded. This disclosure does not impose specific limitations on the commands included in the preset command whitelist.
[0042] The following is an example of the process for classifying execution results.
[0043] According to an exemplary embodiment of this disclosure, in step 102 above, classifying the execution result based on the data volume of the execution result and the proportion of terminal sequences in the execution result to obtain a classification result of the execution result may include: determining the classification result of the execution result as complete retention type in response to the data volume of the execution result being less than or equal to a first threshold and the proportion of terminal sequences in the execution result being less than a third threshold; determining the classification result of the execution result as compressed type in response to the data volume of the execution result being greater than the first threshold and less than a second threshold, or, the proportion of terminal sequences in the execution result being greater than or equal to the third threshold and the data volume of the execution result being less than or equal to the first threshold, or, the data volume of the execution result being greater than or equal to the second threshold and the target command not belonging to the command in the preset list; and determining the classification result of the execution result as discarded type in response to the data volume of the execution result being greater than or equal to the second threshold and the target command belonging to the command in the preset list, wherein the second threshold is greater than the first threshold.
[0044] In this embodiment, when the amount of data in the execution result is small and the terminal sequence in the execution result accounts for a relatively small proportion, the complete execution result is added to the context of the LLM. When the amount of data in the execution result is large or the terminal sequence in the execution result accounts for a large proportion, the compressed execution result is added to the context of the LLM after compression. When the amount of data in the execution result is extremely large and the target command belongs to the command in the preset list, the execution result is directly discarded, and only the discard information indicating that the execution result is discarded is injected into the context of the LLM. Thus, the three-level classification strategy is implemented quickly and simply.
[0045] Specifically, it can be understood that when the amount of data in the execution result is greater than or equal to the second threshold and the target command belongs to the command in the preset list, the classification result of the execution result is discarded; when the amount of data in the execution result is less than or equal to the first threshold and the proportion of the terminal sequence in the execution result is less than the third threshold, the classification result of the execution result is complete and preserved; when other situations occur, the classification result of the execution result is compressed.
[0046] As an example, the first threshold, the second threshold, and the third threshold can be set as needed. For example, the first threshold can be set to 10KB, the second threshold can be set to 1MB, and the third threshold can be set to 40%. This disclosure does not limit the specific threshold.
[0047] As an example, the above-mentioned preset list contains commands whose execution results can be discarded. The specific settings can be configured as needed. For example, log query commands can be set in this preset list, and this disclosure does not limit this.
[0048] For example, with a first threshold of 10KB, a second threshold of 1MB, and a third threshold of 40%, the execution results can be divided into three categories based on the amount of data in the execution results and the proportion of terminal sequences in the execution results, and then processed differently for each category: The DISCARD type is suitable for scenarios such as large-volume log queries, continuous monitoring commands, and unimportant execution results. Specifically, when the data volume of the execution result of a target command is greater than or equal to 1MB and the target command belongs to a preset list (such as `docker logs --tail 10000`), the execution result is discarded, and only the metadata of the execution result, such as the command, return code, and execution result size, is retained. The content to be added to the LLM context is then determined based on the retained metadata. In this disclosure, the DISCARD type can also be referred to as the first type.
[0049] The COMPRESS category is suitable for scenarios such as build logs, test reports, and code inspection results. Specifically, when the data size of the execution result of the target command exceeds 10KB but is less than 1MB, or contains a high-density terminal sequence (i.e., the terminal sequence accounts for more than or equal to 40%) and the data size of the execution result is less than or equal to 10KB, or the data size of the execution result is greater than or equal to 1MB and the target command is not in a preset list, then the execution result is compressed. In this disclosure, the COMPRESS category can also be referred to as the second type.
[0050] The FULL type is suitable for scenarios such as simple queries, short output commands, and critical diagnostic information. Specifically, when the output data size of the target command's execution result is less than or equal to 10KB and there is no high-density noise (i.e., the terminal sequence ratio is less than 40%), the complete content of the execution result is preserved. In this disclosure, the FULL type can also be referred to as the third type.
[0051] According to an exemplary embodiment of this disclosure, the processing of the execution result in step 103 corresponding to the classification result of the execution result may include: in response to the classification result of the execution result being complete and retained, using the complete execution result as the processing result of the execution result; in response to the classification result of the execution result being compressed, compressing the execution result and using the compressed result as the processing result of the execution result; in response to the classification result of the execution result being discarded, discarding the execution result and using the discard information as the processing result of the execution result, wherein the discard information indicates that the execution result has been discarded.
[0052] In this embodiment, the execution results are processed accordingly based on the above three-level classification strategy.
[0053] As an example, still using a first threshold of 10KB, a second threshold of 1MB, and a third threshold of 40%, if the execution result is classified as discardable, the execution result is discarded, and only the metadata of the execution result (such as command, return code, and execution result size) is retained. The discard information is then determined based on the metadata and used as the processing result. For example, if `docker logs app --tail 10000` produces 5MB of logs, only the following discard information, "Output 5MB, discarded," is injected into the LLM context. If the execution result is classified as compressed, the execution result is compressed, and the compressed result is used as the processing result. For example, if `npm test` produces 50KB of execution result, the execution result is compressed into a 5KB summary (such as failed test cases + statistics), and this 5KB summary is added to the LLM context. If the execution result is classified as fully retained, the complete execution result is used as the processing result. For example, if `ls -la` produces 2KB of execution result, the complete execution result is added to the LLM context.
[0054] According to an exemplary embodiment of the present disclosure, when the format of the execution result is not within a preset format set, the above-mentioned compression of the execution result may include: retaining a first predetermined number of lines in the header section and a second predetermined number of lines in the tail section of the execution result; deleting the remaining lines in the execution result other than the first predetermined number of lines and the second predetermined number of lines; and inserting a truncation mark between the first predetermined number of lines and the second predetermined number of lines.
[0055] Since the lines in the header section (i.e., at the beginning) of the execution result are generally used to capture command startup information and environment configuration, while the lines in the tail section (i.e., at the end) of the execution result are generally used to capture the final result and error summary, this embodiment can compress a large volume of execution results into a smaller volume while ensuring the retention rate of key information. This reduces irrelevant information and speeds up LLM diagnosis, thereby improving the accuracy of the compressed execution results.
[0056] As an example, the first and second predetermined quantities mentioned above can be set as needed, and the first and second predetermined quantities can be the same or different; this disclosure does not limit this. The truncation marker mentioned above can be a preset marker that can be set as needed, such as "...(N lines omitted)..."; this disclosure does not limit this. The set of preset formats mentioned above can be set as needed, and may include, but is not limited to, the following formats: compilation logs, test reports, and code inspection results.
[0057] As an example, the operation of compressing the execution result in the above embodiments can also be called a "head and tail preservation strategy." This head and tail preservation strategy is expressed in terms of lines. If necessary, it can also be expressed in terms of bits, bytes, etc., and this disclosure does not limit this. For example, assuming it is expressed in terms of bits, the first predetermined number of lines in the head section and the second predetermined number of lines in the tail section of the above execution result can also be expressed as: the third predetermined number of bits in the head section and the fourth predetermined number of bits in the tail section of the execution result, wherein the first predetermined number of lines contains the third predetermined number of bits, and the second predetermined number of lines contains the fourth predetermined number of bits, and this disclosure does not limit this.
[0058] As an example, taking a first and second predetermined quantity of 100 lines each, when the execution result is obtained and the classification result of the execution result is determined to be compressed, the beginning and end retention strategy can be as follows: retain the first 100 lines of the execution result (i.e., the 100 lines in the header section, that is, from line 1 to line 100) to capture command startup information and environment configuration, and retain the last 100 lines of the execution result (i.e., the 100 lines in the tail section, that is, from the last line to the 100th line from the end) to capture the final result and error summary. Then, insert a truncation mark "...(N lines omitted)..." in the middle part of the execution result.
[0059] It should be noted that after compressing the execution result by adopting the beginning and end preservation strategy, a summary can be extracted from the compressed execution result to further reduce the data volume of the compressed execution result. For example, semantic extraction can be performed on the compressed execution result, which is not limited in this disclosure.
[0060] According to an exemplary embodiment of this disclosure, in response to the presence of a line containing an erroneous keyword among the remaining lines, the line containing the erroneous keyword is retained. This embodiment's operation can also be referred to as an "error-priority strategy." Applying this "error-priority strategy" ensures that critical error information is not truncated and guarantees a critical information retention rate exceeding 95%, ensuring that the LLM can obtain sufficient diagnostic information.
[0061] As an example, when adopting an error-first strategy, a scan can be performed on the rows deleted during the implementation of the aforementioned "first and last retention" strategy (i.e., the remaining rows mentioned above) to identify whether any of these rows to be deleted contain error keywords. If rows containing error keywords are identified through the scan, in addition to the rows included in the "first and last retention" strategy, the identified rows containing error keywords are additionally retained to ensure that critical diagnostic information is not truncated. It should be noted that error keywords can be, but are not limited to, those shown in Table 1 below: Table 1 Examples of Incorrect Keywords
[0062] As an example, when both the beginning and end retention strategy and the error priority strategy are used simultaneously, taking a first and second predetermined number of lines as both 100, and assuming the original execution result (i.e., the original output) has 1000 lines, the details are as follows: Lines 1-50: Files compiled successfully Lines 51-900: More successful compilations Lines 901-920: Compilation error details Lines 921-1000: Linking and Cleanup Suppose that the compressed result obtained after applying the head-and-tail retention strategy and the error-first strategy to the original execution result is 220 lines, as shown below: Lines 1-100: The first 100 lines (startup information) [Truncation mark: 780 lines omitted] Lines 701-720: Error lines (errors are prioritized and retained). Lines 901-1000: The last 100 lines (final result) For example, the code for intelligent truncation using the head-and-tail retention strategy and the error-first strategy can be, but is not limited to, the following:
[0063] It should be noted that after adopting the beginning and end retention strategy and the error priority strategy, the compression result can also be extracted to further reduce the data volume of the compression result. For example, semantic extraction can be performed on the compression result, which is not limited in this disclosure.
[0064] As an example, this disclosure is not limited to the head-and-tail retention strategy and the error-first strategy. For instance, a semantic boundary strategy can be used instead of the head-and-tail retention strategy and the error-first strategy, and this disclosure does not limit its application. The aforementioned semantic boundary strategy can be implemented using an LLM (Liquid Management Model), that is, the execution result is input into an LLM, and the LLM extracts key semantic information as the compressed execution result, and this disclosure does not limit its application in this regard.
[0065] According to another exemplary embodiment of this disclosure, when the format of the execution result is within a preset format set, the above-mentioned compression of the execution result may include: determining a summary format corresponding to the format of the execution result; extracting summary information from the execution result based on the summary format; and obtaining a compression result by representing the summary information as a summary format.
[0066] This embodiment provides a dedicated summary format for specific formats (such as compilation logs, test reports, code inspection results, etc.), enabling the direct extraction of summary information such as failed test cases and error locations. This significantly reduces the amount of data in the execution results while ensuring the semantic correctness of the output and improving the efficiency of error diagnosis.
[0067] As an example, the above set of preset formats can be set as needed. For example, it may include, but is not limited to, the following formats: compilation logs, test reports, and code inspection results.
[0068] As an example, for compile logs, the compression results and summary formats may include, but are not limited to, the following: Identify the following compilation logs:
[0069] Extract summary information:
[0070] The compressed result corresponding to the summary format:
[0071] For example, the code that generates the compressed result corresponding to the compilation log can be, but is not limited to, the following:
[0072] As an example, for test reports, the compressed results and summary formats can be, but are not limited to, the following: Identify the following test reports:
[0073] Extract summary information:
[0074] The compressed result corresponding to the summary format:
[0075] It should be noted that the structured information extracted above only needs to be convenient for subsequent automated processing. Moreover, for execution results in new specific formats, corresponding summary formats can be designed. That is, this disclosure supports the extension of new execution result format recognizers.
[0076] As an example, this embodiment can also use a lightweight LLM to perform real-time summarization of the execution results. LLM-based summarization has higher quality, can understand complex semantics, and is highly adaptable, without the need to write summarization rules for each format.
[0077] According to an exemplary embodiment of this disclosure, when the classification result of the execution result is compressed or fully preserved, before performing the processing corresponding to the classification result of the execution result in step 103 above, the terminal sequence in the execution result can also be removed (also referred to as "filtering the terminal sequence in the execution result"), wherein the terminal sequence includes at least terminal control characters or ANSI escape sequences.
[0078] In this embodiment, filtering ANSI escape sequences can eliminate noise interference such as color encoding, and filtering terminal control characters can restore static text, enabling LLM to directly understand plain text content without processing terminal control logic, thereby improving the semantic understanding accuracy of the execution results.
[0079] According to an exemplary embodiment of this disclosure, the removal of terminal sequences from the execution result may include: in response to the terminal sequence being a terminal control character, removing the terminal control character from the execution result by simulating the dynamic process of the terminal control character in the terminal; and in response to the terminal sequence being an ANSI escape sequence, removing the ANSI escape sequence from the execution result by using regular expressions.
[0080] This embodiment allows for the convenient and quick deletion of terminal control characters and ANSI escape sequences.
[0081] As an example, in the case of a carriage return character in the terminal control strings, the execution result can be split into multiple segments based on the carriage return character; only the last segment is retained. For example, assuming the execution result is: "Downloading...10%\rDownloading... 50%\rDownloading... 100%", the corresponding processing logic could be, but is not limited to, the following: 1) Divide into multiple segments by \r; 2) Each segment covers the previous segment; 3) Only keep the last segment; The execution result after the above filtering process is "Downloading... 100%".
[0082] For example, the code for handling carriage return overwrites can be, but is not limited to, the following:
[0083] As an example, in the case of a backspace character in the terminal control characters, the execution result can be scanned from front to back, and in response to the detection of a backspace character, the character preceding the backspace character can be deleted. For example, assuming the execution result is "Hello\b\b\b\b\bWorld", the corresponding processing logic could be, but is not limited to, the following: 1) When encountering \b, delete the preceding character; 2) Construct the final text character by character; The execution result after the above filtering process is "World".
[0084] As an example, if the terminal sequence in the execution result is an ANSI escape sequence, the mechanism for identifying and removing ANSI escape sequences can be, but is not limited to, the following: 1) CSI Sequence Processing (Control Sequence Introducer): Format: ESC [<parameters><command>; Example: \x1b[31m (red text), \x1b[1;32m (bold green); Processing method: Remove completely, retain plain text content. .
[0085] 2) OSC Sequence Processing (Operating System Command): Format: ESC ]<command>;<parameter>BEL; Example: \x1b]0;Window title\x07 (Set terminal title); Solution: Remove completely .
[0086] 3) Cursor control sequence: ESC [ n A (move up n rows); ESC [ n B (move down n rows); ESC [n C (right shift n columns); ESC [ n D (shift left by n columns); Processing method: Remove the sequence and do not simulate cursor movement. .
[0087] For example, the code that removes ANSI escape sequences using regular expressions can be, but is not limited to, the following:
[0088] To better understand the above embodiments, the following is in conjunction with... Figure 2 and Figure 3 The above embodiments of this disclosure will be systematically described.
[0089] Figure 2 A system architecture diagram illustrating a method for adding context for LLM is shown, such as... Figure 2 As shown, this disclosure proposes a method for adding context to an LLM system using a "three-level classification strategy + four-layer filtering + intelligent summarization or intelligent truncation" approach. The core architecture of this system includes the following layers: Output classification layer: identifies the command type of the target command and classifies it according to the command's execution result (i.e., Figure 2 Based on the amount of data output and the proportion of terminal sequences, the execution results of the target command are divided into three categories: DISCARD, COMPRESS, and FULL. Terminal sequence filtering layer: Removes ANSI escape sequences and terminal control characters from the execution result; Length control layer: performs intelligent truncation and / or extracts summaries of the intelligent truncation results for execution results classified as compressed. In addition to the head and tail preservation strategy and error priority strategy, semantic boundary strategy can also be used during intelligent truncation, which is not limited in this disclosure. During summary extraction, the total number of lines, total number of bytes, and truncation ratio of the intelligent truncation results can be extracted, which is not limited in this disclosure.
[0090] Structured summary layer: Extracts key information from execution results in a specific format and displays it in the corresponding summary format.
[0091] Figure 3 This demonstrates the specific process of adding an LLM context, such as... Figure 3As shown, after obtaining the execution result of the target command, the size of the data in the execution result is detected. If the data size is greater than or equal to 1MB and the command belongs to the preset list, the execution result is classified as discarded. If the data size is less than 1MB but greater than 10KB, or if the proportion of terminal sequences in the execution result is greater than or equal to 40% and the data size is less than or equal to 10KB, or if the data size is greater than or equal to 1MB and the command does not belong to the preset list, the execution result is classified as compressed. If the data size is less than or equal to 10KB and the proportion of terminal sequences in the execution result is less than 40%, the execution result is classified as intact and retained. If the execution result is classified as "complete preservation," the terminal sequence in the execution result is first filtered. Then, the filtered execution result is used as the processing result and added to the LLM context. If the execution result is classified as "compressed," the terminal sequence in the execution result is first filtered. Then, for execution results belonging to a specific format set (such as compilation logs, test reports, and code inspection results), intelligent extraction is performed according to the corresponding digest format, and the compressed result corresponding to the digest format is obtained. This processing result is added to the LLM context. For execution results not belonging to a specific format set, intelligent truncation is performed, such as implementing a beginning and end preservation strategy (or even a further error priority strategy). The specific process has been described above and will not be elaborated here. If the execution result is classified as "discardable," the metadata of the execution result is retained, and discard information is determined based on the metadata. This discard information is used to mark that the execution result has been discarded. Figure 3 (Add annotation information) to the context of the LLM.
[0092] like Figure 3 The classification decision of the execution results shown can be illustrated in Table 2 below: Table 2 Examples of Execution Result Classification Decisions
[0093] In summary, this disclosure, while retaining key information, compresses a large volume of execution results to a size that the context window can handle, preventing core task information from being squeezed out and resolving the contradiction between the volume of execution results and the context window. Furthermore, this disclosure identifies and removes terminal control characters such as ANSI escape sequences, carriage return overlays, and backspace redraws, preventing them from interfering with LLM text understanding and resolving semantic pollution of terminal sequences. Moreover, this disclosure dynamically determines the retention value of execution results based on the data volume of the execution results, the proportion of terminal sequences, and the command type, implementing a differentiated processing strategy. Finally, this disclosure extracts key information and generates corresponding structured summaries for execution results in specific formats (such as compilation logs, test reports, code inspection results, etc.), achieving intelligent summarization of structured output.
[0094] In addition, based on solving the above-mentioned problems, this disclosure also achieves the following technical effects: 1) Reduced response latency: Compression of execution results reduces serialization and transmission time; discarding execution results classified as discardable avoids the processing overhead of large volumes of logs; overall response time is reduced by 40-60%. 2) Reduced API call costs: Lexicon consumption is reduced by 70-90%; API costs charged per lexicon are significantly reduced; cost savings are significant for high-frequency usage scenarios. 3) Improved user experience: Reduced waiting time and improved interaction smoothness; key information is not lost, and diagnostic accuracy is improved; supports large-scale log analysis tasks.
[0095] It should be noted that this disclosure can also truncate and compress the execution results in real time during command execution, instead of waiting for the command to complete before processing. This results in lower memory usage, making it suitable for scenarios with extremely large output, and it allows processing to start earlier, reducing waiting time.
[0096] Alternatively, the method disclosed herein can be replaced by the following: 1) storing the complete execution results in a vector database, with LLM retrieving relevant fragments through semantic retrieval. This supports semantic relevance retrieval rather than keyword matching, and allows retrieval of execution results from historical commands, supporting cross-session analysis; 2) dividing the execution results into multiple levels (such as summary, details, and complete), and then LLM can load these levels as needed. This provides greater flexibility, allowing more details to be loaded gradually as needed, and supports incremental analysis.
[0097] Figure 4 This is a block diagram illustrating an apparatus for adding context to an LLM according to an exemplary embodiment. (Refer to...) Figure 4 The device includes: The acquisition unit 40 is configured to acquire the execution result of the target command, wherein the target command is the command called during the inference process of the large language model; the classification unit 42 is configured to classify the execution result according to the amount of data in the execution result and the proportion of terminal sequences in the execution result, and obtain the classification result of the execution result, wherein the classification result is one of three types: complete retention, compressed, and discard. The execution result corresponding to the complete retention type will be completely retained, the execution result corresponding to the compressed type will be compressed, and the execution result corresponding to the discard type will be discarded; the processing unit 44 is configured to perform processing on the execution result corresponding to the classification result of the execution result; the adding unit 46 is configured to add the processing result of the execution result to the context of the large language model.
[0098] According to an exemplary embodiment of this disclosure, the classification unit 42 is further configured to: determine the classification result of the execution result as complete retention type in response to the data volume of the execution result being less than or equal to a first threshold and the proportion of terminal sequences in the execution result being less than a third threshold; determine the classification result of the execution result as compressed type in response to the data volume of the execution result being greater than the first threshold and greater than the second threshold, or the proportion of terminal sequences in the execution result being greater than or equal to the third threshold and the data volume of the execution result being less than or equal to the first threshold, or the data volume of the execution result being greater than or equal to the second threshold and the target command not belonging to the command in the preset list; and determine the classification result of the execution result as discarded type in response to the data volume of the execution result being greater than the second threshold and the target command belonging to the command in the preset list, wherein the second threshold is greater than the first threshold.
[0099] According to an exemplary embodiment of this disclosure, the processing unit 44 is further configured to: in response to the classification result of the execution result being of the complete retention type, use the complete execution result as the processing result of the execution result; in response to the classification result of the execution result being of the compressed type, compress the execution result and use the compressed result as the processing result of the execution result; in response to the classification result of the execution result being of the discard type, discard the execution result and use the discard information as the processing result of the execution result, wherein the discard information indicates that the execution result has been discarded.
[0100] According to an exemplary embodiment of the present disclosure, when the format of the execution result is not within a preset format set, the processing unit 44 is further configured to retain a first predetermined number of lines in the header segment and a second predetermined number of lines in the tail segment of the execution result; delete the remaining lines in the execution result other than the first predetermined number of lines and the second predetermined number of lines; and insert a truncation mark between the first predetermined number of lines and the second predetermined number of lines.
[0101] According to an exemplary embodiment of this disclosure, the processing unit 44 is further configured to retain the line containing the erroneous keyword in response to the existence of a line containing the erroneous keyword in the remaining lines.
[0102] According to an exemplary embodiment of this disclosure, when the format of the execution result is within a preset format set, the processing unit 44 is further configured to determine a summary format corresponding to the format of the execution result; extract summary information from the execution result based on the summary format; and obtain a compressed result by representing the summary information in a summary format.
[0103] According to an exemplary embodiment of this disclosure, when the classification result of the execution result is compressed or fully preserved, the processing unit 44 is further configured to remove the terminal sequence in the execution result before performing processing on the execution result corresponding to the classification result of the execution result, wherein the terminal sequence includes at least terminal control characters or ANSI escape sequences.
[0104] According to an exemplary embodiment of this disclosure, the processing unit is further configured to remove terminal control characters from the execution result in response to the terminal sequence being terminal control characters by simulating the dynamic process of terminal control characters in the terminal; and to remove ANSI escape sequences from the execution result by using regular expressions in response to the terminal sequence being an ANSI escape sequence.
[0105] According to embodiments of this disclosure, an electronic device may be provided. Figure 5 This is a block diagram of an electronic device 500 according to an embodiment of the present disclosure. The electronic device includes at least one memory 501 and at least one processor 502. The at least one memory stores a set of computer-executable instructions. When the set of computer-executable instructions is executed by the at least one processor, a method for adding context of a large language model according to an embodiment of the present disclosure is performed.
[0106] As an example, electronic device 500 may be a PC, tablet, personal digital assistant, smartphone, or other device capable of executing the aforementioned set of instructions. Here, electronic device 1000 is not necessarily a single electronic device, but may be a collection of any devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. Electronic device 500 may also be part of an integrated control system or system manager, or may be configured to interconnect with a portable electronic device locally or remotely (e.g., via wireless transmission) through an interface.
[0107] In electronic device 500, processor 502 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, processor 502 may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, etc.
[0108] The processor 502 can execute instructions or code stored in memory, and the memory 501 can also store data. Instructions and data can also be sent and received over a network via a network interface device, wherein the network interface device can employ any known transmission protocol.
[0109] The memory 501 may be integrated with the processor 502, for example, by placing RAM or flash memory within an integrated circuit microprocessor. Alternatively, the memory 501 may include a separate device, such as an external disk drive, a storage array, or other storage device usable by any database system. The memory 501 and the processor 502 may be operatively coupled, or may communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor 502 to read files stored in the memory 501.
[0110] In addition, the electronic device 500 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the electronic device can be interconnected via a bus and / or network.
[0111] According to embodiments of this disclosure, a computer-readable storage medium may also be provided, wherein when instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor causes to perform the context addition method of the large language model of this disclosure. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, servers, etc. Furthermore, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.
[0112] According to embodiments of this disclosure, a computer program product is provided, including computer instructions, which, when executed by a processor, implement a method for adding context to a large language model according to embodiments of this disclosure.
[0113] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.
[0114] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.
Claims
1. A method for adding context to a large language model, characterized in that, include: Obtain the execution result of the target command, wherein the target command is a command called during the inference process of the large language model; Based on the amount of data in the execution results and the proportion of terminal sequences in the execution results, the execution results are classified to obtain the classification results of the execution results. The classification results are one of three types: complete retention, compressed, and discard. The execution results corresponding to the complete retention type will be completely retained, the execution results corresponding to the compressed type will be compressed, and the execution results corresponding to the discard type will be discarded. The execution result is then processed according to the classification result of the execution result; The processing result of the execution is added to the context of the large language model.
2. The adding method according to claim 1, characterized in that, The step of classifying the execution results based on the data volume of the execution results and the proportion of terminal sequences in the execution results to obtain the classification results of the execution results includes: If the amount of data in the execution result is less than or equal to a first threshold and the proportion of terminal sequences in the execution result is less than a third threshold, the classification result of the execution result is determined to be complete and preserved. In response to the following: the data volume of the execution result is greater than the first threshold and less than the second threshold; or, the proportion of terminal sequences in the execution result is greater than or equal to the third threshold and the data volume of the execution result is less than or equal to the first threshold; or, the data volume of the execution result is greater than or equal to the second threshold and the target command does not belong to the commands in the preset list, the classification result of the execution result is determined to be compressed; and If the amount of data in the execution result is greater than or equal to the second threshold and the target command belongs to the command in the preset list, the classification result of the execution result is determined to be discarded. Wherein, the second threshold is greater than the first threshold.
3. The adding method according to claim 1, characterized in that, The processing of the execution result in accordance with the classification result of the execution result includes: In response to the classification result of the execution result being the complete preserved type, the complete execution result is taken as the processing result of the execution result; In response to the classification result of the execution result being of the compressed type, the execution result is compressed and the compressed result is used as the processing result of the execution result; In response to the classification result of the execution result being of the discard type, the execution result is discarded, and the discard information is used as the processing result of the execution result, wherein the discard information indicates that the execution result has been discarded.
4. The adding method as described in claim 3, characterized in that, When the format of the execution result is not within the preset format set, the compression of the execution result includes: Retain a first predetermined number of rows in the header section and a second predetermined number of rows in the tail section of the execution result; Delete all rows in the execution result except for the first predetermined number of rows and the second predetermined number of rows; Insert a truncation marker between the first predetermined number of rows and the second predetermined number of rows.
5. The adding method as described in claim 4, characterized in that, Also includes: In response to the presence of a row containing an incorrect keyword among the remaining rows, the row containing the incorrect keyword is retained.
6. The adding method as described in claim 3, characterized in that, When the format of the execution result is within a preset format set, compressing the execution result includes: Determine the summary format corresponding to the format of the execution result; Extract summary information from the execution result based on the summary format; The compression result is obtained by representing the summary information in the summary format.
7. The adding method as described in claim 1, characterized in that, If the classification result of the execution result is compressed or fully preserved, before performing the processing corresponding to the classification result on the execution result, the method further includes: Remove the terminal sequence from the execution result, wherein the terminal sequence includes at least terminal control characters or ANSI escape sequences.
8. The adding method as described in claim 7, characterized in that, The removal of terminal sequences from the execution result includes: In response to the terminal sequence being terminal control characters, the terminal control characters are removed from the execution result by simulating the dynamic process of the terminal control characters in the terminal. In response to the terminal sequence being an ANSI escape sequence, the ANSI escape sequence is removed from the execution result using regular expressions.
9. A device for adding context to a large language model, characterized in that, include: The acquisition unit is configured to acquire the execution result of a target command, wherein the target command is a command invoked during the inference process of a large language model; The classification unit is configured to classify the execution result according to the amount of data in the execution result and the proportion of terminal sequences in the execution result, and obtain the classification result of the execution result. The classification result is one of three types: complete retention, compressed, and discard. The execution result corresponding to the complete retention type will be completely retained, the execution result corresponding to the compressed type will be compressed, and the execution result corresponding to the discard type will be discarded. The processing unit is configured to perform processing on the execution result corresponding to the classification result of the execution result; The addition unit is configured to add the processing result of the execution result to the context of the large language model.
10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method for adding context to a large language model as described in any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor causes the processor to perform the method of adding context to a large language model as described in any one of claims 1 to 6.
12. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by the processor, they implement the method for adding context to a large language model as described in any one of claims 1 to 8.