Context virtualization method for tool-type agents

Abstract

The application provides a context virtualization method of a tool-type intelligent agent, and belongs to the technical field of computers. The method comprises the following steps: an intelligent agent constructs a copy of a request message, and if a text tool result exists in the copy of the request message, a virtualization condition judgment is performed on the text tool result; if the virtualization condition is met, the complete source text of the text tool result is stored in an external storage medium, and a storage location index is stored; corresponding text content is read from the external storage medium according to the location index, and a summary check is performed; if the check is passed, the text tool result is virtually processed to obtain a marker; the text tool result in the copy of the request message is replaced by the marker to obtain a bounded context request, and the request is sent to a large language model to enable the large language model to perform reasoning. The problem that a large text tool result is repeatedly transmitted and continuously occupies a large number of word units in multiple rounds of reasoning requests can be solved.

Description

Technical Field

[0001] This application belongs to the field of computer technology, and in particular relates to a context virtualization method for tool-type intelligent agents. Background Technology

[0002] In tool-based AI agent applications, the agent is typically driven by a large language model, which is responsible for understanding the task, planning steps, and deciding which external tools to invoke. During operation, the host program interacts with the large language model. Each time the host program needs the large language model to perform inference, it reconstructs a request message based on the current dialogue context between the agent and the large language model and sends this message to the large language model. The large language model then performs inference based on the dialogue context in the request message and returns the corresponding inference result. The dialogue context typically includes historical conversation records, tool invocation parameters, and textual tool results returned by the external tool.

[0003] Because the host program reconstructs the request message based on the current dialogue context during each model inference, this means that once a large text tool result enters the dialogue context, it will be repeatedly transmitted in every subsequent request. For example, the output of a single test execution (text tool result) might contain thousands of lines of logs. Once this output enters the dialogue context, it will consume thousands or even tens of thousands of words of input cost in each subsequent round of inference. Since the context window capacity that a large language model can handle in a single inference is limited, dialogue context management is necessary to control the number of words it occupies.

[0004] In related technologies, solutions to the above problems mainly fall into two categories. The first is passive compression: when the context window is about to overflow, the host program performs digest-style compression on the historical session records and replaces the original historical session records with this digest. The second is truncation: directly truncating excessively long text tool results, retaining only the beginning and discarding the end.

[0005] However, both of these methods have significant shortcomings in practical applications. Since passive compression and truncation are both lossy processes, once the original text tool results are compressed or truncated, the large language model can no longer obtain complete information. When subsequent inference needs to reference discarded details, such as specific error line numbers, a record in the search results, or a specific paragraph in a file, the task cannot be completed, or the tool call needs to be re-executed, resulting in redundant calculations and wasted resources. Furthermore, these two schemes are mostly passively triggered only when the context window is about to overflow, causing each inference request before that to bear unnecessary lexical overhead, forming a pattern of wasting first and then cleaning up, resulting in high inference costs. Summary of the Invention

[0006] This application provides a context virtualization method for tool-type intelligent agents, which can solve the problems in related technologies where large text tool results are repeatedly transmitted in multi-round inference requests, continuously occupying a large number of words, and the original information cannot be completely recovered after compression.

[0007] In a first aspect, embodiments of this application provide a context virtualization method for a tool-type intelligent agent, the method comprising: The agent constructs a copy of the request message, which is used to request the large language model to perform inference on the target task. If a text tool result exists in the copy of the request message, then a virtualization condition judgment is performed on the text tool result. The text tool result is the text output generated and returned by the external tool after the agent calls the external tool. If the text tool results meet the preset virtualization conditions, the complete source text of the text tool results is stored in the external storage medium, and the location index of the complete source text of the text tool results in the external storage medium is also stored. The corresponding text content is read from the external storage medium based on the location index, and the text content is compared with the complete source text of the text tool results for summary verification. If the verification passes, the text tool result is virtualized to obtain a token. The token is used to indicate the storage location of the complete source text of the text tool result in the external storage medium. The number of tokens in the token is less than the number of tokens in the complete source text of the text tool result. The text tool result in the copy of the request message is replaced with a token to obtain a bounded context request. The bounded context request is then sent to the large language model so that the large language model can perform inference based on the bounded context request.

[0008] The first aspect of the beneficial effects is as follows: When there is a need to request a large language model for inference, the agent constructs a copy of the request message. The copy is identical to the original request message and does not directly modify the original message. Therefore, the original request message and the agent's existing history remain unchanged. Furthermore, if the copy contains text tool results, it performs a virtualization condition check. This automatically identifies large tool results worthy of virtualization, avoiding indiscriminate processing of all tool results and preventing meaningless storage and validation of small tool results—saving only a few lexical units while adding extra overhead. When the virtualization condition is met, the complete source text is stored on external storage media, and the location index is recorded. Therefore, the original information is completely preserved, unlike compression or truncation which can lead to text loss. The location index also provides information for subsequent retrieval. The system provides a basis for this process. Then, it reads back the stored content based on the location index and performs a summary verification with the original complete source text. This ensures that the stored content is completely consistent with the original text, preventing storage damage or tampering and guaranteeing reliable recoverability. After successful verification, the text tool result is virtualized to generate a token. The token indicates the storage location and has fewer lexical units than the complete source text, thus replacing large sections of the original text in the context and significantly reducing the number of lexical units occupied by the context. Finally, the token replaces the text tool result in the copy, obtaining a bounded context request, which is then sent to the large language model for inference. Therefore, the large language model sees a reduced bounded context during each inference, no longer carrying the complete large result. This saves lexical units from the first request, solving the problem of lexical waste caused by repeatedly transmitting large results and the passive response of wasting resources before cleaning up in related technologies.

[0009] In one possible implementation, the method further includes: Receive text viewing requests from the large language model. Text viewing requests are used to request to view a specified page in the text tool results. Text viewing requests are generated by the large language model when it determines the specified page to be viewed based on the tags. The specified page is read from external storage media based on the text viewing request, and the content of the specified page is sent to the large language model.

[0010] In this implementation, the agent receives a text viewing request generated by the large language model based on the tokens. This request is used to view a specific page in the text tool results, enabling the large language model to proactively initiate a request when it needs to view the virtualized content, rather than only seeing the tokens. Subsequently, the agent reads the content of the specified page from the external storage medium according to the request and sends it to the large language model, enabling the large language model to accurately obtain the complete original text of the specified page, achieving lossless recovery. Furthermore, the access to the page triggered by the tokens and specified by the request is path-determined and auditable, solving the problem in related technologies where information cannot be recovered once it is compressed or truncated, and subsequent inference cannot reference discarded details.

[0011] In one possible implementation, the agent includes a text viewing tool for retrieving text tool results from an external storage medium; the method further includes: If the text tool results that have passed the digest verification exist in the external storage medium, the agent sets the text viewing tool to a model-visible state; the model-visible state refers to the state that allows the large language model to initiate a text viewing request. Based on the text viewing request, the specified page is read from external storage media, and the content of the specified page is sent to the large language model, including: The agent determines the target location of the specified page in the external storage medium based on the text viewing request; The text viewer is invoked to read the specified page from the target location on the external storage medium and the content of the specified page is sent to the large language model.

[0012] In this implementation, the agent includes a text viewing tool used to retrieve text tool results from external storage media, providing a unified tool interface for on-demand reading and avoiding direct manipulation of storage by the large language model. When text tool results that have passed digest verification exist in the external storage media, the agent sets the text viewing tool to be visible to the model, exposing the tool only to the large language model when there is indeed recoverable content, avoiding interference caused by invalid exposure and improving system stability. Subsequently, the agent determines the target location of the specified page based on the text viewing request, calls the text viewing tool to read the specified page and sends it to the large language model. Precise reading is completed through deterministic location indexing and tool invocation, making the access path predictable and auditable. This solves the problems in related technologies such as the lack of a unified access interface, the inability of the model to search and read persistent content on demand, and the uncertainty of the search path caused by relying on the autonomous decision-making of the large language model.

[0013] In one possible implementation, if a text tool result exists in the copy of the request message, then a virtualization conditional judgment is performed on the text tool result, including: The results from the text tool are virtualized to obtain candidate tokens; If the difference between the number of lexical units in the complete source text of the text tool result and the number of lexical units in the candidate tokens reaches the preset lexical unit saving threshold, then the text tool result is determined to meet the virtualization conditions.

[0014] In this implementation, the text tool results are virtualized to obtain candidate tokens. The candidate tokens are pre-rendered to provide a basis for subsequent calculation of savings, making the virtualization condition judgment have a quantifiable comparison object. Then, the difference between the number of tokens in the complete source text and the number of tokens in the candidate tokens is calculated. The text tool result is determined to meet the virtualization condition only when the difference reaches a preset saving token threshold. This can automatically identify large text tool results that are truly worth virtualizing, avoiding unnecessary storage and verification processing for small results. This solves the problem in related technologies that cannot automatically identify large tool results worth virtualizing, while reducing unnecessary processing overhead and realizing an active virtualization strategy based on the token saving threshold.

[0015] In one possible implementation, a summary verification is performed on the text content and the complete source text of the text tool results, including: Summarize the complete source text of the text content and the text tool result separately to obtain a first summary corresponding to the text content and a second summary corresponding to the complete source text of the text tool result. Compare the first and second abstracts; If the first and second summaries match, the verification is considered successful. If the first and second summaries are inconsistent, the verification is deemed to have failed.

[0016] In this implementation, the text content read from the external storage medium and the complete source text of the text tool results are processed separately to generate a first summary and a second summary. These separate summaries provide the basis for subsequent comparisons, making the verification process quantifiable and executable. The first and second summaries are then compared; if they match, the verification passes; otherwise, it fails. This accurately determines whether the stored content is completely consistent with the original text, ensuring the integrity of the storage and resolving the issue of potential content corruption or tampering after storage. In the case of a failed verification, the request message sent to the large language model is cancelled, preventing the large language model from obtaining incomplete or erroneous information and preventing inference based on incorrect information. This further ensures the reliability of information recovery and avoids the large language model directly obtaining information that may exceed the context window, thus preventing resource waste.

[0017] In one possible implementation, the copy of the request message includes a copy of the historical session record; In this context, when an agent builds a copy of a request message, the historical session record remains unchanged.

[0018] In this implementation, the copy of the request message includes a copy of the historical session record, enabling the copy to fully reflect the dialogue history, providing the necessary context for the large language model, and providing a complete message structure for the results of the subsequent text replacement tool. When the agent constructs the copy of the request message, the historical session record remains unchanged, thus not modifying the agent's persistent historical record, achieving low-intrusion integration without modifying the original dialogue history management logic. This solves the problems of loss of original information and complex integration caused by directly modifying the historical record in related technologies, while preserving the original historical record.

[0019] In one possible implementation, if the verification passes, the text tool result is virtualized to obtain tokens, including: The complete source text of the text tool result is divided into multiple logical pages, and the number of tokens in each logical page does not exceed the preset page token threshold. Select a representative page from multiple logical pages. The representative page includes at least one of the following: home page, middle page, and last page. Generate a text tool result identifier based on the representative page, and assemble the text tool result identifier and the representative page to obtain a marker.

[0020] In this implementation, the complete source text is divided into multiple logical pages, with the number of lexical units in each logical page not exceeding a preset threshold. This divides large blocks of text into pages of manageable size, facilitating subsequent page-by-page reading and locating, and avoiding reading excessive amounts of content at once. At least one of the first, middle, or last pages is selected as a representative page from these logical pages. This allows the large language model to obtain a representative portion of the text tool results upon first encountering the marker, aiding in understanding the context and determining whether further viewing is necessary. This addresses the problem of the model being unable to perceive compressed or truncated content. A text tool result identifier is generated based on the representative page. The text tool result identifier and the representative page are then assembled to form a marker, which contains both index information and representative content. This provides a clear page index for subsequent text viewing requests, generating a deterministic and bounded model-visible view, thus resolving the issues of lacking stable markers and page selection.

[0021] In one possible implementation, before the agent constructs a copy of the request message, the method further includes: Upon receiving a user message, the agent sends the user message to the large language model so that the large language model can reason about the user message. Receive the inference results fed back from the large language model; When the inference result instructs the agent to invoke an external tool, the agent invokes the external tool and receives the text tool result generated and returned by the external tool.

[0022] In this implementation, before constructing a copy of the request message, the user message is received and sent to the large language model for inference, clarifying the starting point of the interaction between the agent and the large language model, and providing a clear trigger source for subsequent tool calls and the generation of text tool results. The inference results fed back by the large language model are received, and when the inference results indicate the invocation of external tools, the external tools are invoked and the text tool results generated and returned by the external tools are received, clarifying the generation process of text tool results. This ensures that the context virtualization method is accurately applied to the multi-round inference scenario of the tool-type agent, and that the active virtualization strategy can be executed as soon as the tool result enters the visible context of the model for the first time. This solves the problem of repeated transmission and continuous consumption of lexical units after large tool results enter the context.

[0023] Secondly, embodiments of this application provide a context virtualization device for a tool-type intelligent agent, the device comprising: The main processing module is used to construct a copy of the request message, which is used to request the large language model to perform inference on the target task. The main processing module is also used to perform virtualization condition judgment on the text tool result if there is a text tool result in the copy of the request message. The text tool result is the text output generated and returned by the external tool after the agent calls the external tool. The main processing module is also used to store the complete source text of the text tool result to an external storage medium when the text tool result meets the preset virtualization conditions, and to store the location index of the complete source text of the text tool result in the external storage medium; The main processing module is also used to read the corresponding text content from the external storage medium according to the location index, and to perform summary verification on the text content and the complete source text of the text tool results; The virtualization module is used to virtualize the text tool results when the verification passes, and obtain tokens. The tokens are used to indicate the storage location of the complete source text of the text tool results in the external storage medium. The number of tokens is less than the number of tokens in the complete source text of the text tool results. The virtualization module is also used to replace the text tool results in the copy of the request message with tokens to obtain a bounded context request, and send the bounded context request to the large language model so that the large language model can perform inference based on the bounded context request.

[0024] Thirdly, this application also provides an electronic device. The electronic device includes a memory, one or more processors, and a computer program stored in the memory and executable on the processor. The electronic device executes the computer program to implement any of the implementations of the first aspect described above.

[0025] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the method of any of the implementations of the first aspect described above.

[0026] Fifthly, this application also provides a computer program product that, when run on an electronic device, causes the electronic device to execute any of the implementation methods of the first aspect described above.

[0027] Sixthly, this application also provides an intelligent agent, which includes a host program, a text viewing tool, and a virtualization unit. When the intelligent agent runs on an electronic device, it causes the electronic device to execute the method of any of the implementations of the first aspect described above.

[0028] Seventhly, this application also provides a context virtualization system for a tool-type intelligent agent, which includes the intelligent agent and large language model described in the sixth aspect above. When the system runs on an electronic device, it causes the electronic device to execute the method of any implementation of the first aspect above.

[0029] It is understood that the beneficial effects of the second to seventh aspects mentioned above can be found in the relevant descriptions in the first aspect above, and will not be repeated here. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a schematic diagram of the architecture of a context virtualization system for a tool-type intelligent agent provided in an embodiment of this application; Figure 2 This is a flowchart of a context virtualization method for a tool-type intelligent agent provided in an embodiment of this application; Figure 3 This is a schematic diagram of the context virtualization process provided in an embodiment of this application; Figure 4 This is a schematic diagram of automatic virtualization provided in an embodiment of this application; Figure 5 This is a structural block diagram of a context virtualization device for a tool-type intelligent agent provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0032] Figure 1 This is a schematic diagram of the architecture of a context virtualization system for a tool-type intelligent agent provided in an exemplary embodiment of this application. The system 100 includes an intelligent agent 110, an inference service provider 120, and an external tool 130.

[0033] The intelligent agent 110 refers to an AI (Artificial Intelligence) system capable of autonomously calling tools to complete tasks. Internally, it includes a host program 101, a tool calling module 102, a virtualization unit (RecapOS Runtime) 103, a text viewing tool (Access book) 104, and an external storage medium (Book Store) 105. It is worth noting that the intelligent agent 110 in this application can also be called a tool-type intelligent agent.

[0034] The host program 101 receives user messages, constructs request messages and their copies, and communicates with the inference service provider 120. The tool invocation module 102 invokes the external tool 130 and receives the text tool results returned by the external tool 130. The virtualization unit 103 performs virtualization processing on the text tool results, including storing the complete source text to the external storage medium 105 and generating identifiers. The text viewing tool 104 reads the contents of a specified page of the text tool results from the external storage medium 105 on demand. The external storage medium 105 includes a metadata directory and a Blob Store (binary large object store), where the metadata directory stores metadata such as the location index of the text tool results, and the Blob Store stores the complete source text of the text tool results.

[0035] The inference service provider 120 includes a large language model (LLM), which has a context window for receiving request messages sent by the host program 101 and performing inference based on the bounded context in the request messages.

[0036] In terms of connectivity, user messages are input to the host program 101; the host program 101 can communicate with the tool invocation module 102, the virtualization unit 103, and the inference service provider 120; the host program 101 sends a request message to the inference service provider 120, requesting the inference service provider 120 to perform inference; the virtualization unit 103 is connected to the external storage medium 105 and is used to write the complete source text to the external storage medium 105; the tool invocation module 102 is connected to the text viewing tool 104; the text viewing tool 104 is connected to the external storage medium 105 and is used to read the content of a specified page from the external storage medium 105.

[0037] In one embodiment, based on the above system architecture, the processing flow of the context virtualization method for tool-type intelligent agents provided in this application is as follows.

[0038] User messages are input to the host program 101 in agent 110. When the host program 101 needs to request the large language model for inference, it constructs a copy of the request message. The request message is used to request the large language model in the inference service provider 120 to perform inference on the target task. The copy of the request message contains historical session records, and the host program 101 does not modify the original historical records of agent 110 when constructing the copy of the request message.

[0039] If a text tool result exists in the copy of the request message, the virtualization unit 103 performs a virtualization condition judgment on the text tool result. The text tool result is the text output generated and returned by the external tool 130 after the agent 110 calls the external tool 130 through the tool invocation module 102. Specifically, the virtualization unit 103 renders the candidate tokens corresponding to the text tool result and calculates the difference between the number of tokens in the complete source text of the text tool result and the number of tokens in the candidate tokens; when the difference reaches a preset token-saving threshold, the virtualization unit 103 determines that the text tool result meets the preset virtualization conditions.

[0040] If the text tool result meets the preset virtualization conditions, the virtualization unit 103 stores the complete source text of the text tool result in the external storage medium 105, and stores the location index of the complete source text in the external storage medium 105. Specifically, the external storage medium 105 includes a metadata directory and a Blob Store. The virtualization unit 103 stores the location index in the metadata directory and stores the complete source text in the Blob Store.

[0041] The virtualization unit 103 reads the corresponding text content from the external storage medium 105 according to the location index, and performs a summary verification on the text content and the complete source text of the text tool result.

[0042] Specifically, the virtualization unit 103 performs summary generation processing on the text content and the complete source text respectively to obtain a first summary and a second summary; if the first summary and the second summary are consistent, the verification is determined to be successful; if the first summary and the second summary are inconsistent, the verification is determined to be unsuccessful, and the request message to be sent to the large language model is canceled.

[0043] If the verification passes, the virtualization unit 103 generates a token. The token is used to indicate the storage location of the complete source text in the external storage medium 105, and the number of tokens in the token is less than the number of tokens in the complete source text.

[0044] Specifically, the virtualization unit 103 divides the complete source text into multiple logical pages, with the number of lexical units in each logical page not exceeding a preset page lexical unit threshold; selects at least one of the first page, middle page, and last page as a representative page from the multiple logical pages; generates a text tool result identifier based on the representative page; and assembles the text tool result identifier and the representative page to obtain a marker.

[0045] The host program 101 replaces the text tool result in the copy of the request message with a token to obtain a bounded context request. The host program 101 sends the bounded context request to the inference service provider 120 so that the large language model in the inference service provider 120 can perform inference based on the bounded context request.

[0046] When the large language model needs to view a specific page in the results of the text viewing tool, it initiates a text viewing request through the text viewing tool 104. Based on the text viewing request, the intelligent agent 110 reads the content of the specified page from the external storage medium 105 through the text viewing tool 104 and sends the content of the specified page to the large language model, enabling the large language model to continue inference based on the content of the specified page.

[0047] Optionally, when the tool invocation instruction returned by the large language model instructs agent 110 to invoke external tool 130, host program 101 invokes external tool 130 through tool invocation module 102. Tool invocation module 102 receives the text tool result returned by external tool 130 and provides the text tool result to host program 101. Host program 101 reconstructs a copy of the request message, and virtualization unit 103 performs the aforementioned virtualization processing on the text tool result, generating a bounded context request, which is then sent to inference service provider 120, thereby realizing context virtualization in the multi-round inference process.

[0048] Figure 2 This is a flowchart of a context virtualization method for a tool-type intelligent agent provided in an exemplary embodiment of this application. This method is applied to intelligent agents and can also be applied to, for example... Figure 1 The system shown can also be applied to systems containing intelligent agents or such as Figure 1 The terminal device of the system shown includes the following steps.

[0049] Step 201: The agent constructs a copy of the request message.

[0050] The request message is used to request a large language model to perform inference for the target task.

[0051] The copy of the request message includes a copy of the historical session record; wherein, when the agent constructs the copy of the request message, the historical session record remains unchanged.

[0052] Historical conversation records refer to the dialogue content saved by the agent in previous rounds of interaction. This includes messages previously entered by the user, previous inference results from the large language model, tool call parameters generated when the agent previously invoked external tools, and text tool results previously returned by external tools. For example, if a user asks, "Help me run a test script," and the large language model replies, "Please invoke the test tool," and the agent invokes the test tool and returns thousands of lines of logs, all of this content will be saved as historical conversation records. If this is the first request to the large language model for inference, the historical conversation record will be empty or will only contain the current user's message. When constructing a copy of the request message, the agent does not modify the historical conversation record itself; instead, it copies the historical conversation record to the copy, leaving the original historical conversation record unchanged.

[0053] The copy of the request message refers to the process where, when an agent needs to request a large language model for inference, it copies the original request message intended for the large language model. This copy is used because this approach does not directly modify the existing historical conversation records between the agent and the large language model; instead, it only replaces the text tool results present in the copy. The request message can be viewed as a template, containing the dialogue context and text tool results needed for the large language model to complete inference. After processing the copy, the agent uses this processed copy as the actual request message sent to the large language model. In other words, what is ultimately sent to the large language model is a virtualized copy of the request message, not the original request message itself.

[0054] In one embodiment, the copy of the request message is a request message prepared to be sent to the large language model inference service provider. This request message typically includes the text tool results returned after the host program calls an external tool; these text tool results can be very long, such as thousands of lines of log data. In addition, the request message also includes the dialogue history and tool call parameters required for this inference. This solution only replaces the large text tool results in this copy to be sent, without modifying the original historical session records saved by the host program.

[0055] In one embodiment, when the host program constructs a copy of the request message, the process does not modify the host program's historical session records. Furthermore, if the host program previously truncated the text tool result, it needs to provide the complete, trusted source text or a reference to it when constructing the copy. This ensures that the complete content is stored during subsequent virtualization, rather than just the truncated, incomplete version. This process prevents agents from directly truncating excessively long tool outputs after invoking the tool, resulting in the loss of the complete original text.

[0056] In other words, step 201 is performed when a request is needed to perform reasoning from a large language model. There are various situations where a request is needed, such as when a user inputs a new message, an external tool returns a text tool result, or the agent needs to continue with the next step of reasoning.

[0057] In either case, whenever an agent needs to request a large language model for reasoning, it will construct a copy of the request message and perform subsequent virtualization processing on the text tool results in the copy.

[0058] In an optional embodiment, as an example, upon receiving a user message, the agent sends the user message to a large language model so that the large language model can infer the user message. The agent receives the inference result from the large language model; if the inference result instructs the agent to invoke an external tool, the agent invokes the external tool and receives the text tool result generated and returned by the external tool.

[0059] External tools refer to external programs or services that an intelligent agent can invoke to complete specific tasks, such as command-line execution tools, file reading tools, network request tools, and code search tools. Text tool results are the text output returned by the external tool after execution.

[0060] For example, when an agent calls a test execution tool to run a test script, the test execution tool returns thousands of lines of test logs, which are the results of the text tool.

[0061] Since large language models process text using tokens as the basic unit, and a token is the smallest unit of measurement for large language models to process text, which is approximately equal to 0.75 English words or 0.5 Chinese characters, thousands of log lines usually correspond to thousands or even tens of thousands of tokens.

[0062] The number of lexical units that a large language model can process in a single inference iteration is limited; this upper limit is called the context window. If thousands of lines of log data are directly sent to the large language model in the request message, a large number of lexical units will be repeatedly consumed in each subsequent round of inference.

[0063] Step 202: If a text tool result exists in the copy of the request message, then perform a virtualization condition judgment on the text tool result.

[0064] Text tool results are text outputs generated and returned by external tools after the agent invokes them.

[0065] Virtualization condition judgment for text tool results refers to determining whether the text tool results need to be virtualized and processed into tokens with smaller word counts. The basis for virtualization condition judgment for text tool results is a preset virtualization recognition rule.

[0066] The virtualization recognition rules include: (1) small results remain unchanged, and small results are text tool results with insufficient word saving; (2) skill results, the output of the text viewing tool itself, and existing complete tokens are not virtualized; (3) system messages, developer messages, host program control messages, compression requests and compression summaries remain unchanged.

[0067] The skill result is the text output returned by the agent after calling a predefined skill module, and the skill result is not virtualized.

[0068] The output of the text viewing tool itself is the page content that the agent reads from the external storage medium and returns after calling the text viewing tool. The output of the text viewing tool itself is not virtualized to avoid forming a cycle of virtualization, reading, re-virtualization, and re-reading.

[0069] Existing complete identifiers are those that already exist in the copy of the current request message, and whose corresponding complete source text has been stored in external storage media and passed digest verification. Virtualization condition checks are not performed on existing complete identifiers to avoid redundant processing. In other words, these are identifiers that were generated and stored before the copy of the current request message was built.

[0070] System messages are messages generated by intelligent agent systems or inference service providers to control the inference process or set model behavior.

[0071] Developer messages are pre-configured messages by agent developers to deliver specific instructions or context to large language models.

[0072] Host program control messages are messages generated by the host program to control the internal processes of the intelligent agent or coordinate the interactions between modules.

[0073] A compression request is a message initiated by the host program to compress historical session records into a summary to free up context window space.

[0074] A compressed digest is a summary text generated by the host program after it executes a compression request, used to replace the original historical session records.

[0075] In one embodiment, the above process of judging based on virtualization conditions can be summarized as follows: first, filtering out non-target text from the text tool results; non-target text includes at least one of skill results, the output of the text viewing tool itself, existing complete tokens, system messages, developer messages, host program control messages, compression requests, and compression digests.

[0076] After filtering is completed, a judgment is made on cases where the word saving is insufficient to meet the preset threshold value.

[0077] Optionally, the results from the text tool can be virtualized to obtain candidate tokens.

[0078] The process of virtualizing the text tool results to obtain candidate tokens is called rendering, which is to assemble the text according to a preset format. The following is an example of a candidate token.

[0079] <context book_name="pytest-output" pages="1,9,18 / 18"> [Preview of the original text on pages 1, 9, and 18]< / context> This candidate token is used to indicate the large language model that the complete source text of the text tool results has been saved as "pytest-output". "pytest-output" is book_name (text tool result identifier, also known as stable identifier, used to uniquely identify the virtualized text tool results), which has a total of 18 pages, of which 3 pages are currently displayed.

[0080] Candidate tokens are used for evaluation before replacement. They are only used after successful storage and verification. Candidate tokens and officially used tokens can have the same text.

[0081] If the difference between the number of lexics in the complete source text and the number of lexics in the candidate tokens of the text tool result reaches the preset lexicography saving threshold, then the text tool result is determined to meet the virtualization conditions. If the difference between the number of lexics in the complete source text and the number of lexics in the candidate tokens of the text tool result does not reach the preset lexicography saving threshold, then the text tool result is determined to not meet the virtualization conditions, and the result is left unchanged; virtualization of the text tool result is then cancelled.

[0082] In one embodiment, the preset threshold for saving tokens is 1024 tokens, that is, when the number of tokens saved (save tokens = source tokens) is 1024. Virtualization is performed when the number of candidate tokens (tokens) is ≥1024. The preset token-saving threshold can be configured to other values ​​according to actual needs, such as 512, 2048, etc., to adapt to models with different context window sizes. Here, tokens is the plural form of token.

[0083] By calculating the amount of lexical savings and comparing it with a preset threshold, large text tool results that are truly worth virtualizing can be automatically identified. For small results where the amount of lexical savings is less than the threshold, they are left as is, avoiding unnecessary storage and validation processing, thereby effectively reducing lexical consumption while minimizing additional processing overhead. Since this judgment is performed proactively when building a copy of the request message, rather than being passively triggered when the context window is about to overflow, lexical overhead can be saved from the very first request.

[0084] Step 203: If the text tool result meets the preset virtualization conditions, store the complete source text of the text tool result to the external storage medium, and store the location index of the complete source text of the text tool result in the external storage medium.

[0085] In one embodiment, the external storage medium includes a metadata directory and a binary large object store (BlobStore), which is a content-addressable storage. Content-addressable storage refers to a persistent storage method that uses the summary value of the data content as the storage address, and its physical carrier can be at least one of a local hard drive, an embedded database, or a remote object storage. The agent writes the complete source text of the text tool result to the binary large object store and stores the position index of the complete source text in the binary large object store to the metadata directory.

[0086] In one embodiment, prior to storage, the agent derives a stable block identifier (block_id) and a text tool result identifier (book_name) for each virtualized text tool result.

[0087] The text tool result identifier is determined based on the operation target when the agent invokes an external tool: when the operation target is to read a file, the filename's name stem is used as the text tool result identifier; for example, when reading the file / tmp / pieces_top.txt, the text tool result identifier is pieces_top. When the operation target is a network request, the domain name is used as the text tool result identifier; for example, when executing curl google.com, the text tool result identifier is google.com. When the operation target is a search operation, the search query term is used as the text tool result identifier. When there is a naming conflict in the text tool result identifier, an incremental suffix is ​​added to the text tool result identifier, such as -2, -3, etc. Both block identifiers and text tool result identifiers are stored in the metadata directory.

[0088] The metadata directory also stores metadata such as the source path of the complete source text, lexical estimation values, and summary values. This metadata is retained at the agent runtime level and is not exposed to the large language model.

[0089] Step 204: Read the corresponding text content from the external storage medium according to the location index, and perform a summary verification between the text content and the complete source text of the text tool result.

[0090] The complete source texts of the text content and the text tool results are processed separately to generate a first summary corresponding to the text content and a second summary corresponding to the complete source text of the text tool results.

[0091] The first and second summaries are compared; if they match, the verification is considered successful; if they do not match, the verification is considered unsuccessful.

[0092] In one embodiment, the summary generation process is a hash calculation, that is, hashing the complete source text of both the text content and the text tool result to obtain the corresponding first and second summaries. The hash calculation can employ at least one of algorithms such as SHA-256, MD5, and SHA-1.

[0093] For example, the complete source text is "Test execution completed, 3 errors found". The second digest is obtained by hashing it using the SHA-256 algorithm. The text content read from the external storage medium is also "Test execution completed, 3 errors found". The first digest is obtained by hashing it using the SHA-256 algorithm.

[0094] By generating a first and second digest through hash calculation, it is possible to verify, with minimal computational cost, whether the text content read back from the external storage medium is completely consistent with the original complete source text. Because hash values ​​are highly sensitive to content changes, any storage corruption, missing data, or tampering can be accurately detected, thus ensuring the authenticity and reliability of the complete source text stored on the external storage medium and guaranteeing the trustworthiness of information recoverability.

[0095] If the verification passes, proceed to step 205 and the subsequent steps.

[0096] If the validation fails, cancel sending the request message to the large language model.

[0097] Specifically, the verification failure includes at least one of the following situations: (1) the binary large object is missing, that is, the write operation is unsuccessful and the corresponding complete source text cannot be found in the external storage medium; (2) the source text under the stable identifier is tampered with, that is, the text content read back from the external storage medium is inconsistent with the complete source text before writing.

[0098] If any of the above situations occur, the agent will abort the preparation of this request and report an error, without exposing invalid tokens to the large language model or sending the complete source text to the large language model.

[0099] Step 205: If the verification passes, the text tool result is virtualized to obtain the marker.

[0100] The token is used to indicate the storage location of the complete source text of the text tool result in the external storage medium. The number of tokens in the token is less than the number of tokens in the complete source text of the text tool result.

[0101] Optionally, the complete source text of the text tool result can be divided into multiple logical pages, with the number of terms in each logical page not exceeding a preset page term threshold.

[0102] For example, the target is approximately 256 tokens per page (maximum 320 tokens), and page boundaries should preferably be line boundaries or word boundaries.

[0103] Select a representative page from multiple logical pages. The representative page includes at least one of the following: homepage, middle page, and last page.

[0104] The default overview selects a maximum of three deterministic pages, namely one page each for the first page, middle page, and last page, to ensure concise markup.

[0105] Generate a text tool result identifier from the representative page, and then assemble the text tool result identifier and the representative page to obtain a tag. Below is an example of a tag.

[0106] <context book_name="pytest-output" pages="1,9,18 / 18"> [page 1 / 18] Source text visible on homepage [page 9 / 18] Source text is visible on the middle page [page 18 / 18] Source text visible on the last page< / context> Only two attributes are visible to the large language model: book_name (a text tool result identifier, a stable model-readable identifier) ​​and pages (visible representative pages and total number of pages).

[0107] Internal metadata such as block_id, blob path, digest, source path, and token estimation are stored at the runtime level and are not exposed to the large language model.

[0108] Step 206: Replace the text tool result in the copy of the request message with a token to obtain a bounded context request, and send the bounded context request to the large language model so that the large language model can perform inference based on the bounded context request.

[0109] In one optional embodiment, during reasoning based on bounded context requests, if the large language model needs to view a specific page in the text tool results, it determines the specified page to be viewed based on the marker and generates a text viewing request. At this time, the agent receives the text viewing request and returns the specified page to the large language model.

[0110] Optionally, it can receive text viewing requests from the large language model. These requests are used to request the viewing of a specific page from the text tool results. The text viewing request is generated by the large language model when it determines the specific page to be viewed based on the tags.

[0111] The specified page is read from external storage media based on the text viewing request, and the content of the specified page is sent to the large language model.

[0112] The intelligent agent includes a text viewing tool, which is used to retrieve the results of the text tool from external storage media.

[0113] If a text tool result that has passed digest verification exists in the external storage medium, the agent sets the text viewing tool to a model-visible state. Model-visible state refers to the state that allows the large language model to initiate text viewing requests. That is, the text viewing tool is not exposed to the large language model before a text tool result that has passed digest verification exists; once a text tool result that has passed digest verification exists, the text viewing tool is set to a model-visible state, and this state remains unchanged in subsequent requests.

[0114] Based on the text viewing request, the agent determines the target location of the specified page in the external storage medium.

[0115] The text viewer is invoked to read the specified page from the target location on the external storage medium and the content of the specified page is sent to the large language model.

[0116] In one optional embodiment, the text viewing tool includes the following five operations.

[0117] 1. The list_shelves operation is used to list the shelves to which the results of the text tool belong, categorized by source.

[0118] 2. The List_Books operation lists all stored text tool results and can optionally filter within a single shelf.

[0119] 3. Search page operations (search_pages) are used to search for all pages of text tool results based on the query terms, and return the hit pages and context fragments.

[0120] 4. Read Pages operation (read_pages) is used to read known pages of the results of a specified text tool, supporting single page, range and multi-page selection (such as 1, 9, 18 or 1-3, 8).

[0121] 5. The compute_text operation is used to perform bounded deterministic text computation operations on the results of a single text tool.

[0122] The schema of the text viewing tool has the following lifecycle: the schema of the text viewing tool is omitted and not exposed to the large language model until a text tool result that has passed digest validation is not available in the external storage medium; once the first text tool result that has passed digest validation is created or loaded, the complete schema of the text viewing tool is exposed and remains unchanged in subsequent requests.

[0123] In an optional embodiment, all five operations of the text viewing tool are available simultaneously, with access contracts enforced through operation-level input validation; wherein, the page reading operation returns the complete selection range, without being limited by a fixed number of pages or an output size cap; the other operations maintain their respective result quantity or computational limitations.

[0124] In an optional embodiment, the output of the text viewing tool remains native and is not virtualized again into tokens to avoid circular processing.

[0125] Through the five operations of the aforementioned text viewing tools, the large language model can enumerate, search, read, and compute the stored text tool results using a unified and stable interface, covering all scenarios of on-demand access. Through pattern lifecycle management, it ensures that tools are only exposed to the large language model when recoverable content exists, avoiding interference caused by invalid exposure. Through operation-level verification, it guarantees that the access contract is enforced, and page reading operations are not limited by a fixed number of pages or output size, ensuring the complete return of the required content. By making the output non-virtualizable, it prevents the cycle of "virtualization, reading, re-virtualization, and re-reading," improving system stability and processing efficiency.

[0126] In an optional embodiment, when the large language model reads a specified page through a text viewing tool, if there is a page selector format error, the reading request exceeds the limit, or an unsupported operation type, the agent returns an error message instead of returning the erroneous page content.

[0127] The storage directory of the external storage medium is located outside the workspace of the agent task and is not part of the task evidence and file system security boundary, thus preventing accidental operation or deletion by the agent task.

[0128] In an optional embodiment, the agent may also have a compression isolation mechanism, specifically: the host program in the agent originally has a context compression function, used to compress historical messages into summaries when the session history is too long; the virtualization processing of this application does not interfere with this native compression function. When the host program initiates a context compression request, the compression request is processed through a pass-through branch. A pass-through branch means that the compression request does not go through the virtualization process of this application and is processed directly in the original way of the host program. In the pass-through branch, the virtualization unit (the program used to perform context virtualization processing) is not called, the pattern of the text viewing tool is not injected into the large language model, the markers are not rewritten, and the data is not written to external storage media. The host program constructs, trims, sends, and stores the compression request in its native way. After the compressed summary generated by the compression request becomes part of the active history of the host program, subsequent ordinary inference requests re-execute the virtualization processing of this application.

[0129] In an optional embodiment, the agent may also have a storage and fault handling mechanism, specifically manifested as follows: writing the complete source text of the text tool result into a content-addressed binary large object storage. Content addressing refers to calculating a digest value based on the text content and using this digest value as the storage address; the binary large object storage is a portion of the external storage medium specifically for storing the complete source text. After writing, the write-back content is read from the binary large object storage, and digest calculations are performed on the write-back content and the complete source text before writing, comparing the two digest values. If they match, the verification passes; if they do not match, the verification fails. The following situations all result in explicit failure: missing binary large object, source text under a stable identifier being tampered with, incorrect page selector format, read request exceeding limits, or unsupported operation type. For storage or verification errors, the current request preparation is aborted and the error is reported; invalid identifiers are not exposed to the large language model, nor is the complete source text sent to the large language model. For page number format errors, out-of-bounds errors, or unsupported operation types, an explicit error is returned for the caller to correct. Furthermore, the storage directory of the external storage medium is located outside the agent task workspace and does not belong to the task evidence and file system security boundary.

[0130] In an optional embodiment, the agent may also have a stress monitoring mechanism, specifically: before the host program initiates a normal inference call to the inference service provider, the virtualization unit monitors two independent stress indicators; wherein, the first stress indicator is the estimated number of tokens in the normal request after virtualization processing (that is, after the virtualization processing of this application, the normal request that the host program is preparing to send to the large language model, approximately how many tokens it occupies), and the second stress indicator is the estimated number of tokens in the compressed input constructed by the host program based on the active native history (that is, the estimated number of tokens in the compressed request constructed by the host program based on the currently active native history without the processing of this application); when either estimated value reaches the available input threshold of the request, the host program initiates native compression; the virtualization unit does not add additional compression limit parameters.

[0131] Through a compression isolation mechanism, the virtualization processing and the host program's native compression strategy operate on two independent levels. Compression requests do not trigger virtualization, inject text viewing tools, rewrite tokens, or write to external storage media, thus allowing the two mechanisms to coexist without conflict. Through a stress monitoring mechanism, the host program can initiate native compression when needed, and this application does not add additional compression limiting parameters, ensuring the flexibility of the compression strategy. Through storage and fault handling mechanisms, any storage or verification error will abort request preparation and report the error, without exposing invalid tokens to the large language model; page number format errors, out-of-bounds errors, or unsupported operation types will return explicit errors, facilitating correction by the caller; the storage directory is placed outside the task workspace, preventing accidental operation or deletion of stored data by the task, ensuring the security and independence of the stored data.

[0132] Indicative, such as Figure 3 As shown, Figure 3 This is a schematic diagram of the context virtualization process.

[0133] The process includes: the large language model 301 instructs the agent to initiate an external tool 130 call; after the external tool 130 is called, it returns a complete text tool result 303; the text tool result 303 is input into the virtualization unit (RecapOSRuntime) 103; the virtualization unit 103 stores the complete source text of the text tool result 303 into the external storage medium (BookStore) 105, and replaces the text tool result 303 with a marker, which contains the text tool result identifier (book_name) and a representative page, generating a bounded context 306.

[0134] Bounded context 306 is sent to the large language model 301 so that the large language model 301 can perform inference based on the bounded context.

[0135] When the large language model 301 needs to view a specific page in the text tool result 303, the large language model 301 sends a request to the agent. The agent's host program reads the content of the specified page from the external storage medium 105 through the text viewing tool (access_book) 104 and sends the content to the large language model 301.

[0136] Therefore, the complete text tool results are no longer repeatedly transmitted with each inference request; the model only reads the specified page as needed through the text viewing tool.

[0137] Indicative, such as Figure 4 As shown, Figure 4 It is a flowchart of automatic virtualization.

[0138] This virtualization process, also known as the materialization process, is essentially the process of replacing the text tool results with markers and storing the source text content of the text tool results on an external storage medium.

[0139] The process includes the following steps S401 to S409.

[0140] S401, a copy of the host program build request message.

[0141] S402, Determine whether the copy of the request message contains the text tool result.

[0142] S403, keep the request message unchanged if the text tool result is not included.

[0143] S404, when including text tool results, determine whether the number of lexical units saved when performing virtualization processing on the text tool results reaches a preset threshold.

[0144] S405, under virtualization conditions, stores the complete source text of the text tool results to an external storage medium and derives stable block_id and book_name for it.

[0145] S406, In cases where virtualization conditions are not met, keep the request message unchanged.

[0146] S407, perform digest verification and determine whether to perform virtualization processing based on the digest verification result.

[0147] S408, if the verification passes, generates a marker that includes a representative page and a text tool result identifier.

[0148] S409: If the verification fails, cancel sending the request message to the large language model.

[0149] In summary, the context virtualization method for tool-type intelligent agents provided in this application, when there is a need to request a large language model for inference, constructs a copy of the request message. The copy of the request message is identical to the content of the original request message to be sent, and does not directly modify the original request message. Therefore, the original request message and the agent's original historical records remain unchanged. Furthermore, if the copy contains text tool results, virtualization conditions are determined. This automatically identifies large tool results worthy of virtualization, avoiding indiscriminate processing of all tool results and preventing meaningless storage and verification of small tool results, which saves few lexical units but increases additional overhead. When the virtualization conditions are met, the complete source text is stored on an external storage medium and the location index is recorded. Therefore, the original information is completely preserved, unlike compression or truncation which would result in loss of the original text. The lost text is retrieved, and the location index provides a basis for subsequent retrieval. The stored content is then read back based on the location index and its summary is verified against the original complete source text. This ensures that the stored content is completely consistent with the original text, preventing storage damage or tampering and guaranteeing reliable recoverability. After successful verification, the text tool result is virtualized to generate a marker. The marker indicates the storage location and has fewer lexical units than the complete source text, thus replacing large sections of the original text in the context and significantly reducing the number of lexical units occupied by the context. Finally, the marker replaces the text tool result in the copy, obtaining a bounded context request, which is sent to the large language model for inference. Therefore, the large language model sees a reduced bounded context during each inference, no longer carrying the complete large result. Lexical units are saved from the first request, solving the problem of lexical waste caused by repeatedly transmitting large results and the passive response of wasting resources before cleaning up in related technologies.

[0150] Corresponding to the context virtualization method for tool-type intelligent agents in the above embodiments, Figure 5 A structural block diagram of the apparatus provided in an embodiment of this application is shown. For ease of explanation, only the parts relevant to the embodiment of this application are shown. The apparatus 500 includes...

[0151] The main processing module 510 is used to construct a copy of the request message, which is used to request the large language model to perform inference on the target task. The main processing module 510 is also used to perform virtualization condition judgment on the text tool result if there is a text tool result in the copy of the request message. The text tool result is the text output generated and returned by the external tool after the agent calls the external tool. The main processing module 510 is also used to store the complete source text of the text tool result to an external storage medium when the text tool result meets the preset virtualization conditions, and to store the location index of the complete source text of the text tool result in the external storage medium; The main processing module 510 is also used to read the corresponding text content from the external storage medium according to the location index, and to perform summary verification on the text content and the complete source text of the text tool result; The virtualization module 520 is used to virtualize the text tool result when the verification passes, and obtain a token. The token is used to indicate the storage location of the complete source text of the text tool result in the external storage medium. The number of tokens in the token is less than the number of tokens in the complete source text of the text tool result. The virtualization module 520 is also used to replace the text tool result in the copy of the request message with a token to obtain a bounded context request, and send the bounded context request to the large language model so that the large language model can perform inference based on the bounded context request.

[0152] It should be noted that the information interaction and execution process between the above-mentioned devices / modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0153] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0154] To implement the above embodiments, this application also proposes an electronic device. Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application.

[0155] like Figure 6 As shown, the above-mentioned electronic device 600 includes: The system includes a memory 610 and at least one processor 620, and a bus 630 connecting different components (including the memory 610 and the processor 620). The memory 610 stores a computer program, and when the processor 620 executes the program, it implements the context virtualization method of the tool-type intelligent agent according to the embodiments of this application.

[0156] Bus 630 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MCA) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0157] Electronic device 600 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by electronic device 600, including volatile and non-volatile media, removable and non-removable media.

[0158] Memory 610 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 640 and / or cache memory 650. Electronic device 600 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 660 can be used to read and write non-removable, non-volatile magnetic media (… Figure 6 Not shown; usually referred to as a "hard drive"). Although Figure 6 As not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 630 via one or more data media interfaces. Memory 610 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0159] A program / utility 680 having a set (at least one) of program modules 670 may be stored in, for example, memory 610. Such program modules 670 include—but are not limited to—an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 670 typically perform the functions and / or methods described in the embodiments of this application.

[0160] Electronic device 600 can also communicate with one or more external devices 690 (e.g., keyboard, pointing device, display 691, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 695. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 693. As shown, network adapter 693 communicates with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0161] The processor 620 executes various functional applications and data processing by running programs stored in the memory 610.

[0162] It should be noted that the implementation process and technical principles of the electronic device in this embodiment are explained in the foregoing description of the context virtualization method of the tool-type intelligent agent in the embodiments of this application, and will not be repeated here.

[0163] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the above-described method embodiments.

[0164] This application provides a computer program product that, when run on an electronic device, enables the electronic device to implement the steps described in the various method embodiments above.

[0165] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code to an electronic device, a recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some regions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0166] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0167] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0168] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0169] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0170] In the foregoing description, specific details such as particular system structures and techniques have been set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application can also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted to avoid unnecessary detail that could obscure the description of this application.

[0171] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0172] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0173] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0174] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0175] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0176] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for context virtualization of a tool-type intelligent agent, characterized in that, The method includes: The agent constructs a copy of the request message, which is used to request the large language model to perform inference on the target task; If a text tool result exists in the copy of the request message, then a virtualization condition judgment is performed on the text tool result. The text tool result is the text output generated and returned by the external tool after the agent calls the external tool. The judgment basis for the virtualization condition judgment on the text tool result is a preset virtualization recognition rule. The virtualization recognition rule includes: small results remain unchanged, and the small results are text tool results whose word saving is less than the threshold value. If the text tool result meets the preset virtualization conditions, the complete source text of the text tool result is stored in an external storage medium, and the location index of the complete source text of the text tool result in the external storage medium is also stored. The corresponding text content is read from the external storage medium according to the location index, and the text content is compared with the complete source text of the text tool result for summary verification. If the verification passes, the text tool result is virtualized to obtain a token. The token is used to indicate the storage location of the complete source text of the text tool result in the external storage medium. The number of tokens in the token is less than the number of tokens in the complete source text of the text tool result. The text tool result in the copy of the request message is replaced with the token to obtain a bounded context request, and the bounded context request is sent to the large language model so that the large language model can perform inference based on the bounded context request.

2. The method according to claim 1, characterized in that, The method further includes: The system receives a text viewing request from the large language model, the text viewing request being used to request viewing a specified page in the text tool results; the text viewing request is generated by the large language model when it determines the specified page to be viewed based on the marker; The specified page is read from the external storage medium according to the text viewing request, and the content of the specified page is sent to the large language model.

3. The method according to claim 2, characterized in that, The intelligent agent includes a text viewing tool, which is used to obtain text tool results from the external storage medium; the method further includes: If the text tool results that have passed the digest verification exist in the external storage medium, the agent sets the text viewing tool to a model-visible state; the model-visible state refers to a state that allows the large language model to initiate the text viewing request. The step of reading the specified page from the external storage medium according to the text viewing request and sending the content of the specified page to the large language model includes: The intelligent agent determines the target location of the specified page in the external storage medium based on the text viewing request; The text viewing tool is invoked to read the specified page from the target location on the external storage medium, and the content of the specified page is sent to the large language model.

4. The method according to any one of claims 1 to 3, characterized in that, If a text tool result exists in the copy of the request message, then a virtualization condition judgment is performed on the text tool result, including: The results of the text tool are virtualized to obtain candidate tokens; If the difference between the number of lexical units in the complete source text of the text tool result and the number of lexical units in the candidate token reaches a preset lexical unit saving threshold, then the text tool result is determined to meet the virtualization condition.

5. The method according to any one of claims 1 to 3, characterized in that, The step of performing a summary verification on the complete source text of the text content and the text tool result includes: Summarize the text content and the complete source text of the text tool result respectively to obtain a first summary corresponding to the text content and a second summary corresponding to the complete source text of the text tool result. The first summary and the second summary are compared; If the first digest and the second digest are consistent, then the verification is successful; If the first digest and the second digest are inconsistent, the verification is determined to have failed.

6. The method according to any one of claims 1 to 3, characterized in that, If the verification passes, the text tool result is virtualized to obtain a token, including: The complete source text of the text tool result is divided into multiple logical pages, and the number of lexical units in each logical page does not exceed a preset page lexical unit threshold. Select a representative page from the plurality of logical pages, wherein the representative page includes at least one of the following: home page, middle page, and last page; A text tool result identifier is generated based on the representative page, and the text tool result identifier and the representative page are assembled to obtain the marker.

7. The method according to any one of claims 1 to 3, characterized in that, Before the agent constructs a copy of the request message, the method further includes: Upon receiving a user message, the agent sends the user message to the large language model so that the large language model can reason about the user message; Receive the inference results fed back by the large language model; If the inference result instructs the agent to invoke the external tool, the agent invokes the external tool and receives the text tool result generated and returned by the external tool.

8. An electronic device comprising a memory, one or more processors, and a computer program stored in the memory and executable on the one or more processors, characterized in that, When the one or more processors execute the computer program, the electronic device performs the method as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images