Agent context processing method and device, electronic device, and computer medium

Abstract

The present disclosure provides an agent context processing method and device, system, electronic equipment and medium, relating to the technical field of artificial intelligence. The specific scheme is: the received user input message and the reply information generated by the large language model in the last round are added to the history message array as new nodes; a constraint entity in the user input message is extracted through a new asynchronous processing task, and is updated to a long-term entity memory dictionary; whether the history message array meets a preset threshold condition is detected; in response to detecting that the history message array meets the threshold condition, the old message segment in the history message array is converted into an abstract node, and the abstract node and the recent message segment are reorganized into a target message array; the long-term entity memory dictionary is serialized into a structured text, the structured text and a top-level message node encapsulated by a static system instruction are linearly spliced with the target message array, a request body array of the current round is generated and sent to the large language model.

Description

Technical Field

[0001] This disclosure belongs to the field of artificial intelligence technology, specifically involving educational technology, large-scale language models and other technical fields, and in particular, a method and apparatus for intelligent agent context processing, electronic devices and computer-readable storage media. Background Technology

[0002] With the popularization of Large Language Models (LLMs), multi-turn dialogue AI agents built on LLMs have been widely used in complex task processing, such as itinerary planning, legal consultation, and code generation.

[0003] In the implementation of multi-turn dialogue computer systems, the system must maintain a single-dimensional array of time-series messages and, each time a user makes a request, serialize the entire array of time-series messages and transmit it completely to the inference interface of the large language model in order to maintain the "context" of the dialogue.

[0004] In real-world industrial-grade long-range communication scenarios, this conventional mechanism, which relies on "transmitting the entire message array," has the following inherent underlying technical flaws:

[0005] Some large models have absolute input context window limitations. As the number of interaction rounds increases, the message history array grows linearly, eventually causing the request size to exceed the model's physical limitations, leading to system memory overflow or API (Application Programming Interface) denial-of-service. In addition, cloud-based LLM interfaces are usually charged based on the number of input tokens. Repeated transmission of redundant historical data can significantly slow down inference latency, and the operating cost will increase exponentially.

[0006] When dealing with extremely long contexts, LLM's attention mechanism exhibits a U-shaped performance curve (i.e., high recall at the beginning and end of the text, but a significant drop in the middle). In complex planning tasks, the core hard constraints set by the user in the early stages (such as budget limits or specific requirements) are often pushed to the middle of the context as the dialogue progresses, making the agent prone to "amnesia" and triggering severe factual illusions.

[0007] To alleviate the above problems, existing technologies typically employ either the sliding window truncation method (directly discarding data from N rounds ago) or the global recursive summarization method (compressing old dialogues into summaries). The former causes the agent to completely forget the facts confirmed earlier; the latter, being a lossy compression, erases the recent context in natural language. When the user's latest input contains pronouns (such as "just do it the second solution you just mentioned"), the large model will lose its "pronoun resolution" ability, leading to logical breaks.

[0008] In summary, existing intelligent agent architectures are trapped in technical bottlenecks when processing long sequence tasks, namely "retaining the full amount leads to overflow and amnesia" and "brutal compression leads to failure of reference resolution". A new context processing architecture is urgently needed to break through this underlying limitation. Summary of the Invention

[0009] This disclosure provides an intelligent agent context processing method and apparatus, an electronic device, and a computer-readable storage medium.

[0010] According to the first aspect, a method for intelligent agent context processing is provided, comprising: in response to receiving a user input message, appending the user input message and the response information generated by the large language model in the previous round as new nodes to the historical message array; creating an asynchronous processing task outside the main dialogue flow, extracting constraint entities from the user input message through the asynchronous processing task, and updating the constraint entities to the long-term entity memory dictionary, the long-term entity memory dictionary being independent of the historical message array; detecting whether the historical message array meets a preset threshold condition, the threshold condition including a dual-dimensional monitoring condition of message quantity threshold and word segmentation unit load threshold; in response to detecting that the historical message array meets the threshold condition, dividing the historical message array into old message fragments and recent message fragments, converting the old message fragments into summary nodes by calling a summary extraction algorithm or summary generation model, and recombining the summary nodes and recent message fragments into a target message array; serializing the long-term entity memory dictionary into structured text, encapsulating the structured text and static system instructions into a top-level message node, linearly concatenating the top-level message node and the target message array according to a preset priority order, generating and sending the request body array of the current round to the large language model, so that the large language model generates the response information of the current round.

[0011] According to a second aspect, an intelligent agent context processing device is provided, comprising: an append unit configured to, in response to receiving a user input message, append the user input message and the response information generated by a large language model in the previous round as new nodes to a historical message array; an extraction unit configured to create an asynchronous processing task outside the main dialogue flow, extract constraint entities from the user input message through the asynchronous processing task, and update the constraint entities to a long-term entity memory dictionary, the long-term entity memory dictionary being independent of the historical message array; and a detection unit configured to detect whether the historical message array meets preset threshold conditions, the threshold conditions including dual-dimensional monitoring of message quantity threshold and word segmentation unit load threshold. The conditions include: a segmentation unit configured to, in response to detecting that the historical message array meets a threshold condition, segment the historical message array into old message fragments and recent message fragments, convert the old message fragments into summary nodes by calling a summary extraction algorithm or summary generation model, and reassemble the summary nodes and recent message fragments into a target message array; and a sending unit configured to serialize the long-term entity memory dictionary into structured text, encapsulate the structured text and static system instructions into a top-level message node, linearly concatenate the top-level message node and the target message array according to a preset priority order, generate and send the current round's request body array to the large language model, so that the large language model generates the current round's response information.

[0012] According to a third aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform a method as described in any implementation of the first aspect.

[0013] According to a fourth aspect, a computer-readable storage medium is provided having computer instructions stored thereon for causing a computer to perform the method as described in any implementation of the first aspect.

[0014] The intelligent agent context processing method and apparatus provided in the embodiments of this disclosure firstly, in response to receiving a user input message, appends the user input message and the response information generated by the large language model in the previous round as new nodes to the historical message array; secondly, an asynchronous processing task is created outside the main dialogue flow, and the constraint entities in the user input message are extracted through the asynchronous processing task, and the constraint entities are updated to the long-term entity memory dictionary, which is independent of the historical message array; thirdly, it detects whether the historical message array meets the preset threshold conditions, the threshold conditions including a dual-dimensional monitoring condition of message quantity threshold and word segmentation unit load threshold; then... Subsequently, in response to the detection that the historical message array meets the threshold condition, the historical message array is segmented into old message fragments and recent message fragments. The old message fragments are transformed into summary nodes by calling a summary extraction algorithm or summary generation model, and the summary nodes and recent message fragments are recombined into the target message array. Finally, the long-term entity memory dictionary is serialized into structured text, and the structured text and static system instructions are encapsulated into top-level message nodes. The top-level message nodes and the target message array are linearly concatenated according to a preset priority order to generate and send the current round's request body array to the large language model, so that the large language model can generate the current round's response information. Thus, by creating an asynchronous processing task outside the main dialogue flow, it is ensured that the mandatory constraints determined by the user in the early stage always exist at the top of the large model's attention calculation, which is immune to the long text attention dilution effect and ensures the absolute accuracy of business execution. A two-dimensional threshold detection based on the number of messages and the load of the word segmentation unit is introduced, and hard slicing and dimensionality reduction transformation are performed when the limit is exceeded. This mechanism forcibly converges the input data volume of the large language model from linear growth with each round to a constant-level range fluctuation. While completely eliminating the risk of memory overflow, it can save a lot of word element interface call costs and significantly reduce the latency of the first character output. When performing message dimensionality reduction compression, recent message fragments are preserved, ensuring that the large model can receive recent native dialogue text without any loss. This allows for accurate parsing of relative referential instructions in user input that strongly depend on recent context, such as "previous option" and "second option," avoiding the logical breaks caused by traditional indiscriminate compression. Entities adopt a non-blocking asynchronous processing task mechanism, and the monitoring of the word segmentation unit load adopts a pre-computation and register accumulation mode. This avoids the main thread CPU (Central Processing Unit) blocking caused by performing full word segmentation on a very large string before sending a request in traditional solutions, greatly improving the response speed and concurrent processing capability of the gateway layer.

[0015] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0016] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0017] Figure 1 This is a flowchart of an embodiment of the intelligent agent context processing method according to the present disclosure;

[0018] Figure 2 This is a sequence diagram of data handling and asynchronous calls in the intelligent agent context processing method disclosed herein;

[0019] Figure 3 This is an overall execution timing diagram of the intelligent agent context processing method disclosed in this paper;

[0020] Figure 4 This is a schematic diagram of a structure of an embodiment of the intelligent agent context processing device according to the present disclosure;

[0021] Figure 5 This is a block diagram of an electronic device used to implement the intelligent agent context processing method of the embodiments of this disclosure. Detailed Implementation

[0022] Unless otherwise expressly stated, throughout the specification and claims, the term "comprising" or its variations such as "including" or "comprises" shall be understood to include the stated elements or components without excluding other elements or other components.

[0023] To overcome the problems in the prior art, this disclosure proposes an agent context processing method. Figure 1 A flow 100 is shown as an embodiment of an agent context processing method according to the present disclosure, the agent context processing method including the following steps:

[0024] Step 101: In response to receiving a user input message, the user input message and the response information generated by the large language model in the previous round are appended as new nodes to the historical message array.

[0025] In this embodiment, in a multi-turn dialogue scenario, user input messages are typically natural language text sent by the client via HTTP or WebSocket protocols. The agent context processing method, running on top of this, maintains a historical message array in memory. This array consists of multiple structured message objects (typically containing role and content attributes) arranged in chronological order. When a new message is received, the agent encapsulates it into a standard object and appends it to the end of the array.

[0026] In this embodiment, the response information generated in the previous round by the large language model is the information obtained by inputting the request body array obtained after processing the user input message in the previous round with the intelligent agent context processing method disclosed herein into the large language model.

[0027] In this embodiment, after receiving a user input message in the current round, the execution entity running on the agent context processing method parses the user input message into a standardized message object (including message content, timestamp, role identifier, and optional session / tool ​​call metadata), and adds it to the end of the historical message array as a new node along with the response information generated by the large language model in the previous round. This allows for incremental updates of the dialogue context without rewriting existing historical nodes, ensuring that subsequent large language model inference or tool calls can build context and maintain state based on the latest complete message sequence.

[0028] Step 102: Create an asynchronous processing task outside the main dialogue flow, extract the constraint entities from the user input message through the asynchronous processing task, and update the constraint entities to the long-term entity memory dictionary.

[0029] In this embodiment, the asynchronous processing task refers to a non-blocking execution unit derived from the main thread of the computer system that processes backbone network requests (such as requesting a response from a large language model). The operation of the asynchronous processing task does not occupy the waiting time of the backbone dialogue, realizing information processing with zero additional latency and improving the high concurrency throughput of the system.

[0030] In this embodiment, the constraint entities include key parameters that have a decisive limiting effect on the current business task (e.g., numerical threshold constraints, conditional logical constraints, such as the budget ceiling in itinerary planning). The divergent natural language is transformed into structured variables that can be precisely computed by a computer to eliminate the illusion of fact in large language models.

[0031] In this embodiment, the long-term entity memory dictionary is independent of the historical message array, and is physically represented as a hash table (Hash Map) structure allocated in the computer's running memory (such as RAM), independent of the linear message queue. The addressing time complexity of the long-term entity memory dictionary is constant O(1). The data in this dictionary has the highest survival priority and will never be discarded in conventional compression algorithms, thus physically immunizing against the attention dilution effect when large language models process long texts.

[0032] In this embodiment, the data stream branches the moment a user input message is received in the main thread processing logic of the computer. The main process continues to queue for inference of the main model; the bypass process puts the message into the extraction pipeline.

[0033] In this embodiment, after receiving user input messages, the gateway server, acting as a producer, encapsulates the user input messages into asynchronous events and pushes them to a designated topic in a distributed message queue. The main process then continues to execute without waiting for the extraction results. Step 102 includes: using an independently deployed entity extraction microservice as a consumer to asynchronously subscribe to and pull user input messages from the distributed message queue to obtain a preliminary entity set. The entity extraction microservice does not rely on an auxiliary large language model but instead calls a pre-trained named entity recognition sequence labeling model (such as a model based on the BERT-CRF architecture) to perform word segmentation and entity boundary prediction on the user input messages, outputting a preliminary entity set. Subsequently, the preliminary entity set is semantically aligned and disambiguated with a pre-set domain knowledge graph to obtain standardized constrained entities, which are then updated to the long-term entity memory dictionary via remote calls.

[0034] Optionally, after receiving user input messages, the main dialogue process corresponding to the main dialogue flow writes the memory pointer or text copy of the user input message into the shared memory area through the inter-process communication mechanism at the operating system level. A daemon process running independently in the background asynchronously reads the message from the shared memory area for processing through polling or event notification mechanisms, thereby achieving physical isolation and non-blocking computation from the main dialogue flow. Step 102 includes: using a daemon process running independently in the background to asynchronously read user input messages from the shared memory area through polling or event notification mechanisms; using the daemon process to quickly match user input messages using a trie-based multi-pattern matching algorithm to locate all business keywords; for complex statements containing quantity, negation, or conditional constraints, calling a syntactic dependency parsing algorithm to construct a syntactic modification tree between words in the sentence; and, according to a preset syntactic rule template, pruning constraint entities containing restrictive conditions from the modification tree and writing them into a long-term entity memory dictionary.

[0035] In this embodiment, updating the constraint entity to the long-term entity memory dictionary includes: determining whether the key corresponding to the constraint entity already exists in the long-term entity memory dictionary; in response to the key already existing, further identifying the data type of the constraint entity's value; in response to the data type being a scalar value or a string, forcibly overwriting the old value corresponding to the key with the new value of the constraint entity; in response to the data type being a list or set, extracting the list of old values ​​corresponding to the key, merging the new value of the constraint entity with the list of old values ​​to remove duplicates, and updating the long-term entity memory dictionary with the merged and deduplicated new list as the target value.

[0036] Optionally, updating the constraint entity to the long-term entity memory dictionary includes: before performing the update operation, reading the historical old value corresponding to the key of the constraint entity in the long-term entity memory dictionary; calculating the difference offset between the new value of the constraint entity and the historical old value, and determining whether the difference offset exceeds a preset logical conflict threshold; in response to not exceeding the logical conflict threshold, directly updating the new value to the long-term entity memory dictionary; in response to exceeding the logical conflict threshold, pausing the direct update operation, and encapsulating the new value and the historical old value together as an arbitration request, sending it to the auxiliary language model for intent fusion or conflict determination; receiving the arbitration result output by the auxiliary language model, and performing an update or retention operation on the long-term entity memory dictionary based on the arbitration result.

[0037] Step 103: Check whether the historical message array meets the preset threshold conditions.

[0038] In this embodiment, the threshold conditions include two-dimensional monitoring conditions: message quantity threshold and word segmentation unit load threshold.

[0039] In this embodiment, the word segmentation unit is the basic data unit for text processing at the bottom layer of the large language model. Before natural language is input into the model, it must be segmented into a sequence of tokens by a word segmenter. The word segmentation unit load refers to the total number of tokens after the current text is converted. The size of the word segmentation load directly determines whether the underlying model will experience physical window overflow (OOM) and the computational cost of the call.

[0040] In this embodiment, before inputting the historical message array into the large language model, the execution entity first performs threshold detection on the historical message array: on the one hand, it counts the number of historical message entries in the array and compares it with a preset message quantity threshold; on the other hand, it performs word segmentation on the message text in the historical message array to obtain a word segmentation unit sequence and calculates its total word segmentation unit load (e.g., the context occupancy represented by the number of word segmentation units), and then compares it with a preset word segmentation unit load threshold; when the number of messages reaches / exceeds the message quantity threshold and / or the word segmentation unit load reaches / exceeds the word segmentation unit load threshold, it is determined that the historical message array does not meet the threshold conditions, and corresponding control strategies are triggered (e.g., pruning, summarizing, or segmenting the historical messages) to control the scale of the subsequent input context within an acceptable range.

[0041] Optionally, the threshold condition can also be a monitoring condition with only the segmentation unit load threshold dimension. Step 103 above includes: extracting the content strings of all message objects in the historical message array; calculating the total global character length of the content strings; obtaining the preset character-segmentation unit conversion coefficient according to the language type of the current dialogue (for example, the Chinese character coefficient is set to 1.5 and the English character coefficient is set to 0.3); multiplying the total global character length by the conversion coefficient to calculate the estimated segmentation load; and comparing the estimated segmentation load with the segmentation unit load threshold to determine whether the threshold condition for triggering compression is met.

[0042] Optionally, the threshold condition can also be a monitoring condition that includes a system memory safety threshold. Step 103 above includes: serializing the current historical message array in memory into a standard network transmission format string (such as a JSON string); calling the underlying memory measurement interface of the operating system to obtain the physical byte size occupied by the network transmission format string; comparing the physical byte size with a preset system memory safety threshold (such as 100 KB); and determining that the historical message array meets the threshold condition in response to the physical byte size exceeding the system memory safety threshold.

[0043] Step 104: In response to the detection that the historical message array meets the threshold condition, the historical message array is divided into old message fragments and recent message fragments. The old message fragments are converted into summary nodes by calling the summary extraction algorithm or summary generation model, and the summary nodes and recent message fragments are recombined into the target message array.

[0044] In this embodiment, the summary node is a string that extracts and compresses a high-dimensional, complex array structure consisting of multiple alternating dialogue roles into a single-dimensional scalar through a summary extraction algorithm or a small language model.

[0045] In this embodiment, the summary node role attribute is usually forced to be set to the system level in order to give it a higher background knowledge weight in subsequent model inference.

[0046] like Figure 2 As shown, the threshold monitor is used to detect whether the historical message array meets the threshold condition. If the historical message array meets the threshold condition, step 1 is performed: a CompressSignal is sent to the slice executor; the slice executor performs step 2: reads the historical message array from memory; the slice executor performs step 3: performs slice calculation; and step 4: divides the historical message array into old message fragments and recent message fragments; the slice executor then performs step 5: sends the old message fragments to the serialization engine. The target message array is then obtained through an auxiliary dimensionality reduction model.

[0047] In this embodiment, when the system detects that the length of the historical message array, the cumulative number of lexical units, or the time span meets the preset threshold conditions, the system triggers a message compression process: First, the historical message array is divided into old message segments and recent message segments in chronological order; then, the old message segments are subjected to dimensionality reduction transformation, and the key information (such as user goals, confirmed facts, constraints, and completed conclusions) is extracted into structured summary nodes, and the corresponding time range, reference index, and integrity verification information are attached to the summary nodes; finally, the summary nodes are recombined with the recent message segments as a high-density "summary message" in the original order to form the target message array, thereby significantly reducing context occupancy and improving the efficiency of subsequent reasoning and invocation without destroying the continuity and traceability of the dialogue.

[0048] The above-mentioned segmentation of the historical message array into old message fragments and recent message fragments includes: obtaining the timestamp attribute of each message object in the historical message array; setting a preset session pause time threshold (e.g., 30 minutes); traversing backwards from the end (latest message) to the head (oldest message) of the historical message array, and calculating the timestamp difference between adjacent message objects in turn; in response to the first detection that the timestamp difference between adjacent message objects is greater than the session pause time threshold, marking the position as a time breakpoint; dividing all messages from the time breakpoint to the end of the array into recent message fragments, and stripping all messages from the time breakpoint and before the time breakpoint into old message fragments.

[0049] Optionally, the above-mentioned segmentation of the historical message array into old message fragments and recent message fragments includes: extracting the text content of each message object in the historical message array and converting it into a set of semantic vectors through a lightweight embedding model; setting a preset semantic similarity decay threshold; using the semantic vector of the latest user input message at the end of the historical message array as a reference, calculating the cosine similarity between the semantic vector of the historical message and the reference vector from back to front; in response to detecting that the cosine similarity of a certain historical message falls below the semantic similarity decay threshold, determining that topic drift has occurred at that position, and marking that position as a topic boundary node; dividing all messages belonging to the current topic after the topic boundary node into recent message fragments, and stripping all messages belonging to the historical topic before the topic boundary node into old message fragments.

[0050] In this embodiment, the above-mentioned summary extraction algorithm includes: a text feature extraction algorithm, which converts old message fragments into summary nodes by calling the summary extraction algorithm or summary generation model, and reassembles the summary nodes and recent message fragments into a target message array. Specifically, it includes: extracting the text content of all message objects in the old message fragments; calling a statistical text feature extraction algorithm (such as TF-IDF algorithm or TextRank algorithm) on the local computer system to perform word segmentation, part-of-speech tagging and weight calculation on the text content; extracting the core nouns or noun phrases with the highest weight from the text content according to a preset quantity threshold (such as Top-10) as a keyword set representing the historical context; concatenating the keyword set into a tagged string according to a preset template (e.g., "[historical topic tags]: minors, high-speed rail"); instantiating the summary node, injecting the tagged string into the content attributes of the summary node, and concatenating and reassembling the summary node and recent message fragments in memory into a target message array.

[0051] Optionally, the above-mentioned summary generation model includes: a lightweight information extraction model or a syntactic dependency parsing engine. The above-mentioned method of converting old message fragments into summary nodes by calling a summary extraction algorithm or summary generation model, and recombining the summary nodes and recent message fragments into a target message array, may further include: inputting old message fragments into a locally deployed lightweight information extraction model or syntactic dependency parsing engine; identifying entity nodes contained in the old message fragments and the logical relationships between entities, constructing multiple knowledge graph relation triples in the form of (subject, predicate, object) (e.g., identifying (user, rejection, seafood stall), (AI, recommendation, All Seasons Hotel)); serializing all generated relation triples into structured text in a strict JSON array format or a graph database query language (such as Cypher) format; instantiating summary nodes, injecting the structured text into the content attributes of the summary nodes, and placing the summary nodes before the recent message fragments to recombine them into a target message array.

[0052] Step 105: Serialize the long-term entity memory dictionary into structured text, encapsulate the structured text and static system instructions into top-level message nodes, and linearly concatenate the top-level message nodes and target message arrays according to a preset priority order to generate and send the request body array of the current round to the large language model so that the large language model can generate the response information of the current round.

[0053] In this embodiment, the top-level message node is the control node located at the bottom of the stack (i.e., index 0) in the final generated array of request bodies sent to the large model. In the underlying attention calculation mechanism of the large language model, the node at this position has the highest influence weight. Encapsulating the constraint entity here forces the large model to be constrained as a deterministic computational unit that strictly adheres to the external parameters.

[0054] In this embodiment, the serialization of the long-term entity memory dictionary into structured text includes: determining whether the long-term entity memory dictionary is empty; in response to the long-term entity memory dictionary not being empty, allocating a string buffer in memory and writing a preset prompt word block marker; traversing the key-value pair set of the long-term entity memory dictionary and mapping the key-value pairs line by line into a Markdown format list text.

[0055] In this embodiment, the linear concatenation of the top-level message node and the target message array according to a preset priority order includes: placing the top-level message node at the bottom of the stack of the request body array and assigning it the highest attention priority of absolute execution constraints; in response to the existence of a summary node in the target message array, appending the summary node sequentially after the top-level message node as the second-highest priority node providing historical event background; appending recent message fragments from the target message array after the summary node to anchor the recent dialogue context; and appending the user input message as the latest action-driven node to the end of the request body array.

[0056] In this embodiment, the long-term entity memory dictionary is first serialized according to a predetermined field mapping rule to generate structured text that can be stably parsed by the model. Then, the structured text and pre-fixed static system instructions are encapsulated together as top-level message nodes to form global constraints and long-term memory anchors for the dialogue context. Next, the system linearly concatenates the top-level message nodes and the target message array according to a preset priority order (e.g., "system instructions / long-term memory top-level nodes take precedence over business dialogue target messages") to obtain a request body array that meets the input format requirements of the large language model and has a determined order. Finally, the request body array is sent to the large language model as the request payload for a model call, thereby achieving stable constraints on model behavior and highly consistent injection of context information without changing the semantics of the target message.

[0057] like Figure 3 As shown, the intelligent agent context processing method disclosed herein is divided into three main stages: message storage and asynchronous entity extraction, threshold-triggered context compression, and assembly of a request-invoking large model. In the first stage: receiving user input messages and processing them asynchronously in parallel, the front-end interaction layer → threshold monitor: receives the user input message object; the threshold monitor → entity extraction module: sends the user input message in a bypass manner, performs regular expression / NER entity extraction, and returns a set of key-value entities to the threshold monitor after processing; the threshold monitor updates its local long-term entity memory dictionary and pushes the current user input message into the memory historical message array.

[0058] Phase Two: Threshold Judgment. If the condition is met, threshold-triggered dimensionality reduction compression is executed. The threshold monitor counts the array length and total number of word segments in the historical message array to determine if the compression threshold has been reached. If the threshold is not triggered, the compression logic is skipped. If the threshold is triggered, the compression process is executed. In this compression process, the threshold monitor → slicing and dimensionality reduction module: the historical message array is completely transmitted, and the slicing and dimensionality reduction module splits the messages; the slicing and dimensionality reduction module → large language model API: an asynchronous summary request is initiated, and the large language model API returns the compressed summary text; the slicing and dimensionality reduction module reconstructs the user input message array, and the compressed target message array is sent back to the threshold monitor.

[0059] Phase 3: Assemble system prompts, initiate a large model call, threshold monitor → assembly engine: input entity memory + processed historical message array, assembly engine formats entity memory, appends to basic system prompts, concatenates all messages according to dialogue sequence, and generates a complete message list; assembly engine → large language model API: initiate RPC (Remote Procedure Call) remote call, pass the assembled request body array, after the large language model obtains the request body array, it generates the response information for the current round, where the response information for the current round is obtained by the large language model after analyzing the user input message for the current round.

[0060] The intelligent agent context processing method provided in the embodiments of this disclosure firstly, in response to receiving a user input message, appends the user input message and the response information generated by the large language model in the previous round as new nodes to the historical message array; secondly, an asynchronous processing task is created outside the main dialogue flow, and the constraint entities in the user input message are extracted through the asynchronous processing task, and the constraint entities are updated to the long-term entity memory dictionary, which is independent of the historical message array; thirdly, it detects whether the historical message array meets the preset threshold conditions, which include a two-dimensional monitoring condition of message quantity threshold and word segmentation unit load threshold; then... When the historical message array is detected to meet a threshold condition, it is segmented into old message fragments and recent message fragments. The old message fragments are transformed into summary nodes by calling a summary extraction algorithm or summary generation model, and the summary nodes and recent message fragments are recombined into the target message array. Finally, the long-term entity memory dictionary is serialized into structured text, and the structured text and static system instructions are encapsulated into top-level message nodes. The top-level message nodes and the target message array are linearly concatenated according to a preset priority order to generate and send the current round's request body array to the large language model, so that the large language model can generate the current round's response information. Thus, by creating an asynchronous processing task outside the main dialogue flow, it is ensured that the mandatory constraints determined by the user in the early stage always exist at the top of the large model's attention calculation, which is immune to the long text attention dilution effect and ensures the absolute accuracy of business execution. A two-dimensional threshold detection based on message quantity and word segmentation unit load is introduced, and hard slicing and dimensionality reduction transformation are performed when the limit is exceeded. This mechanism forcibly converges the input data volume of the large language model from linear growth with each round to a constant-level range fluctuation. While completely eliminating the risk of memory overflow, it can save a lot of word element interface call costs and significantly reduce the latency of the first character output. When performing message dimensionality reduction compression, recent message fragments are preserved, ensuring that the large model can receive recent native dialogue text without any loss. This allows for accurate parsing of relative referential instructions in user input that strongly depend on recent context, such as "previous option" and "second option," avoiding the logical breaks caused by traditional indiscriminate compression. Entities adopt a non-blocking asynchronous processing task mechanism, and the monitoring of the word segmentation unit load adopts a pre-computation and register accumulation mode. This avoids the main thread CPU blocking caused by full word segmentation of ultra-large strings before sending requests in traditional solutions, greatly improving the response speed and concurrent processing capability of the gateway layer.

[0061] In some optional implementations of this disclosure, the aforementioned asynchronous processing task includes an asynchronous sub-coroutine. An asynchronous processing task is created outside the main dialogue flow, and constraint entities in the user input message are extracted through this asynchronous processing task. This includes: creating a non-blocking asynchronous sub-coroutine outside the main dialogue flow, and using a multi-level degradation extraction engine to extract constraint entities from the user input message within the asynchronous sub-coroutine; the multi-level degradation extraction engine includes a regular expression engine and a model depth engine; the regular expression engine is a hard matching engine based on regular expressions, used for fast matching and extraction of highly regularized intent entities; the model depth engine is a deep extraction engine based on an auxiliary language model, used to construct a request with a limited preset format and send it to the auxiliary language model for feature extraction when the regular expression engine fails to hit core fields or the statement contains complex context.

[0062] In this optional implementation, asynchronous sub-coroutines refer to lightweight concurrent execution units scheduled through an event loop within a single thread, and belong to a specific software engineering implementation of asynchronous processing tasks.

[0063] In this optional implementation, the multi-level degradation extraction engine is an adaptive algorithm routing mechanism that dynamically balances computational cost and extraction accuracy. When a high-level, high-speed algorithm (such as a regular expression scan in CPU memory) fails to hit the target, it automatically "falls back" and calls a low-level, heavy algorithm (such as an auxiliary language model). This can greatly save the expensive computational power of the main large model while ensuring 100% extraction recall.

[0064] In this optional implementation, the main dialogue flow is used by the main thread to push user input messages into a queue for subsequent large language model dialogue inference, while the asynchronous sub-coroutine does not use await to block the main thread, and uses a new task to hand over the user input messages to the parallel entity extraction module for entity extraction.

[0065] To balance extraction accuracy and computational cost, asynchronous sub-coroutines implement a multi-level fallback extraction algorithm. Level 1: Fast hard matching based on high-frequency regular expressions. For highly regularized intent entities (e.g., date, number of days, number of people in a group), it prioritizes fast matching in CPU memory using a pre-compiled regular expression engine. Time complexity: Where M is the string length. Level 2: Deep extraction based on fine-tuning small model / inexpensive large model API. If the regular expression engine fails to hit the core field, or the system judgment statement contains complex context, the model depth engine will construct a request with a strictly defined JSON schema and send it to an auxiliary model specifically used for feature extraction (such as an 8B / 14B level open source model fine-tuned for JSON output, or an inexpensive large model such as GPT-4o-mini).

[0066] In this optional implementation, the request to construct a limited preset format is sent to the auxiliary language model for feature extraction, including: constructing a prompt word template that strictly limits the description of the JSON schema data structure; constraining the auxiliary language model to output a standard JSON object through the prompt word template; validating the standard JSON object, and mapping the key-value pairs in the valid JSON object to the long-term entity memory dictionary.

[0067] The optional implementation provides a method for extracting constraint entities. By deriving non-blocking asynchronous sub-coroutines outside the main dialogue flow, the main thread does not need to wait when extracting constraint entities, thus not increasing the response time of the large language model in generating responses and ensuring the smoothness of user interaction. It introduces a multi-level degradation extraction engine, which prioritizes CPU-memory-based regular expression fast matching for rule-based data and only calls the auxiliary language model in complex contexts. This ensures 100% recall of core facts while greatly saving the expensive computing power and lexical consumption of the main trunk model.

[0068] In some optional implementations of this disclosure, updating the constraint entity to the long-term entity memory dictionary includes: performing an in-situ update operation on the hash table of the long-term entity memory dictionary based on the time series last priority principle; forcibly overwriting the old value corresponding to the key with the value corresponding to the constraint entity in response to the constraint entity's key already existing in the hash table; and performing an insertion operation of a new key-value pair in response to the constraint entity's key not existing in the hash table.

[0069] In this optional implementation, the time-series later priority principle is a conflict resolution strategy when dealing with multi-turn dialogue state machines. That is, the system assumes that the intent expressed by the user at the latest time node (later turn) has absolute overwrite authority, which is used to correct the settings in the earlier dialogue.

[0070] In this optional implementation, the in-situ update operation is a computer memory operation strategy. When a new entity is retrieved, instead of creating a new dictionary copy, the corresponding memory address of the hash table is directly addressed to perform overwrite or insert. This completely erases old and conflicting business facts at the physical level, preventing logical confusion in the large model during reasoning.

[0071] In this optional implementation, the execution entity builds a hash table (key-value dictionary) in memory. When a new constraint is received, an "in-place update" is performed. For example, suppose yesterday the user said "budget 8000" (the system dictionary records budget: 8000). Today the user changes their mind and says "budget changed to 6000". The system will not create a new record, but will directly find the key budget through constant-order O(1) addressing and forcibly overwrite 8000 with 6000.

[0072] The optional implementation provides a method for updating constraint entities to the long-term entity memory dictionary. This method performs in-situ updates based on the principle of time series priority. When a user changes their mind, old and conflicting business facts (such as budgets decreasing from 8000 to 6000) are directly overwritten and erased at the physical memory level. This prevents contradictory information from being input into the large model at the same time and avoids logical chaos. The update of the long-term entity memory dictionary uses hash table addressing, which has a constant time complexity of O(1). Even in ultra-long-range dialogues, it can maintain extremely high memory operation efficiency.

[0073] Traditional frameworks typically call the tokenizer (such as the BPE algorithm) to perform full computation on the entire massive string concatenation before sending the request when monitoring the tokenization unit threshold. When the text is tens of thousands of tokenization units long, the tokenization process can cause severe CPU blocking on the main thread, resulting in a computation delay of hundreds of milliseconds.

[0074] To address the aforementioned technical issues, in some optional implementations of this disclosure, the detection of whether the historical message array meets the preset threshold condition includes: when appending user input messages to the historical message array, independently calling the word segmenter to calculate the number of word segmentation units in the user input message, and accumulating the number of word segmentation units in the global word segmentation status cache; synchronously updating the total number of objects in the historical message array to the global quantity status register; and detecting whether the historical message array meets the preset threshold condition by comparing the accumulated value in the global word segmentation status cache with the word segmentation unit load threshold, and comparing the accumulated value in the global quantity status register with the message quantity threshold.

[0075] In this optional implementation, the global word segmentation state cache is a dedicated integer (e.g., Int32) storage unit in the system's running memory used to store the total token load of the current context. This transforms the massive full word segmentation calculation into a microsecond-level scalar addition (O(1) complexity) when enqueuing messages, completely eliminating the pain point of severe CPU blocking of the main thread before sending requests in traditional solutions.

[0076] In this optional implementation, the following registers and constants are allocated during initialization: Static threshold constants: : An integer constant that defines the maximum number of objects allowed in the message_history array. The setting is usually based on the maximum number of consecutive backtracking rounds designed for the business logic (e.g., set to 40). : An integer constant that defines the maximum number of segmentation units that the system is allowed to pass to the large model API. This value is typically set to 80% of the upper limit of the underlying model's physical window, with the remaining 20% ​​reserved for the AI ​​response to be generated.

[0077] Global quantity status register and global word segmentation status cache: : An integer variable that records the actual length of the message_histor list of historical messages currently in memory. : An integer variable that records the total number of tokenized units accumulated across all elements in the current historical message array message_history.

[0078] To eliminate blocking, this module breaks down the full word segmentation into local, incremental pre-computations, maintaining an accumulator similar to a "cursor". Whenever a new Message object is added to the historical message array... Upon creation, the system immediately calls the word segmenter to calculate the number of word segments for the object using formula (1). The results are then stored in the object's properties.

[0079] (1)

[0080] When a message is pushed into the history queue, as shown in equation (2), the total number of objects in the history message array is updated to the global quantity status register. In the middle; as shown in equation (3), the number of word segmentation units is accumulated in the global word segmentation state cache. In the process, the status register only executes with a time complexity of O(n log n). Scalar addition operation:

[0081] (2)

[0082] (3)

[0083] If the old message pops up during subsequent operations (such as compression and truncation in Module 3), The system performs a symmetric scalar subtraction operation as shown in equation (4) to maintain memory state consistency:

[0084] (4)

[0085] The optional implementation provides a method for detecting whether the historical message array meets the preset threshold conditions. It calculates the word segmentation amount independently only when a new message is enqueued, and updates the global register through scalar addition, which effectively reduces system latency. It breaks down the huge word segmentation calculation into local incremental pre-computation. When the system performs threshold detection, it only needs to perform a branchless Boolean decision (O(1) complexity) once, and accurately controls the "physical level" of the current context with near-zero latency overhead.

[0086] Optionally, the above-mentioned method for setting the word segmentation unit load threshold includes: obtaining the upper limit of the physical input window of the underlying large language model; dividing the upper limit of the physical input window into an input request reserved space and a response generation reserved space according to a preset ratio; and setting the number of word segmentation units corresponding to the input request reserved space as the word segmentation unit load threshold to prevent the large language model from experiencing memory overflow when generating a response.

[0087] In some embodiments of this disclosure, before serializing the long-term entity memory dictionary into structured text, the above-mentioned agent context processing method further includes: subtracting the sum of the number of segmentation units of each message object in the old message fragment from the global segmentation state cache, and adding the number of segmentation units of the summary node, so that the global segmentation state cache is consistent with the physical state of the target message array.

[0088] The agent context processing method provided by this optional implementation undergoes a dramatic change in system memory structure after the destructive array slicing and node reconstruction are completed. By performing a precise scalar operation of "subtracting the old slice load and adding the new summary load" on the global word segmentation state cache, strict synchronization between the monitoring register and the actual memory array state is ensured. This provides a reliable benchmark for accurate threshold monitoring in the next round of dialogue and prevents monitoring failure.

[0089] In some optional implementations of this disclosure, the above-mentioned division of the historical message array into old message fragments and recent message fragments includes: obtaining a preset sliding retain window size; extracting a number of messages equal to the sliding retain window size from the end of the historical message array as recent message fragments, and stripping the remaining messages in the historical message array into old message fragments.

[0090] In this optional implementation, the sliding retain window forces the tail data range (e.g., always retaining the latest 4 messages) to be exempt from compression or deletion when truncating the historical message array. The sliding retain window avoids the fragmentation of referential resolution caused by global compression. It ensures that the large model receives recent native dialogue text without loss, thus accurately understanding user commands such as "choose the second option," which strongly rely on recent text anchoring relationships.

[0091] In this optional implementation, when compression is triggered, the executing entity must not compress all historical records into a compressed file. Instead, the executing entity will set a "sliding retention window" (e.g., 4 records) to cut off the end of the array, taking away the oldest 17 records for compression, while retaining the most recent 4 original conversations intact.

[0092] This optional implementation provides a method for segmenting recent and old message fragments. When truncation is necessary due to context overflow, it forcibly retains recent native messages within a sliding window. This ensures that the large model can receive recent dialogue text without any lossy compression, thereby accurately parsing relative referential instructions in user input that strongly depend on recent context, such as "do it according to the second solution you mentioned in the previous post," avoiding logical breaks caused by global compression.

[0093] In some optional implementations of this disclosure, the above-mentioned summary generation model includes an asynchronous summary model. The above-mentioned transformation of old message fragments into summary nodes by calling a summary extraction algorithm or a summary generation model, and reorganization of summary nodes and recent message fragments into a target message array, includes: serializing multiple alternating message objects in the old message fragments into a long text string; sending the long text string to the asynchronous summary model for summary calculation to obtain a dimensionality reduction result string; instantiating a summary node, forcibly setting the role attribute of the summary node to the system level, and injecting the dimensionality reduction result string into the content attribute of the summary node; and reconstructing the target message array by sequentially concatenating the summary node and recent message fragments on the original memory reference.

[0094] In this optional implementation, the alternating message object refers to a collection of structured objects that alternate in chronological order among different roles such as the user and the AI ​​assistant throughout the multi-turn dialogue history.

[0095] In this optional implementation, the asynchronous summary model is an auxiliary computing node (such as a low-parameter micro-language model) that is independent of the main trunk large language model and is specifically used to perform text summary compression. By calling this auxiliary dimensionality reduction model in a non-blocking remote manner, the system completes the dimensionality compression of historical data without consuming expensive trunk computing power.

[0096] like Figure 2As shown, the serialization engine executes step 6: looping through and extracting and formatting into a long text string; then executes step 7: dispatching the refinement task to the asynchronous requester; the asynchronous requester executes step 8: sending a POST request to the auxiliary dimensionality reduction model, which includes: Prompt + text_payload; the auxiliary dimensionality reduction model executes step 9: returning the dimensionality reduction result string summary_str to the asynchronous requester; the asynchronous requester executes step 10: passing the dimensionality reduction result string summary_str to the slice executor; the slice executor executes step 11: instantiating a new object: summary_node = Message type system, summary_str; the slice executor then executes step 12: overwriting the original memory pointer: message_history becomes [summary_node] + recent_slice; the slice executor executes step 13: callback notification, triggering the register variable rewind.

[0097] In this optional implementation, sending the long text string to the asynchronous digest model for digest calculation to obtain the dimensionality reduction result string includes: using the long text string as the request payload to initiate a non-blocking remote procedure call (RPC) or HTTP POST request to an auxiliary dimensionality reduction model that is independent of the main large language model; and extracting the dimensionality reduction result string from the request's return body.

[0098] In this optional implementation, after slicing, the system targets the old message slice `old_messages_slice`. This `old_messages_slice` was originally a collection of structured Message objects representing multiple alternating roles (User / AI), with dimensions of... The system needs to reduce its dimensionality to a scalar string – the dimensionality reduction result string, i.e. The specific steps are as follows:

[0099] Step 1: Serialization Unpacking. The system iterates through old message fragments (old_messages_slice) and extracts them into single long text strings with clear boundary features, so that the dimensionality reduction model can understand the timing of the dialogue.

[0100] The second step is to use an asynchronous dimensionality reduction RPC call based on a micro-model. Instead of using the high-cost, high-parameter LLM (such as GPT-4o) used in the main dialogue, the serialized long text string is used as the payload to initiate a non-blocking RPC / HTTPPOST request to call a low-cost model dedicated to text extraction (such as a finely tuned Llama-3-8B or a dedicated Summarizer interface).

[0101] After obtaining the dimensionality reduction result string, the system must destroy the old array at the original memory address and create a new array structure that conforms to the large model API format specification. This new array structure is the target message array. The specific steps are as follows:

[0102] As shown in equation (5), a new Message object is instantiated as the container. Its role is forcibly set to "system" (giving it higher underlying weight), and the summary node... The dimensionality reduction result string is injected into the content attribute.

[0103] (5)

[0104] As shown in equation (6), the message_history is reassembled on the original memory reference to obtain the target message array. The system will then generate new summary nodes. Compared to previously retained recent message slices They are linked sequentially in memory.

[0105] (6)

[0106] This optional implementation provides a method for reorganizing the target message array. It serializes the lengthy, old alternating messages and sends them to the auxiliary model for compression, forcibly converging the originally unbounded, linearly growing array size into a constant-level range of fluctuations, completely avoiding the risk of OutOfMemory (OOM). The dimensionality-reduced summary result is instantiated into nodes with a system-level role attribute. Utilizing the physical law that gives the highest weight to system instructions in the underlying attention mechanism of the large model, it both frees up memory space and ensures that long-term historical event context is not forgotten.

[0107] Further reference Figure 4 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of an intelligent agent context processing device, which is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0108] like Figure 4As shown, the intelligent agent context processing device 400 provided in this embodiment includes: an append unit 401, an extraction unit 402, a detection unit 403, a segmentation unit 404, and a sending unit 405. The append unit 401 can be configured to, in response to receiving a user input message, append the user input message and the response information generated by the large language model in the previous round as new nodes to the historical message array. The extraction unit 402 can be configured to create an asynchronous processing task outside the main dialogue flow, extract constraint entities from the user input message through the asynchronous processing task, and update the constraint entities to the long-term entity memory dictionary, which is independent of the historical message array. The detection unit 403 can be configured to detect whether the historical message array meets a preset threshold condition, which includes a two-dimensional monitoring condition of a message quantity threshold and a word segmentation unit load threshold. The aforementioned segmentation unit 404 can be configured to, in response to detecting that the historical message array meets a threshold condition, segment the historical message array into old message fragments and recent message fragments, convert the old message fragments into summary nodes by calling a summary extraction algorithm or summary generation model, and reassemble the summary nodes and recent message fragments into a target message array. The aforementioned sending unit 405 can be configured to serialize the long-term entity memory dictionary into structured text, encapsulate the structured text and static system instructions into a top-level message node, linearly concatenate the top-level message node and the target message array according to a preset priority order, generate and send the current round's request body array to the large language model, so that the large language model generates the current round's response information.

[0109] In this embodiment, the specific processing of the appending unit 401, the extraction unit 402, the detection unit 403, the segmentation unit 404, and the sending unit 405 in the intelligent agent context processing device 400, and the resulting technical effects, can be found in the following references: Figure 1 The relevant descriptions of steps 101, 102, 103, 104, and 105 in the corresponding embodiments will not be repeated here.

[0110] In some embodiments of this disclosure, the asynchronous processing task includes an asynchronous sub-coroutine, and the extraction unit 402 is configured to: create a non-blocking asynchronous sub-coroutine outside the main dialogue flow, and use a multi-level degradation extraction engine to extract constrained entities from user input messages in the asynchronous sub-coroutine; the multi-level degradation extraction engine includes a regular expression engine and a model depth engine; the regular expression engine is a hard matching based on regular expressions, used for fast matching and extraction of highly regularized intent entities; the model depth engine is a deep extraction based on an auxiliary language model, used to construct a request with a limited preset format and send it to the auxiliary language model for feature extraction when the regular expression engine fails to hit the core field or the statement contains a complex context.

[0111] In some embodiments of this disclosure, the extraction unit 402 is configured to: perform an in-situ update operation on the hash table of the long-term entity memory dictionary based on the time series last priority principle; in response to the fact that the key of the constraint entity already exists in the hash table, forcibly overwrite the old value corresponding to the key using the value corresponding to the constraint entity; and in response to the fact that the key of the constraint entity does not exist in the hash table, perform an insertion operation of a new key-value pair.

[0112] In some embodiments of this disclosure, the detection unit 403 is configured to: independently call the word segmenter to calculate the number of word segmentation units in the user input message when appending the user input message to the historical message array, and accumulate the number of word segmentation units in the global word segmentation status cache; synchronously update the total number of objects in the historical message array to the global quantity status register; and detect whether the historical message array meets the preset threshold conditions by comparing the accumulated value in the global word segmentation status cache with the word segmentation unit load threshold, and comparing the accumulated value in the global quantity status register with the message quantity threshold.

[0113] In some embodiments of this disclosure, the above-mentioned intelligent agent context processing device 400 further includes: a word segmentation unit (not shown in the figure), which is configured to: subtract the sum of the number of word segmentation units of each message object in the old message fragment from the global word segmentation state cache, and add the number of word segmentation units of the summary node, so that the global word segmentation state cache is consistent with the physical state of the target message array.

[0114] In some embodiments of this disclosure, the cutting unit 404 is configured to: obtain a preset sliding retain window size; extract a number of messages equal to the sliding retain window size from the end of the historical message array as recent message fragments, and strip the remaining messages in the historical message array into old message fragments.

[0115] In some embodiments of this disclosure, the above-mentioned summary generation model includes an asynchronous summary model, and the above-mentioned segmentation unit 404 is further configured to: serialize multiple alternating message objects in the old message fragment into a long text string; send the long text string to the asynchronous summary model for summary calculation to obtain a dimensionality reduction result string; instantiate a summary node, force the role attribute of the summary node to be set to the system level, and inject the dimensionality reduction result string into the content attribute of the summary node; and on the original memory reference, sequentially connect the summary node and the recent message fragment to reconstruct a target message array.

[0116] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0117] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0118] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their patterns are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0119] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. The RAM 503 may also store various programs and data required for the operation of the electronic device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0120] Multiple components in electronic device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of monitors, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows electronic device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0121] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as agent context processing methods. For example, in some embodiments, the agent context processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the agent context processing method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform agent context processing methods by any other suitable means (e.g., by means of firmware).

[0122] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0123] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable intelligent agent context processing device, such that when executed by the processor or controller, the patterns / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0124] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0125] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0126] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0127] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0128] The foregoing description of specific exemplary embodiments of this disclosure is for illustrative and explanatory purposes. These descriptions are not intended to limit this disclosure to the precise forms disclosed, and it will be apparent that many changes and variations can be made in accordance with the foregoing teachings. The exemplary embodiments were chosen and described in order to explain the specific principles of this disclosure and their practical application, thereby enabling those skilled in the art to implement and utilize various different exemplary embodiments of this disclosure, as well as various different choices and variations. The scope of this disclosure is intended to be defined by the claims and their equivalents.

Claims

1. A method for processing the context of an intelligent agent, the method comprising: In response to receiving a user input message, the user input message and the response information generated by the large language model in the previous round are added as new nodes to the historical message array; An asynchronous processing task is created outside the main dialogue flow. The constraint entity in the user input message is extracted through the asynchronous processing task, and the constraint entity is updated to the long-term entity memory dictionary. The long-term entity memory dictionary is independent of the historical message array. The system detects whether the historical message array meets the preset threshold conditions, which include a dual-dimensional monitoring condition of message quantity threshold and word segmentation unit load threshold. In response to the detection that the historical message array meets the threshold condition, the historical message array is divided into old message fragments and recent message fragments. The old message fragments are converted into summary nodes by calling a summary extraction algorithm or summary generation model. The summary nodes and recent message fragments are then recombined into the target message array. The long-term entity memory dictionary is serialized into structured text, the structured text and static system instructions are encapsulated into a top-level message node, the top-level message node and the target message array are linearly concatenated according to a preset priority order to generate and send the current round of request body array to the large language model, so that the large language model generates the current round of response information.

2. The method according to claim 1, characterized in that, The asynchronous processing task includes asynchronous sub-coroutines. Creating the asynchronous processing task outside the main dialogue flow and extracting constraint entities from the user input message through the asynchronous processing task includes: A non-blocking asynchronous sub-coroutine is created outside the main dialogue process, and a multi-level degradation extraction engine is used in the asynchronous sub-coroutine to extract constraint entities from the user input message. The multi-level degradation extraction engine includes a regular expression engine and a model depth engine; the regular expression engine is a hard matching engine based on regular expressions, used to quickly match and extract highly regularized intent entities; the model depth engine is a deep extraction engine based on an auxiliary language model, used to construct a request with a limited preset format and send it to the auxiliary language model for feature extraction when the regular expression engine fails to hit the core field or the statement contains a complex context.

3. The method according to claim 1, characterized in that, The step of updating the constraint entity to the long-term entity memory dictionary includes: Based on the time-series priority principle, the hash table of the long-term entity memory dictionary is updated in-situ. In response to the fact that the key of the constraint entity already exists in the hash table, the old value corresponding to the key is forcibly overwritten using the value corresponding to the constraint entity; If the key of the constrained entity does not exist in the hash table, an insertion operation for a new key-value pair is performed.

4. The method according to claim 1, characterized in that, The step of detecting whether the historical message array meets the preset threshold condition includes: When appending the user input message to the historical message array, the word segmenter is independently called to calculate the number of word segmentation units of the user input message, and the number of word segmentation units is accumulated in the global word segmentation status cache; The total number of objects in the historical message array is updated to the global quantity status register simultaneously. By comparing the accumulated value in the global word segmentation status cache with the word segmentation unit load threshold, and by comparing the accumulated value in the global quantity status register with the message quantity threshold, it is detected whether the historical message array meets the preset threshold conditions.

5. The method according to claim 4, characterized in that, Before serializing the long-term entity memory dictionary into structured text, the method further includes: The sum of the number of segmentation units of each message object in the old message fragment is subtracted from the global segmentation state cache, and the number of segmentation units of the summary node is added to ensure that the global segmentation state cache is consistent with the physical state of the target message array.

6. The method according to claim 1, characterized in that, The step of dividing the historical message array into old message fragments and recent message fragments includes: Get the preset sliding window size; The number of messages equal to the size of the sliding retain window is taken from the end of the historical message array as recent message fragments, and the remaining messages in the historical message array are stripped into old message fragments.

7. The method according to claim 1, characterized in that, The summary generation model includes an asynchronous summary model. The step of converting the old message fragments into summary nodes by calling a summary extraction algorithm or summary generation model, and then recombining the summary nodes and the recent message fragments into a target message array, includes: Serialize multiple alternating message objects in the old message fragment into a long text string; The long text string is sent to the asynchronous summarization model for summarization calculation to obtain the dimensionality reduction result string; Instantiate the summary node, force the role attribute of the summary node to be set to system level, and inject the dimensionality reduction result string into the content attribute of the summary node; On the original memory reference, the digest node and the recent message fragment are sequentially concatenated and reconstructed into a target message array.

8. An intelligent agent context processing apparatus, the apparatus comprising: The appending unit is configured to, in response to receiving a user input message, append the user input message and the response information generated by the large language model in the previous round as a new node to the historical message array. The extraction unit is configured to create an asynchronous processing task outside the main dialogue flow, extract constraint entities from the user input message through the asynchronous processing task, and update the constraint entities to the long-term entity memory dictionary, which is independent of the historical message array. The detection unit is configured to detect whether the historical message array meets the preset threshold conditions, the threshold conditions including a two-dimensional monitoring condition of message quantity threshold and word segmentation unit load threshold; The segmentation unit is configured to, in response to detecting that the historical message array meets the threshold condition, segment the historical message array into old message fragments and recent message fragments, convert the old message fragments into summary nodes by calling a summary extraction algorithm or a summary generation model, and reassemble the summary nodes and recent message fragments into a target message array; The sending unit is configured to serialize the long-term entity memory dictionary into structured text, encapsulate the structured text and static system instructions into a top-level message node, linearly concatenate the top-level message node and the target message array according to a preset priority order, generate and send the request body array of the current round to the large language model, so that the large language model generates the response information of the current round.

9. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

10. A computer-readable storage medium storing computer instructions thereon, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

Images