Context processing method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application provide a context processing method and device, electronic equipment and storage medium, relating to the technical field of large language models. The method guarantees the timeliness of the response through a threshold triggering mechanism during the operation of an industrial large language model inference engine, realizes "zero loss of key information and high compression rate of non-key information" with the help of structured perception and dynamic importance evaluation, and achieves high compression rate and inference acceleration through a hierarchical compression strategy. Finally, through recoverable reconstruction, the compressed context is strictly compatible with the input specification of the large language model, and fully supports breakpoint continuation, full audit and lossless backtracking of original data, thereby solving the multiple contradictions of context management in accuracy, real-time performance, compliance and edge adaptability in industrial scenarios.

Description

Technical Field

[0001] This invention relates to the field of large language model technology, and more specifically, to a context processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the increasing application of large language models in the industrial field, inference engines driven by large language models are widely deployed in scenarios such as photovoltaic cell quality inspection and predictive maintenance of equipment. These inference engines interact with SCADA systems, MES systems, and visual inspection platforms through tool call interfaces to perform complex tasks including equipment status query, EL image analysis, and generation of process parameter optimization suggestions.

[0003] When the context length of a large language model exceeds the context window limit, the common practice is to simply truncate the earliest dialogue history, retaining only the most recent messages. In industrial scenarios, equipment fault diagnosis often requires tracing back historical alarm records and tool call results from hours or even days ago. The fixed-window truncation method indiscriminately discards old data, potentially truncating critical error codes, safety interlock signals, and incomplete work order information, causing the large language model's inference chain to break and preventing it from providing correct diagnostic conclusions or operational suggestions. Summary of the Invention

[0004] In view of this, the object of the present invention is to provide a context processing method, apparatus, electronic device and storage medium.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, the present invention provides a context processing method applied to an electronic device, the electronic device being equipped with a large language model inference engine, the method comprising: In response to a trigger signal from the large language model inference engine indicating that the current context occupancy has reached a preset threshold, the original context data is obtained from the large language model inference engine. The original context data is subjected to a preset perceptual parsing process to obtain a set of context fragments, and the importance score of each context fragment in the set of context fragments is obtained; Based on the importance score of each context segment in the context segment set, the context segments in the context segment set are subjected to hierarchical compression processing to obtain compressed context data; The compressed context data is subjected to a preset reconstruction process so that the processed compressed context data conforms to the input specifications of the large language model inference engine.

[0006] Optionally, the step of performing preset perception parsing processing on the original context data includes: The original context data is parsed in a structured manner, and the structured elements are extracted and stored in the metadata dictionary; Perform semantic classification on all natural language text fragments in the original context data, and assign a category label to each natural language text fragment; Each natural language text fragment, after being assigned a category label, is annotated with a task lifecycle to obtain multiple context fragments.

[0007] Optionally, the step of obtaining the importance score of each context segment in the context segment set includes: For each context segment, a base score is set for the context segment based on the task stage to which the context segment belongs; The content of the context segment is matched according to preset rules, and the basic score of the context segment is adjusted according to the matching results. The importance score of the context segment is determined based on the adjusted basic score.

[0008] Optionally, the step of determining the importance score of the context segment based on the adjusted base score includes: The adjusted base score is used as the importance score of the context segment; or The context segment is subjected to task relevance prediction and redundancy judgment to obtain an auxiliary score for the context segment. The auxiliary score of the context segment is then fused with the adjusted base score of the context segment to obtain the importance score of the context segment.

[0009] Optionally, the step of performing hierarchical compression processing on each context segment of the context segment set based on the importance score of each context segment in the context segment set includes: Obtain the available token budget for the large language model inference engine; The context fragments are sorted in descending order of importance score, and each context fragment is compressed sequentially based on the available token budget and the score range in which its importance score falls.

[0010] Optionally, the step of sequentially compressing the sorted context fragments based on the available token budget and the score range in which the importance score of each context fragment falls includes: For the current context segment being processed, if the number of tokens used in the compression process does not exceed the available token budget, then a target compression strategy is determined based on the score range of the importance score of the current context segment being processed, and the current context segment is compressed according to the target compression strategy. If the number of tokens used in the compression process exceeds the available token budget, a target compression strategy will be determined based on the score range of the importance score of the current context fragment to be processed, and the current context fragment will be compressed according to the downgraded target compression strategy.

[0011] Optionally, the step of performing a preset reconstruction process on the compressed context data includes: Construct the triplet information of the compressed context data, and save the triplet information and the execution state snapshot corresponding to the tool call chain that is currently in an incomplete state; The associated index information is embedded in the compressed context data, and the compressed context data with the embedded associated index information is converted into the standard input format of the large language model inference engine to obtain the converted compressed context data. A compression indicator tag is embedded in the converted compressed context data so that the large language model inference engine can recognize the converted compressed context data.

[0012] In a second aspect, the present invention provides a context processing apparatus applied to an electronic device, the electronic device being equipped with a large language model inference engine, the apparatus comprising: The acquisition module is used to acquire the original context data from the large language model inference engine in response to a trigger signal that the current context occupancy has reached a preset threshold reported by the large language model inference engine. The processing module is used to perform preset perceptual parsing processing on the original context data to obtain a set of context fragments, and obtain the importance score of each context fragment in the set of context fragments; according to the importance score of each context fragment in the set of context fragments, perform hierarchical compression processing on each context fragment in the set of context fragments to obtain compressed context data; and perform preset reconstruction processing on the compressed context data to make the processed compressed context data conform to the input specifications of the large language model inference engine.

[0013] Thirdly, the present invention provides an electronic device including a processor and a memory, the memory storing machine-executable instructions executable by the processor, the processor executing the machine-executable instructions to implement the context processing method described in the first aspect above.

[0014] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the context processing method as described in the first aspect above.

[0015] The context processing method, apparatus, electronic device, and storage medium provided in this invention respond to a trigger signal from a large language model inference engine reporting that the current context occupancy has reached a preset threshold. The method obtains raw context data from the large language model inference engine; performs preset perceptual parsing processing on the raw context data to obtain a set of context fragments, and obtains the importance score of each context fragment in the set; performs hierarchical compression processing on each context fragment in the set based on the importance score of each context fragment to obtain compressed context data; and performs preset reconstruction processing on the compressed context data to make the processed compressed context data conform to the input specifications of the large language model inference engine. Because the embodiments of the present invention ensure timely response through a threshold triggering mechanism during the operation of the industrial large language model inference engine, and achieve "zero loss of key information and high compression ratio of non-key information" through structured perception and dynamic importance assessment, and then achieve high compression ratio and inference acceleration through a layered compression strategy; finally, through recoverable reconstruction, it ensures that the compressed context is both strictly compatible with the input specifications of the large language model and fully supports breakpoint resume, full audit and lossless backtracking of original data, thereby solving the multiple contradictions of context management in industrial scenarios in terms of accuracy, real-time performance, compliance and edge adaptability.

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This figure shows a schematic block diagram of an electronic device provided by an embodiment of the present invention; Figure 2 A flowchart illustrating a context processing method provided by an embodiment of the present invention is shown; Figure 3 A flowchart illustrating the implementation process of step S103 provided in an embodiment of the present invention is shown. Figure 4 A flowchart illustrating the implementation process of step S104 provided in an embodiment of the present invention is shown. Figure 5 A functional block diagram of a context processing device provided in an embodiment of the present invention is shown.

[0019] Icons: 100 - Electronic device; 110 - Memory; 120 - Processor; 130 - Communication module; 200 - Context processing device; 201 - Acquisition module; 202 - Processing module. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0022] It should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0023] In industrial scenarios, intelligent agents driven by large language models need to continuously make multiple tool calls with SCADA systems, MES systems, and visual inspection platforms, leading to a rapid accumulation of context data that exceeds the model's processing capacity. Existing technologies, such as fixed truncation, one-time summarization, or retrieval-enhanced generation, cannot simultaneously address multiple rigid requirements, including critical information preservation, continuous toolchain execution, audit traceability, and edge resource constraints. The fundamental reason is that these solutions simply treat the context as an undifferentiated text stream, failing to recognize the inherent industrial structured semantics (such as equipment identifiers, error codes, work order numbers, timestamps, and tool call IDs), distinguish the actual roles of different text fragments in the task lifecycle (e.g., tool descriptions in the planning phase, real-time parameters in the execution phase, and full data retention required in the audit phase), and fail to establish a verifiable, locatable, and reproducible mapping between the compressed results and the original data.

[0024] In response, embodiments of the present invention provide a context processing method, apparatus, electronic device, and storage medium, which will be described in detail below.

[0025] Please refer to Figure 1 This is a block diagram of an electronic device 100. The electronic device 100 includes a memory 110, a processor 120, and a communication module 130. The memory 110, processor 120, and communication module 130 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0026] The memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0027] The processor 120 is used to read / write data or programs stored in the memory 110 and to perform corresponding functions.

[0028] The communication module 130 is used to establish a communication connection between the electronic device 100 and other communication terminals through the network, and to send and receive data through the network.

[0029] It should be understood that, Figure 1 The structure shown is only a schematic diagram of the electronic device 100. The electronic device 100 may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0030] Please refer to Figure 2 The context processing method includes steps S101 to S104.

[0031] S101, in response to the trigger signal that the current context occupancy amount reported by the large language model inference engine has reached a preset threshold, obtain the original context data from the large language model inference engine.

[0032] When the current context of the large language model inference engine approaches a preset threshold (e.g., reaching 70% of the context window capacity), the compression process is triggered. The original context data includes user command messages, large language model response messages, tool call requests and return results (such as tool_use / tool_result messages defined by the Model Context Protocol (MCP), equipment status data pushed by the SCADA system or Manufacturing Execution System (MES), and references to historical diagnostic reports, etc.

[0033] S102, perform preset perceptual parsing processing on the original context data to obtain a set of context fragments, and obtain the importance score of each context fragment in the set of context fragments.

[0034] In this embodiment of the invention, the process of “performing preset perception parsing processing on the original context data” can be as follows: performing structured parsing on the original context data, extracting structured elements and storing them in a metadata dictionary; performing semantic classification on all natural language text fragments in the original context data, assigning category labels to each natural language text fragment; and performing task lifecycle annotation on each natural language text fragment after assigning category labels to obtain multiple context fragments.

[0035] In other words, the electronic device 100 uses a predefined regular expression library and an industrial communication protocol parser (supporting Modbus RTU / TCP, Open Platform Unified Architecture (OPC UA) and other technologies) to identify and mark structured elements such as device identifiers (e.g., Device_ID_7890), work order numbers (e.g., WO-20260112-005), ISO 8601 format timestamps, status codes and error codes (e.g., ERROR_4045), tool call IDs (e.g., call_abc123), and process parameter values ​​(e.g., temperature 325.7℃, pressure 1.2MPa) from the raw context data, and stores them in the metadata dictionary in the form of key-value pairs.

[0036] Meanwhile, the electronic device 100 uses a lightweight Natural Language Processing (NLP) classification model (such as the TinyBERT model optimized by ONNX, with a model size of less than 50 megabytes and a single inference latency of less than 10 milliseconds) to perform semantic classification on natural language text fragments in the context, classifying them into one of four categories: user command category, tool response category, system log category, or large language model inference category, and attaching the corresponding classification label to the metadata of the text fragment.

[0037] Furthermore, the electronic device 100 uses a task execution state machine model to annotate the task lifecycle of each text fragment. This state machine defines four stages: planning (tool selection and planning stage), execution (tool invocation and execution stage), integration (result integration stage), and audit (post-audit stage). The annotations reflect the currently active tool invocation chain and its dependencies (e.g., call_1: el_analyzer → call_2: defect_classifier → call_3: report_generator), incomplete subtasks (e.g., "Pending execution: Generate maintenance suggestions"), and abnormal interruption points (e.g., "Tool timeout, second retry pending execution"). The final output is a collection of context objects with complete meta tags. Each context fragment is organized in JSON or Protocol Buffers (Protobuf) format and includes fields such as type (fragment type), is_critical (whether it is critical), task_phase (task phase), and structured_entities (structured entity list).

[0038] Furthermore, in this embodiment of the invention, the process of “obtaining the importance score of each context segment in the context segment set” can be as follows: for each context segment, a basic score is set for the context segment based on the task stage to which the context segment belongs; the content of the context segment is matched according to a preset rule, and the basic score of the context segment is adjusted according to the matching result, and the importance score of the context segment is determined based on the adjusted basic score.

[0039] The implementation process of "determining the importance score of the context segment based on the adjusted base score" can be as follows: using the adjusted base score as the importance score of the context segment; or performing task relevance prediction and redundancy judgment on the context segment to obtain the auxiliary score of the context segment, and then fusing the auxiliary score of the context segment with the adjusted base score of the context segment to obtain the importance score of the context segment.

[0040] In other words, the electronic device 100 calculates a comprehensive importance score S for each context fragment, with a value range of [0,1]. This score can be obtained by weighted fusion of the following three parallel dimensions.

[0041] The first dimension is the assignment of task stage weights, which dynamically sets the base weights based on the current task stage. This includes: when in the planning stage, the weight of tool descriptions and capability specifications is 1.0, and the weight of historical execution records is reduced to 0.5; when in the execution stage, the weight of the current tool's input parameters and intermediate outputs is 1.0, and the weight of other tool descriptions is reduced to 0.3; when in the integration stage, the weight of each tool's final return result is 1.0, and the weight of intermediate debugging logs is reduced to 0.4; when in the audit stage, the weight of all segments is set to 1.0.

[0042] The second dimension is industrial semantic rule base matching, which involves matching content and adjusting scores through a built-in rule engine: segments containing keywords such as ERROR, ALERT, and FAULT receive an additional 0.5 points; segments involving safety interlocking devices such as emergency stop signals, over-temperature protection, and leakage protection are forcibly set to 1.0; segments with three or more consecutive successful logs of the same type receive a deduction of 0.3 points; segments containing work order numbers and work order status "in progress" receive an additional 0.3 points; segments with timestamps older than a preset aging threshold (e.g., two hours) and without cross-references receive a deduction of 0.2 points.

[0043] The third dimension is a lightweight model-assisted scoring (optional). When computing resources allow, a fine-tuned model is called to determine whether the segment affects the next decision and to calculate its semantic redundancy with the preceding text (cosine similarity greater than 0.85 is considered redundant). The resulting auxiliary score is incorporated into the comprehensive score with a weight of 0.2.

[0044] The scores for the three dimensions are calculated using the formula S = W_phase S_phase + W_rule S_rule + W_model S_model, where W_phase, W_rule, and W_model are the fusion coefficients of stage weight, rule weight, and model weight, respectively, and can be set to 0.4, 0.4, and 0.2. If model-assisted scoring is not enabled, the first two, W_phase and W_rule, should be adjusted to 0.5.

[0045] S103. Based on the importance score of each context fragment in the context fragment set, perform hierarchical compression processing on each context fragment in the context fragment set to obtain compressed context data.

[0046] Please refer to Figure 3 In a possible implementation, step S103 includes sub-steps S103-1 to S103-2.

[0047] S103-1. Obtain the available Token budget of the large language model inference engine.

[0048] S103-2. Sort each context fragment in descending order according to the importance score, and perform compression processing on the sorted context fragments in turn according to the available Token budget and the score range where the importance score of each context fragment is located.

[0049] Among them, in the embodiment of the present invention, for the currently processed context fragment, if the number of used Tokens in the compression process does not exceed the available Token budget, the target compression strategy is determined according to the score range where the importance score of the currently processed context fragment is located, and the currently processed context fragment is compressed according to the target compression strategy; if the number of used Tokens in the compression process exceeds the available Token budget, the target compression strategy determined according to the score range where the importance score of the currently processed context fragment is located is degraded, and the currently processed context fragment is compressed according to the degraded target compression strategy.

[0050] In other words, after obtaining the importance score of each context fragment, the electronic device 100 first reads the total capacity of the current context window of the large language model inference engine (such as 128K tokens), deducts the system reserved area (such as ten percent for system prompt words and safety boundaries) and the latest round of non-compressible user instructions, and obtains the available Token budget Budget_available.

[0051] Then sort all context fragments in descending order according to the importance score S; then traverse the sorted fragments in turn and determine the target compression strategy according to the interval where its S value is located: when S >= s1 (the value of s1 can be 0.8), execute the anchor point retention strategy, retain all content as it is, establish a metadata index containing information such as fragment ID, type, timestamp, associated tool call ID, etc., and mark is_anchor=true; when s2 <= S < s1 (the value of s1 can be 0.4), execute the intelligent summary strategy, aggregate consecutive fragments of the same type (such as aggregating ten EL detection success logs into "10 EL detections all passed, yield 100%"), generate a structured summary using a predefined industrial summary template, insert a [SUMMARY] mark before the summary, and record the storage pointer of the original fragment (such as database table name plus primary key range, or file path plus byte offset); when S < s2, execute the high compression or discard strategy, use Protobuf binary encoding to achieve more than fifty percent volume compression for fragments containing structured data, directly delete repetitive plain text logs and record the deletion summary (such as "10 duplicate image loading success logs have been deleted"), and save the original storage addresses of all deleted fragments at the same time.

[0052] During the compression process, the electronic device continuously accumulates the number of used tokens. If it exceeds Budget_available, an adaptive degradation mechanism is triggered, which downgrades unprocessed low-score segments from smart digests to high compression and discards them, or downgrades non-mandatory anchor segments to smart digests, until the budget constraint is met.

[0053] Finally, a structured compressed log is generated, recording the original size, compressed size, strategy used, and deletion summary of each segment, and written to the audit log storage in JSON format.

[0054] S104, Perform preset reconstruction processing on the compressed context data to make the processed compressed context data conform to the input specifications of the large language model inference engine.

[0055] Please refer to Figure 4 In a possible implementation, step S104 includes sub-steps S104-1 to S104-3.

[0056] S104-1, Construct the triplet information of the compressed context data, and save the triplet information and the execution state snapshot corresponding to the tool call chain that is currently in an incomplete state.

[0057] S104-2, embed association index information into the compressed context data, and convert the compressed context data with embedded association index information into the standard input format of the large language model inference engine to obtain the converted compressed context data.

[0058] S104-3, embed compression indicator tags in the transformed compressed context data so that the large language model inference engine can recognize the transformed compressed context data.

[0059] When performing preset reconstruction processing on the compressed context data in this embodiment of the invention, the electronic device 100 generates a unique context version number (formatted as CTX_{first eight bits of content hash}, for example, CTX_a3b7c9d2) for subsequent incremental compression and backtracking positioning.

[0060] Calculate the SHA-256 hash values ​​of the original context and the compressed context respectively, form a triple of (original hash, compressed hash, version number) and store it in the version management table so that when the original context needs to be restored later, the data can be verified by hash comparison.

[0061] Embed a lightweight JSON-formatted index pointer in the compressed context. This pointer contains the original storage location (such as the database table name and primary key range, or file path, byte offset, and length), the number of original fragments, the number of tokens before compression, and the compression strategy identifier. The number of tokens occupied by the index pointer is extremely small (usually <20 tokens), but it is sufficient to accurately locate the original context data.

[0062] For a tool call chain that is currently incomplete, the electronic device 100 saves a complete state snapshot, which includes the call stack (i.e., the current step in the tool call chain and the completed call steps), the input parameters and return results of each completed call, the input parameters of the currently pending call, and a retry counter (recording the number of timeouts or failure retries). Then, the compressed context is converted into the standard input format required by the large language model inference engine, namely the messages array (a list of JSON objects containing role and content fields), and compression indicator markers (such as [SUMMARY: last 5 logs aggregated], [COMPRESSED: 120 entries →protobuf]) are embedded in the content field to clearly indicate which content has been compressed to the large language model.

[0063] When an audit request is triggered subsequently (such as "view the original EL detection log from 14:00 to 14:15"), the electronic device 100 locates the original data based on the version number and index pointer, first verifies the consistency of the SHA-256 hash value, then reads it from the persistent storage layer and restores it to the standard context format, and finally returns the complete original context to the requester. The entire backtracking process is independent of the current large language model inference session and does not affect its real-time operation.

[0064] In summary, the context processing method provided by this invention is not simply a deletion or summary of text, but rather a closed-loop and controllable context lifecycle management system built on the electronic device, driven by industrial task logic, anchored by structured semantics, and guaranteed by verifiable mapping. It not only significantly improves the context compression rate (reaching 80% to 85% in actual tests), but also ensures zero loss of critical information such as error codes, security signals, and tool parameters, guarantees the ability to resume execution from breakpoints in multi-step tool call chains, meets the mandatory quality traceability and process auditing requirements of industries such as photovoltaics and semiconductors, and adapts to the limited computing and memory resources of edge servers. It truly achieves the technical goals of "compression without distortion, deletion that can be restored, uninterrupted execution, and auditing with evidence" in industrial-grade large language model applications.

[0065] To perform the corresponding steps in the above embodiments and various possible methods, an implementation of the context processing device 200 is given below. Further, please refer to... Figure 5 , Figure 5 This is a functional block diagram of a context processing device 200 provided in an embodiment of the present invention. It should be noted that the context processing device 200 provided in this embodiment has the same basic principle and technical effects as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments. The context processing device 200 includes: The acquisition module 201 is used to acquire raw context data from the large language model inference engine in response to a trigger signal that the current context occupancy has reached a preset threshold, as reported by the large language model inference engine.

[0066] The processing module 202 is used to perform preset perceptual parsing processing on the original context data to obtain a set of context fragments and obtain the importance score of each context fragment in the set of context fragments; based on the importance score of each context fragment in the set of context fragments, perform hierarchical compression processing on each context fragment in the set of context fragments to obtain compressed context data; and perform preset reconstruction processing on the compressed context data to make the processed compressed context data conform to the input specifications of the large language model inference engine.

[0067] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory 110 shown is either stored in or embedded in the operating system (OS) of the electronic device 100, and can be used by... Figure 1 The processor 120 executes the program. Meanwhile, the data and program code required to execute the above modules can be stored in the memory 110.

[0068] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0069] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0070] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0071] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A context processing method, characterized in that, Applied to an electronic device equipped with a large language model inference engine, the method includes: In response to a trigger signal from the large language model inference engine indicating that the current context occupancy has reached a preset threshold, the original context data is obtained from the large language model inference engine. The original context data is subjected to a preset perceptual parsing process to obtain a set of context fragments, and the importance score of each context fragment in the set of context fragments is obtained; Based on the importance score of each context segment in the context segment set, the context segments in the context segment set are subjected to hierarchical compression processing to obtain compressed context data; The compressed context data is subjected to a preset reconstruction process so that the processed compressed context data conforms to the input specifications of the large language model inference engine.

2. The context processing method as described in claim 1, characterized in that, The step of performing preset perception parsing processing on the original context data includes: The original context data is parsed in a structured manner, and the structured elements are extracted and stored in the metadata dictionary; Perform semantic classification on all natural language text fragments in the original context data, and assign a category label to each natural language text fragment; Each natural language text fragment, after being assigned a category label, is annotated with a task lifecycle to obtain multiple context fragments.

3. The context processing method as described in claim 1, characterized in that, The step of obtaining the importance score of each context fragment in the context fragment set includes: For each context segment, a base score is set for the context segment based on the task stage to which the context segment belongs; The content of the context segment is matched according to preset rules, and the basic score of the context segment is adjusted according to the matching results. The importance score of the context segment is determined based on the adjusted basic score.

4. The context processing method as described in claim 3, characterized in that, The step of determining the importance score of the context segment based on the adjusted base score includes: The adjusted base score is used as the importance score of the context segment; or The context segment is subjected to task relevance prediction and redundancy judgment to obtain an auxiliary score for the context segment. The auxiliary score of the context segment is then fused with the adjusted base score of the context segment to obtain the importance score of the context segment.

5. The context processing method as described in claim 1, characterized in that, The step of performing hierarchical compression processing on each context segment of the context segment set based on the importance score of each context segment in the context segment set includes: Obtain the available token budget for the large language model inference engine; The context fragments are sorted in descending order of importance score, and each context fragment is compressed sequentially based on the available token budget and the score range in which its importance score falls.

6. The context processing method as described in claim 5, characterized in that, The step of sequentially compressing the sorted context fragments based on the available token budget and the score range of the importance score of each context fragment includes: For the current context segment being processed, if the number of tokens used in the compression process does not exceed the available token budget, then a target compression strategy is determined based on the score range of the importance score of the current context segment being processed, and the current context segment is compressed according to the target compression strategy. If the number of tokens used in the compression process exceeds the available token budget, a target compression strategy will be determined based on the score range of the importance score of the current context fragment to be processed, and the current context fragment will be compressed according to the downgraded target compression strategy.

7. The context processing method as described in claim 1, characterized in that, The step of performing a preset reconstruction process on the compressed context data includes: Construct the triplet information of the compressed context data, and save the triplet information and the execution state snapshot corresponding to the tool call chain that is currently in an incomplete state; The associated index information is embedded in the compressed context data, and the compressed context data with the embedded associated index information is converted into the standard input format of the large language model inference engine to obtain the converted compressed context data. A compression indicator tag is embedded in the converted compressed context data so that the large language model inference engine can recognize the converted compressed context data.

8. A context processing apparatus, characterized in that, Applied to an electronic device, the electronic device being equipped with a large language model inference engine, the device includes: The acquisition module is used to acquire the original context data from the large language model inference engine in response to a trigger signal that the current context occupancy has reached a preset threshold reported by the large language model inference engine. The processing module is used to perform preset perceptual parsing processing on the original context data to obtain a set of context fragments, and obtain the importance score of each context fragment in the set of context fragments; according to the importance score of each context fragment in the set of context fragments, perform hierarchical compression processing on each context fragment in the set of context fragments to obtain compressed context data; and perform preset reconstruction processing on the compressed context data to make the processed compressed context data conform to the input specifications of the large language model inference engine.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor to implement the context processing method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the context processing method as described in any one of claims 1-7.

Images