Dynamic kv cache replacement method and system based on semantic correlation degree

Abstract

The application discloses a dynamic KV cache replacement method and system based on semantic correlation degree, and relates to the technical field of unstructured long document retrieval cache management; the application firstly acquires the type and the stage attribute information of the current review task, collects the semantic feature, the historical attention distribution, the semantic label of the cached KV segment and the evidence chain correlation attribute data, calculates the semantic correlation degree score of each cached KV segment, compares the score with the retention threshold and the elimination threshold respectively, locks the segment higher than the retention threshold, removes the segment lower than the elimination threshold, preferentially eliminates the low-score segment of the intermediate-score segment when the cache space is insufficient, and triggers the RAG recall required uncached segment after scheduling is completed. The application considers multi-dimensional business attributes to complete cache scheduling, adapts to dynamic requirements in different review stages, can be directly connected to the existing long document database deployment, and is suitable for multiple types of professional long document review scenes.

Description

Technical Field

[0001] This invention relates to the field of unstructured long document retrieval and caching management technology, and in particular to a dynamic key-value cache replacement method and system based on semantic relevance. Background Technology

[0002] In professional scenarios such as lawyer document review, tax audit, and contract compliance review, the case files, audit working papers, and contract texts to be processed typically reach thousands of pages. Currently, retrieval-enhanced generation technology combined with large language models is commonly used to complete automated reviews. During the review process, different segments from long documents need to be frequently retrieved as reasoning context. Some complex review tasks also need to associate evidence content and professional rule provisions across documents, placing high demands on retrieval response speed, contextual completeness, and content relevance. Database pre-caching is the core means to reduce retrieval latency. The cache replacement strategy directly determines the utilization rate of cache space, context hit rate, and system stability, and is one of the core supporting technologies of long document review systems.

[0003] Current mainstream cache replacement strategies primarily employ time-series strategies such as LRU and FIFO. These strategies determine retention or eviction solely based on time-series attributes like access time and queuing order, neglecting the semantic attributes and business relevance of the cached content. In long document review scenarios, core content highly relevant to the current review task may be evicted due to lack of recent access, requiring repeated retrieval from the underlying database and slowing down overall inference. Some improved semantic caching strategies rely solely on the semantic similarity between cached content and the current query, failing to consider specific attributes like task stage differences, evidence chain relationships, and professional rule citation logic in professional review scenarios. They only achieve single-dimensional content matching, unable to adapt to the dynamic needs of multi-stage reviews or identify key business-level content.

[0004] Existing cache replacement strategies are designed for general scenarios and are not optimized for the business characteristics of long document review in specialized fields. They cannot identify key content such as evidence chain nodes and rule reference nodes, nor can they dynamically adjust the scheduling logic according to changes in the review stage. During operation, problems such as missing key evidence, scattered elimination of upstream and downstream fragments of the evidence chain, and incomplete contextual information are prone to occur. Ultimately, this leads to information omissions and contradictions in the review conclusions generated by the large language model, making it difficult to meet the reliability requirements of professional review scenarios. Existing solutions also cannot adjust the scheduling logic according to business attributes. The same strategy cannot adapt to the needs of different types of review tasks, requiring a lot of manual parameter configuration for different scenarios, resulting in high maintenance costs. Summary of the Invention

[0005] The present invention proposes a dynamic key-value cache replacement method and system based on semantic correlation to solve the problems mentioned in the prior art.

[0006] In a first aspect, embodiments of this application provide a dynamic key-value cache replacement method based on semantic relevance, comprising the following steps: For retrieval and access scenarios of unstructured long document databases, the system obtains the attribute information of the current review task, including task type, current task stage, source identifier of the long document to be processed, and mapping relationship with related professional rule bases. The task types cover three categories: lawyer file review, tax audit working paper review, and contract compliance review. The task stages include four categories: preliminary review, evidence sorting, rule matching, and conclusion issuance. The KV cache is a front-end retrieval cache for unstructured long document databases, using a memory-level key-value pair storage structure. The system collects the semantic features of the current query request, the attention distribution sequence generated during historical interactions, the semantic labels corresponding to the cached key-value segments, and the evidence chain association attributes. The attention distribution sequence is the cumulative value of the token-level attention weights generated when the large language model processes historical query requests. The semantic labels corresponding to the key-value segments include three categories: evidence type, rule number, and related party information. The evidence chain association attributes include two categories: the node level of the segment in the evidence chain and the identifiers of the associated upstream and downstream evidence segments. The semantic relevance score of each cached key-value segment is calculated based on the collected multi-dimensional data; the semantic relevance score of each key-value segment is then compared with the preset retention threshold and elimination threshold. Key-value segments with semantic relevance scores higher than the retention threshold are locked in the cache space, while key-value segments with semantic relevance scores lower than the eviction threshold are removed from the cache. When the remaining cache space is insufficient and there are uncached key-value segments waiting to be called, the RAG recall logic is triggered to retrieve the target key-value segment, and at the same time, the key-value segments in the scheduling state are dynamically adjusted after priority sorting.

[0007] Furthermore, the dynamic KV cache replacement method based on semantic relevance also includes dynamic identification logic for task stages. Specifically, it updates the identifier of the current task stage in real time based on the user interaction operation sequence, the completed review node markings, and the node matching rules of the professional review process. The user interaction operation sequence includes four categories: annotation operation, query operation, jump operation, and export operation. Each review process node corresponds to a set of operation feature combinations. When the matching degree between the collected operation sequence and the feature combination of a certain node reaches a preset matching threshold, the current task stage is determined to be updated to the corresponding node. Different task stages correspond to different retention thresholds and elimination thresholds, and different weight coefficients are used for each dimension when calculating the corresponding semantic relevance score. The evidence chain association weight coefficient is the highest for the evidence sorting stage, the professional rule association weight coefficient is the highest for the rule matching stage, the task matching degree weight coefficient is the highest for the preliminary review stage, and the historical attention weight coefficient is the highest for the conclusion issuance stage.

[0008] Furthermore, it also includes priority locking logic for key evidence (KV) fragments. When a KV fragment is marked as a core evidence node or a professional rule reference node, its semantic relevance score is directly assigned the highest value, and it is continuously retained in the cache regardless of insufficient cache space. It is only triggered to be cleaned up after the corresponding review task is completed. When a user performs operations such as marking a KV fragment as core evidence, referencing it to the review conclusion, or associating it with a rule clause, the system automatically raises the priority of the KV fragment to the highest level, and at the same time, the semantic relevance scores of its associated upstream and downstream evidence KV fragments are simultaneously increased by a preset amount.

[0009] Furthermore, the semantic relevance score is a quantitative indicator of the relevance between the retrieval results of unstructured long documents and the current query requirements. It is used to support the priority scheduling of the database's front-end cache. The calculation process of the semantic relevance score first dynamically calibrates the values ​​of the four weight coefficients α, β, γ, and δ based on the current task type, and then completes the quantitative calculation using the following formula: ; in, Let represent the semantic relevance score of the i-th cached key-value segment, and α represent the weight coefficient corresponding to the current task stage. Let represent the matching score between the i-th key-value segment and the current task stage, and β represent the historical attention weight coefficient. γ represents the cumulative historical attention value corresponding to the i-th KV segment, and γ represents the evidence chain association weight coefficient. Let represent the correlation score between the i-th key-value segment and the currently locked evidence chain, and δ represent the professional rule correlation weight coefficient. This represents the matching score between the i-th KV segment and the associated professional rule base. The calibration process for each weight coefficient first calls the preset scene weight mapping table to match the initial weight value corresponding to the current task type and task stage, and then dynamically adjusts the weight value based on the hit results of the past 20 query requests.

[0010] Furthermore, in the dynamic cache adjustment step, when the remaining cache space is insufficient, the key-value segments to be scheduled are first sorted from low to high according to their semantic relevance scores, and the key-value segments to be scheduled with the lowest scores are removed first. If the remaining space is still insufficient to store the key-value segments to be recalled after removal, the key-value segments to be scheduled are traversed again to perform the removal operation until the cache space meets the requirements. At the same time, when the difference in semantic relevance scores between two key-value segments is less than the preset difference threshold, the key-value segment with the more recent last access time is retained first.

[0011] Furthermore, when the RAG re-recall logic is triggered by the vector index module of the unstructured long document database, it first converts the current query request into a semantic vector, and then performs cosine similarity calculation with the pre-computed semantic vectors of all segments in the long document segment library. The top N segments with the highest similarity are selected as the key-value segments to be recalled. The value of N is dynamically adjusted according to the size of the remaining cache space. If the remaining space is sufficient, N takes a larger value; if the remaining space is insufficient, N takes a smaller value. The newly recalled key-value segments will first be stored in the temporary area of ​​the cache, and then adjusted to the corresponding cache area after the semantic relevance score calculation is completed.

[0012] Furthermore, the marking process for core evidence nodes is jointly determined based on user annotation operations and the automatic identification results of the evidence chain reasoning model. The evidence chain reasoning model adopts a pre-trained graph neural network model. The input is the semantic vector and associated attribute information of all cached KV fragments, and the output is the probability value of each KV fragment belonging to a core evidence node. When the probability value is higher than the preset probability threshold, the KV fragment is automatically marked as a core evidence node. The marking of professional rule reference nodes is determined based on the matching results of the rule number and rule reference statement contained in the KV fragment. It adopts a combination of exact matching and fuzzy matching. First, it matches whether the KV fragment contains a preset rule number prefix. If the match is successful, the exact matching of the rule number is performed. If the match is unsuccessful, the semantic similarity matching of the rule clause is performed.

[0013] Secondly, embodiments of this application provide a dynamic key-value cache replacement system based on semantic association, comprising the following modules: The task information collection module is used to connect to the retrieval request interface of the unstructured long document database, the metadata storage module, the inference output interface of the large language model, and the underlying long document storage database interface to obtain the attribute information of the current review task, the semantic features of the current query request, the attention distribution sequence generated during the historical interaction process, the semantic tags and evidence chain association attributes corresponding to the cached KV fragments. The semantic relevance calculation module has a built-in preset weight mapping table, dynamic calibration logic, and quantization calculation logic. It is used to calculate the semantic relevance score of each cached KV segment based on the collected multi-dimensional structured data and supports multi-task parallel computing. The threshold comparison module has built-in configurable retention and elimination thresholds. The threshold values ​​can be customized by users according to their own business needs, or the system's preset scenario thresholds can be selected. It is used to compare the semantic relevance score of each KV segment with the preset retention and elimination thresholds respectively, and output the status label results corresponding to each KV segment. The cache scheduling module interfaces with the underlying KV cache database and supports atomic cache locking, removal, and marking operations. It is used to lock KV fragments with semantic relevance scores higher than the retention threshold in the cache space, remove KV fragments with semantic relevance scores lower than the eviction threshold from the cache, mark KV fragments between the two thresholds as pending scheduling, and perform priority sorting and removal operations on pending KV fragments when the remaining cache space is insufficient. The re-recall trigger module has a built-in RAG retrieval engine and connects to the underlying long document vector database. It supports high-throughput semantic similarity retrieval with retrieval latency that meets the needs of real-time interaction. When there is an uncached key-value segment to be called, it triggers the RAG re-recall logic to retrieve the target key-value segment and pushes the newly recalled key-value segment to the semantic relevance calculation module to complete the scoring calculation.

[0014] Thirdly, embodiments of this application provide an electronic device, including a processor and a memory, wherein instructions are stored in the memory and loaded and executed by the processor to implement the dynamic KV cache replacement method based on semantic association in any of the above embodiments.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the dynamic KV cache replacement method based on semantic association degree according to any of the above-described embodiments.

[0016] Compared with existing technologies, the beneficial effects of this invention are: This invention calculates the scheduling priority of cached fragments by comprehensively considering multiple attributes, including task stage matching degree, historical attention distribution, evidence chain correlation degree, and professional rule matching degree. It no longer relies solely on temporal attributes or single-dimensional semantic similarity to complete cache scheduling. This enables the retained cached content to better meet the actual needs of the current review task, reduces the occupation of cache space by irrelevant content, increases the hit probability of cached content, reduces unnecessary underlying database query operations, and lowers the overall retrieval latency.

[0017] This invention sets up a key content priority locking logic, which can automatically identify core evidence nodes, professional rule reference nodes and other content that has an important impact on the review conclusion, thus avoiding the wrong elimination of such content. At the same time, it can simultaneously improve the scheduling priority of related upstream and downstream segments, ensure the integrity of the evidence chain, reduce the deviation of the review conclusion due to incomplete context information, and improve the consistency of the review results.

[0018] This invention employs a dynamic weight adjustment mechanism, which automatically adjusts the weight ratio of each dimension attribute according to the core needs of different review stages. It can also adaptively calibrate the scheduling logic based on the hit rate of historical queries. It eliminates the need for manual configuration of dedicated caching rules for different review scenarios, resulting in better adaptability and making it suitable for various professional long document review scenarios such as lawyer document review, tax audit, and contract compliance. Furthermore, the caching scheduling logic of this invention can directly interface with the front-end caching module of existing unstructured long document databases without modifying the underlying architecture of the original database. This lowers the deployment threshold and allows for direct application to performance optimization of existing long document review systems without significant adjustments to existing business processes. Attached Figure Description

[0019] Figure 1 This is a schematic block diagram of the dynamic key-value cache replacement method based on semantic association proposed in this invention; Figure 2 This is a schematic block diagram of the task phase dynamic identification and weight calibration flowchart proposed in this invention; Figure 3 This is a flowchart illustrating the key evidence priority locking and cascading enhancement process proposed in this invention; Figure 4 This is a flowchart of the dynamic adjustment and recall process when the cache space is insufficient, as proposed in this invention. Figure 5 This is a structural block diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] The first aspect of this application provides a dynamic key-value cache replacement method based on semantic association. This dynamic key-value cache replacement method based on semantic association includes the following steps: S1. For retrieval and access scenarios of unstructured long document databases, the system obtains the attribute information of the current review task, including task type, current task stage, source identifier of the long document to be processed, and mapping relationship with related professional rule bases. Task types cover three categories: lawyer document review, tax audit working paper review, and contract compliance review. Task stages include four categories: preliminary review, evidence sorting, rule matching, and conclusion issuance. The KV cache is a front-end retrieval cache for unstructured long document databases, using a memory-level key-value pair storage structure. The key is a unique identifier for a long document fragment, and the value is a combination of the fragment's text content, semantic vector, and attribute tags. The KV fragment is an independent text sub-fragment obtained after segmenting a long document and is the smallest storage unit stored in the KV cache. The mapping relationship with related professional rule bases records the professional rule index association entries corresponding to different documents and business scenarios.

[0022] S2. Collect the semantic features of the current query request, the attention distribution sequence generated during historical interactions, the semantic tags corresponding to the cached KV fragments, and the evidence chain association attributes. The semantic features of the query request are obtained by encoding the query text by the large language model. The attention distribution sequence is the cumulative value obtained by the large language model in real time by accumulating the corresponding token-level attention weights after processing each historical query request, without timed clearing. The attention distribution sequence is the cumulative value of the token-level attention weights generated when the large language model processes historical query requests. The semantic tags corresponding to the KV fragments include three categories: evidence type, rule number, and related party information. The evidence chain association attributes include two categories: the node level of the fragment in the evidence chain and the identifiers of the associated upstream and downstream evidence fragments.

[0023] S3. Calculate the semantic relevance score of each cached key-value (KV) segment based on the collected multi-dimensional data; compare the semantic relevance score of each KV segment with the preset retention threshold and elimination threshold respectively; the retention threshold and elimination threshold can be flexibly adjusted according to the current task stage. For example, the preset retention threshold and elimination threshold are normalized score thresholds in the range of 0 to 1. For example, in the evidence sorting stage, the retention threshold can be configured to 0.7 and the elimination threshold to 0.3; in the preliminary review stage, the retention threshold can be configured to 0.5 and the elimination threshold to 0.2; all thresholds can be customized and modified by the administrator according to the document length and query concurrency.

[0024] S4. Lock KV segments with semantic relevance scores higher than the retention threshold in the cache space, remove KV segments with semantic relevance scores lower than the eviction threshold from the cache, and mark KV segments between the two thresholds as pending scheduling. The pending scheduling state means that this type of KV segment is not subject to locking protection and can participate in the eviction sorting when the cache space is insufficient.

[0025] S5. When the remaining cache space is insufficient and there are uncached KV fragments to be called, the RAG recall logic is triggered to retrieve the target KV fragment. At the same time, the KV fragments in the scheduling state are dynamically adjusted after priority sorting. The new KV fragments obtained by RAG recall are first stored in the temporary cache area. After the semantic relevance score is calculated, the cache partitioning is completed according to the score matching retention threshold and eviction threshold.

[0026] In one implementation, the dynamic key-value cache replacement method based on semantic correlation also includes dynamic identification logic for task stages. Specifically, it updates the identifier of the current task stage in real time based on the user interaction operation sequence, the completed review node markings, and the node matching rules of the professional review process. The user interaction operation sequence includes four categories: annotation operations, query operations, jump operations, and export operations. Each review process node corresponds to a set of operation feature combinations (for example, the feature combination of the evidence sorting node includes high-frequency text annotation and fragment jump operations). When the matching degree between the collected operation sequence and the feature combination of a certain node reaches a preset matching threshold, the current task stage is determined to be updated to the stage corresponding to that node. For example, the preset matching threshold can be 0.7, and the matching degree uses a normalized value from 0 to 1. If it is higher than 0.7, the operation behavior is considered to conform to the characteristics of the current business node.

[0027] Different task stages correspond to different retention and elimination thresholds, and the weighting coefficients for each dimension are adjusted in different proportions when calculating the semantic relevance score. These weighting coefficients include task matching weighting, historical attention weighting, evidence chain relevance weighting, and professional rule relevance weighting. The evidence chain relevance weighting is highest in the evidence review stage, highest in the rule matching stage, highest in the task matching stage, and highest in the conclusion issuance stage. By adaptively adjusting the cache filtering thresholds and scoring weights at each stage, the system can accurately match the focus of each review stage, reduce the caching of invalid key-value segments, improve retrieval response speed, and ensure that cache scheduling aligns with the core needs of the current stage.

[0028] In one implementation, priority locking logic for key evidence (KV) fragments is also included. When a KV fragment is marked as a core evidence node or a professional rule reference node, its semantic relevance score is directly assigned the highest value, and it is continuously retained in the cache regardless of insufficient cache space. It is only cleared after the corresponding review task is completed. When a user performs operations such as marking a KV fragment as core evidence, referencing a review conclusion, or associating it with a rule clause, the system automatically raises the priority of that KV fragment to the highest level. At the same time, the semantic relevance scores of its associated upstream and downstream evidence KV fragments are simultaneously increased by a preset amount. For example, the preset amount can be 0.2, with an overall score range of 0 to 1, ensuring the integrity of the evidence chain and preventing the wrong elimination of related fragments of key evidence.

[0029] In one implementation, the semantic relevance score is a quantitative indicator of the relevance between the retrieval results of unstructured long documents and the current query requirements. It is used to support the priority scheduling of the database's front-end cache. The calculation process of the semantic relevance score first dynamically calibrates the values ​​of the four weight coefficients α, β, γ, and δ based on the current task type to ensure that the score results conform to the core requirements of different professional review scenarios. The quantitative calculation is completed through the following formula: ; in, Let represent the semantic relevance score of the i-th cached key-value segment, and α represent the weight coefficient corresponding to the current task stage. Let represent the matching score between the i-th key-value segment and the current task stage, and β represent the historical attention weight coefficient. γ represents the cumulative historical attention value corresponding to the i-th KV segment, and γ represents the evidence chain association weight coefficient. Let represent the correlation score between the i-th key-value segment and the currently locked evidence chain, and δ represent the professional rule correlation weight coefficient. This represents the matching score between the i-th key-value segment and the associated professional rule base. All coefficients and scores range from 0 to 1, and are uniformly dimensionless. After calculation, the scores of all key-value segments are normalized to ensure that the contribution ratio of different dimensions meets the preset scenario requirements. The calibration process for each weight coefficient first calls the preset scenario weight mapping table to match the initial weight values ​​corresponding to the current task type and task stage. Then, it dynamically adjusts the weight values ​​based on the hit results of the past 20 query requests. If the hit rate of key-value segments associated with the evidence chain in the past 20 requests is lower than the preset hit rate threshold, the value of the evidence chain association weight coefficient is increased. If the hit rate of rule-related key-value segments is lower than the preset hit rate threshold, the value of the professional rule association weight coefficient is increased, ensuring the dynamic adaptability of the scoring model. For example, the hit rate range is 0 to 1, and the preset hit rate threshold can be 0.4.

[0030] In one implementation, during the dynamic cache adjustment step, when the remaining cache space is insufficient, the key-value (KV) segments to be scheduled are first sorted from low to high according to their semantic relevance scores. The KV segment with the lowest score is removed first. If the remaining space is still insufficient to store the KV segments to be recalled after removal, the KV segments to be scheduled are traversed again to perform the removal operation until the cache space meets the requirements. Throughout the process, locked high-scoring KV segments are not touched. Simultaneously, when the difference in semantic relevance scores between two KV segments is less than a preset difference threshold, the KV segment with the more recent last access time is retained first. This approach balances the access timing advantages of traditional cache replacement strategies, avoids frequent eviction of frequently accessed non-core segments, and reduces the frequency of re-recall. All semantic relevance scores are normalized to a range of 0 to 1, and the preset difference threshold can be 0.05. A score difference less than 0.05 indicates that the two segments have similar relevance.

[0031] In one implementation, when the RAG re-recall logic is triggered by the vector index module of the unstructured long document database, the current query request is first converted into a semantic vector, and then cosine similarity is calculated with the pre-computed semantic vectors of all segments in the long document segment library. The top N segments with the highest similarity are selected as the key-value segments to be recalled. The value of N has a preset range, for example, the value range is 5 to 50, and it is dynamically adjusted according to the size of the remaining cache space. If the remaining space is sufficient, N takes a larger value; if the remaining space is insufficient, N takes a smaller value. The newly recalled key-value segments are not directly written to the formal cache, but are first stored in the temporary area of ​​the cache. After the semantic relevance score is calculated, they are adjusted to the locked cache, the pending scheduling cache, or directly evicted according to the score result to avoid invalid segments occupying cache space for a long time.

[0032] In one implementation, the marking process for core evidence nodes is jointly determined based on user annotation operations and the automatic identification results of the evidence chain reasoning model. The evidence chain reasoning model adopts a pre-trained graph neural network model. The input is the semantic vector and associated attribute information of all cached key-value segments, and the output is the probability value of each key-value segment belonging to a core evidence node. When the probability value is higher than a preset probability threshold, the key-value segment is automatically marked as a core evidence node. The marking of professional rule reference nodes is determined based on the matching results of the rule number and rule reference statement contained in the key-value segment. A combination of exact matching and fuzzy matching is used. First, it matches whether the key-value segment contains a preset rule number prefix. If the match is successful, the exact matching of the rule number is performed. If the match is unsuccessful, the semantic similarity matching of the rule clause is performed. When the similarity is higher than a preset threshold, it is marked as a professional rule reference node. The marking results of the two types of nodes are synchronously updated to the attribute information of the corresponding associated key-value segments, providing a basis for subsequent scoring calculations.

[0033] A second aspect of this application provides a dynamic key-value cache replacement system based on semantic association, comprising the following modules: The task information collection module is used to connect to the retrieval request interface of the unstructured long document database, the metadata storage module, the inference output interface of the large language model, and the underlying long document storage database interface to obtain the attribute information of the current review task, the semantic features of the current query request, the attention distribution sequence generated during the historical interaction process, the semantic tags and evidence chain association attributes corresponding to the cached KV fragments. All the collected data will be standardized and converted into structured data in a unified format. The semantic relevance calculation module has a built-in preset weight mapping table, dynamic calibration logic, and quantization calculation logic. It is used to calculate the semantic relevance score of each cached KV fragment based on the collected multi-dimensional structured data. It supports multi-task parallel computing to meet the needs of high-concurrency review scenarios. The threshold comparison module has built-in configurable retention and elimination thresholds. The threshold values ​​can be customized by users according to their own business needs, or the system's preset scenario thresholds can be selected. It is used to compare the semantic relevance score of each KV segment with the preset retention and elimination thresholds respectively, and output the status label results corresponding to each KV segment. The cache scheduling module interfaces with the underlying KV cache database and supports atomic cache locking, removal, and marking operations. It locks KV fragments with semantic relevance scores higher than the retention threshold in the cache space, removes KV fragments with semantic relevance scores lower than the eviction threshold from the cache, and marks KV fragments between the two thresholds as pending scheduling. At the same time, when the remaining cache space is insufficient, it performs priority sorting and removal operations on the KV fragments to be scheduled, avoiding cache data anomalies caused by concurrent operations. The re-recall trigger module has a built-in RAG retrieval engine and connects to the underlying long document vector database. It supports high-throughput semantic similarity retrieval with retrieval latency that meets the needs of real-time interaction. When there is an uncached key-value segment to be called, it triggers the RAG re-recall logic to retrieve the target key-value segment and pushes the newly recalled key-value segment to the semantic relevance calculation module to complete the scoring calculation.

[0034] The following two examples further illustrate this point: This invention discloses a dynamic key-value (KV) cache replacement method based on semantic correlation, designed for retrieval and access scenarios of unstructured long document databases. First, it connects to the review task management module to obtain the attribute information of the current review task, including task type, current task stage, source identifier of the long document to be processed, and mapping relationship with related professional rule bases. Task types cover three categories: lawyer document review, tax audit working paper review, and contract compliance review. Different task types are automatically matched with corresponding professional rule bases, and the mapping relationship is pre-stored in the system's rule base management module. Task stages include four categories: preliminary review, evidence sorting, rule matching, and conclusion issuance. The initial stage is determined by default as preliminary review, and subsequent stages are dynamically updated based on user operations. The KV cache is a pre-retrieval cache for unstructured long document databases, using a memory-level key-value pair storage structure. The key is a unique identifier for the long document fragment, and the value is a combination of the fragment's text content, semantic vector, and attribute tags. The cache space size can be customized according to the server hardware configuration.

[0035] After the system completes the task attribute retrieval, it collects the semantic features of the current query request, the attention distribution sequence generated during historical interactions, the semantic tags corresponding to cached key-value segments, and the evidence chain association attributes. The semantic features of the query request are generated by calling a pre-trained text embedding model. The attention distribution sequence is the cumulative value of token-level attention weights generated when the large language model processes historical query requests. After each response from the large language model, the system automatically extracts the attention weights and stores them cumulatively by segment dimension. The semantic tags corresponding to key-value segments include three categories: evidence type, rule number, and related party information, which are automatically generated by the pre-annotation module when long documents are added to the database. The evidence chain association attributes include two categories: the node level of the segment in the evidence chain and the identifiers of the associated upstream and downstream evidence segments, which are gradually updated as the evidence chain is constructed during the review process.

[0036] The system calculates the semantic relevance score of each cached key-value (KV) fragment based on the collected multi-dimensional data. Before calculation, the weight coefficients corresponding to each dimension are dynamically calibrated based on the current task type and task stage. The initial values ​​of the weight coefficients are pre-stored in the scene weight mapping table and can be adaptively adjusted based on historical query hits. After calculation, the semantic relevance score of each KV fragment is compared with the preset retention threshold and eviction threshold. Users can customize the adjustment according to their own business needs or directly select the preset scene threshold template.

[0037] The system locks key-value (KV) fragments with semantic relevance scores higher than the retention threshold in the cache. Locked fragments are not removed by any scheduling operations; batch cleanup is only triggered after the corresponding review task is completed. The system directly removes KV fragments with semantic relevance scores lower than the eviction threshold from the cache, freeing up the occupied cache space. The system marks KV fragments with scores between the two thresholds as pending scheduling and stores them in the cache's pending scheduling area. When the remaining cache space is insufficient and there are uncached KV fragments awaiting access, the system first sorts the KV fragments in the pending scheduling area by score from lowest to highest, prioritizing the removal of the fragment with the lowest score. If the score difference between two fragments is less than 0.05, the fragment with the more recent last access time is retained, until the cache space meets the demand.

[0038] After scheduling is completed, the RAG re-recall logic is triggered to retrieve the target KV fragment. The re-recall logic connects to the vector index module of the unstructured long document database and returns the top N fragments with the highest similarity. The number of returned fragments is dynamically adjusted according to the remaining space in the current cache. The newly recalled KV fragments are first stored in the temporary area of ​​the cache, and after the semantic relevance score is calculated, they are moved into the corresponding cache area.

[0039] This invention also discloses the dynamic identification logic for task stages. The system collects user interaction operation sequences in real time, including four categories: annotation operations, query operations, jump operations, and export operations. Every five consecutive operations collected, a pre-trained operation classification model is invoked for feature matching. Each review process node corresponds to a set of operation feature combinations. When the matching degree reaches 0.8 or higher, the current task stage is determined and updated to the corresponding node. Different task stages correspond to different weight ratios: the evidence chain association weight coefficient is the highest for the evidence sorting stage, the professional rule association weight coefficient is the highest for the rule matching stage, the task matching degree weight coefficient is the highest for the preliminary review stage, and the historical attention weight coefficient is the highest for the conclusion issuance stage.

[0040] This invention also discloses a priority locking logic for key evidence (KV) fragments. When a KV fragment is marked as a core evidence node or a professional rule reference node, its semantic relevance score is directly assigned the highest value, and it is continuously retained in the cache regardless of insufficient cache space. When a user performs operations such as marking a KV fragment as core evidence, referencing it to the review conclusion, or associating it with a rule clause, the system automatically raises the priority of that KV fragment to the highest level, and simultaneously increases the semantic relevance scores of its associated upstream and downstream evidence KV fragments by a preset amount.

[0041] The marking process for core evidence nodes is jointly determined based on user annotation operations and the automatic identification results of the evidence chain reasoning model. The evidence chain reasoning model uses a pre-trained graph neural network, trained on a large number of labeled professional review case samples. The input is the semantic vector and related attribute information of all cached key-value (KV) fragments, and the output is the probability value of each KV fragment belonging to a core evidence node. When the probability value is higher than 0.7, the KV fragment is automatically marked as a core evidence node. The marking of professional rule reference nodes is determined based on the matching results of the rule number and rule reference statement contained in the KV fragment. It adopts a combination of exact matching and fuzzy matching. First, it matches whether the KV fragment contains a preset rule number prefix. If the match is successful, it performs an exact matching of the rule number. If the match is unsuccessful, it performs semantic similarity matching of the rule clause. When the similarity is higher than 0.85, it is marked as a professional rule reference node.

[0042] This invention discloses a dynamic key-value cache replacement system based on semantic relevance. The system operates according to the aforementioned method logic and includes a task information collection module, a semantic relevance calculation module, a threshold comparison module, a cache scheduling module, and a re-recall triggering module. The task information collection module interfaces with the retrieval request interface of the unstructured long document database, the metadata storage module, the inference output interface of the large language model, and the underlying long document storage database interface. All collected data undergoes standardization processing, converting it into structured data in a unified format. The semantic relevance calculation module incorporates a preset weight mapping table, dynamic calibration logic, and quantization calculation logic, supporting multi-task parallel computation to meet the needs of high-concurrency review scenarios.

[0043] The threshold comparison module has built-in configurable retention and eviction thresholds. The threshold values ​​can be user-defined or selected from system-preset scenario thresholds. The cache scheduling module interfaces with the underlying KV cache database; all operations are atomic, supporting batch locking, removal, and marking to avoid cache data anomalies caused by concurrent operations. The re-recall trigger module has a built-in RAG retrieval engine, interfaces with the underlying long document vector database, and the retrieval latency meets the requirements of real-time interaction. The system corresponding to this invention can be deployed on general-purpose server equipment; the processor can run it by loading the corresponding instructions stored in the memory, and the corresponding computer program can be distributed by storing it in general-purpose computer-readable storage media.

[0044] The first embodiment is applied to a criminal case file review scenario. A law firm is undertaking a criminal case file review task. The case files to be processed include 1200 pages of scanned documents, such as investigation files, interrogation records, expert opinions, and documentary evidence. After OCR recognition, these are stored in an unstructured long document database. The associated professional rule base is a database of articles from the Criminal Law, the Criminal Procedure Law, and related judicial interpretations. The lead lawyer accesses the corresponding review system of this invention through a terminal device to complete the file review. After the system starts, it first retrieves the attribute information of the current task, identifies the current task type as lawyer file review, and the initial task stage as preliminary review. It calls the scene weight mapping table to match the initial weight ratio, with the task matching degree weight set to the highest value. The retention threshold is set to 0.75, and the elimination threshold is set to 0.3. After initialization, it waits for user operation.

[0045] The lead attorney first inputs a query request regarding the circumstances of the arrest of the individuals involved in the case. The system first checks the cache and finds no corresponding segment. It then triggers the RAG re-recall logic to retrieve the corresponding interrogation transcript segment and store it in the temporary cache area. After completing the semantic relevance score calculation, it moves the segment to the regular cache area. During the process of generating the response using the large language model, the system automatically extracts attention weights and accumulates them into the attention attributes of the corresponding segment.

[0046] After the lead lawyer completed the initial review, they began to analyze the case's evidence chain. The system collected data on the lawyer's continuous actions of marking evidence, jumping to related segments, and exporting the evidence list. The feature combination of these five consecutive actions matched the feature combination of the evidence analysis stage with a score of 0.87. The system automatically updated the current task stage to the evidence analysis stage and adjusted the weight allocation to the highest value for the evidence chain association weight coefficient. The evidence chain reasoning model ran synchronously, inputting the semantic vectors and association attributes of all cached key-value segments. It identified segments about the process of arriving at the scene, injury assessment opinions, and eyewitness testimonies as core nodes in the same evidence chain, with probability values ​​exceeding 0.7. These were automatically marked as core evidence nodes and locked in the cache. The semantic relevance scores of the related upstream and downstream interrogation records and case registration forms were simultaneously increased by a preset amount, and these segments were moved from the waiting area to the locked area.

[0047] When the lead attorney subsequently inputs a query requesting whether the person involved in the case has surrendered themselves to the authorities, the system directly retrieves the core evidence fragments already locked in the cache and the relevant criminal law surrender clause fragments to generate a response, without needing to retrieve the content from the underlying database again. When the cache space reaches a threshold during the review process, the system prioritizes removing procedural document fragments unrelated to the current evidence chain from the area to be scheduled, without touching the locked core content. All caching operations are atomic operations, and there are no concurrency conflicts.

[0048] In this embodiment, the system can automatically adjust the cache scheduling logic according to different stages of the lawyer's case file review, no longer eliminating content based on time-series attributes. The cache hit rate is significantly improved compared to the traditional LRU strategy, and retrieval latency is reduced. The locking mechanism for core evidence fragments ensures the complete retention of evidence chain-related content, avoiding evidence gaps during large language model reasoning. Lawyers do not need to repeatedly manually recall relevant content, making the operation process smoother. The entire caching strategy requires no additional manual parameter configuration, directly adapts to the business needs of criminal case file review, does not require changes to the underlying architecture of the original case file database, and can be directly deployed to existing case file review systems, reducing system optimization costs.

[0049] The second embodiment is applied to the scenario of reviewing annual tax audit working papers for a company. An accounting firm is undertaking an annual tax audit of a manufacturing company. The audit working papers to be processed include 1800 pages of electronic documents, such as the company's accounting vouchers, VAT tax returns, annual corporate income tax settlement statements, cost accounting details, and purchase and sales contracts for the entire year. All documents are stored in an unstructured long document database. The associated professional rule base includes the Corporate Income Tax Law, the Provisional Regulations on Value-Added Tax, and related tax regulatory documents. Auditors access the corresponding review system of this invention through terminal devices to complete the working paper review. After the system starts, it first retrieves the attribute information of the current task, identifies the current task type as tax audit working paper review, and the initial task stage is preliminary review. It calls the scenario weight mapping table to match the initial weight ratio, with the task matching degree weight set to the highest value. The retention threshold is set to 0.7, and the elimination threshold is set to 0.35. After initialization, it waits for user operation.

[0050] Auditors first input a query request for the total amount of R&D expenses of the company in 2023. The system first checks that there is no corresponding segment in the cache, triggers the RAG recall logic to retrieve the corresponding cost accounting details and R&D expense auxiliary account segments and store them in the temporary cache area. After completing the semantic relevance score calculation, they are moved to the regular cache area. During the process of generating the response by the large language model, the system automatically extracts the attention weight and accumulates it into the attention attribute of the corresponding segment.

[0051] After the auditors completed the initial review, they began to verify the compliance of the R&D expense listing. The system collected data from the auditors' continuous execution of the operation of referencing rule clauses, comparing voucher data, and marking non-compliant items. The feature combination of the five consecutive operations reached a matching degree of 0.82 with the feature combination of the rule matching stage. The system automatically updated the current task stage to the rule matching stage and adjusted the weight ratio to the highest value of the professional rule association weight coefficient.

[0052] The system synchronously identifies professional rule reference nodes, matches R&D expense subsidiary ledger fragments containing rule number prefixes, accurately matches the corresponding relevant articles of the Corporate Income Tax Law, marks them as professional rule reference nodes, and locks them in the cache. Simultaneously, the semantic relevance scores of associated R&D personnel hour accounting and R&D equipment procurement contract fragments increase by a preset margin, moving them from the pending scheduling area to the locked area. When auditors subsequently input queries requesting whether the company's R&D expense deduction application amount complies with regulations, the system directly calls the locked core evidence fragments and associated tax rule fragments in the cache to generate a response, without needing to retrieve content from the underlying database. During the review process, when the cache space reaches a threshold, the system prioritizes removing administrative and logistical document fragments unrelated to the current review item from the pending scheduling area, without touching the locked core content. All cache operations are atomic, and there are no concurrency conflicts.

[0053] In this embodiment, the system can automatically adjust the cache scheduling logic according to different stages of tax auditing, no longer evicting content based on time-series attributes. The cache hit rate is significantly improved compared to traditional semantic caching strategies, and retrieval latency is reduced. The locking mechanism for professional rule reference nodes ensures the complete retention of rule clauses and corresponding audit evidence, avoiding rule reference errors during large language model inference. Auditors do not need to repeatedly manually recall relevant content, making the operation process smoother. The entire caching strategy requires no additional manual parameter configuration, directly adapts to the business needs of tax auditing, does not require changes to the underlying architecture of the original audit working paper database, and can be directly deployed to existing audit systems, reducing system optimization costs.

[0054] Figure 1 This is a schematic diagram of the dynamic key-value cache replacement method based on semantic relevance proposed in this invention. The steps are as follows: obtain the attribute information of the current review task; collect query requests and multi-dimensional cache features; calculate the semantic relevance score of the cached fragments; compare the score with the retention and eviction thresholds respectively; perform three-way scheduling according to the comparison results: if it is lower than the eviction threshold, it is removed from the cache space; if it is higher than the retention threshold, it is locked in the cache space; if it is between the two thresholds, it is marked as a pending scheduling state, and dynamic adjustment and recall are triggered when space is insufficient.

[0055] Figure 2This is a schematic flowchart of the task stage dynamic identification and weight calibration process proposed in this invention. The steps are as follows: collect user interaction operation sequences and completed nodes; compare the matching degree with the process node feature combination; update the current task stage identifier in real time; call the scene weight mapping table to match the initial weight; dynamically calibrate the weight by combining the historical request hit rate; and output the correlation coefficient specific to the current stage.

[0056] Figure 3 This is a flowchart of the key evidence priority locking and cascading enhancement process proposed in this invention. The steps are as follows: user operation or graph neural network model identification; determining whether it is core evidence or rule reference; if so, directly assigning the highest value to the semantic relevance score; continuously locking and retaining it regardless of cache space limitations; synchronously increasing the score of related upstream and downstream segments; until the corresponding review task is completed and cleanup is triggered.

[0057] Figure 4 This is a flowchart of the dynamic adjustment and recall process when cache space is insufficient, as proposed in this invention. The steps are as follows: trigger a cache space shortage alarm; sort the segments to be scheduled from lowest to highest score; remove the segment with the lowest score first; determine whether the cache space is sufficient: if not, return to the step of sorting the segments to be scheduled from lowest to highest score; if so, convert the new query request into a semantic vector; calculate the similarity with the segment library to determine the recall quantity; store the newly recalled segments in a temporary area and calculate their scores.

[0058] Figure 5 A structural block diagram of an electronic device according to an embodiment of the present invention is shown. Figure 5 As shown, the electronic device includes a memory 510 and a processor 520. The memory 510 stores a computer program that can run on the processor 520. When the processor 520 executes the computer program, it implements the sensitive data physical isolation protection method in edge-side large model inference in the above embodiments. The number of memories 510 and processors 520 can be one or more.

[0059] The electronic device also includes: The communication interface 530 is used to communicate with external devices and exchange and transmit data.

[0060] If the memory 510, processor 520, and communication interface 530 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0061] Optionally, in a specific implementation, if the memory 510, processor 520, and communication interface 530 are integrated on a single chip, then the memory 510, processor 520, and communication interface 530 can communicate with each other through an internal interface.

[0062] This invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this application.

[0063] This application also provides a chip, which includes a processor for calling and executing instructions stored in a memory, causing a communication device on which the chip is installed to perform the method provided in this application.

[0064] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0065] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting the Advanced Reduced Instruction Set Computing (RISC) machine (ARM) architecture.

[0066] Further, optionally, the aforementioned memory may include read-only memory and random access memory, and may also include non-volatile random access memory. The memory may be volatile or non-volatile, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. Many forms of RAM are available by way of example, but not limitation. Examples include static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

[0067] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0068] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0069] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0070] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0071] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0072] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0073] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0074] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A dynamic key-value cache replacement method based on semantic association, characterized in that, Includes the following steps: For retrieval and access scenarios of unstructured long document databases, obtain the attribute information of the current review task, including task type, current task stage, source identifier of the long document to be processed, and mapping relationship with related professional rule bases; The system collects the semantic features of the current query request, the attention distribution sequence generated during historical interactions, the semantic labels corresponding to the cached key-value segments, and the evidence chain association attributes. The attention distribution sequence is the cumulative value of the token-level attention weights generated when the large language model processes historical query requests. The semantic labels corresponding to the key-value segments include three categories: evidence type, rule number, and related party information. The evidence chain association attributes include two categories: the node level of the segment in the evidence chain and the identifiers of the associated upstream and downstream evidence segments. The semantic relevance score of each cached key-value segment is calculated based on the collected multi-dimensional data; the semantic relevance score of each key-value segment is compared with the preset retention threshold and elimination threshold respectively; Key-value segments with semantic relevance scores higher than the retention threshold are locked in the cache space, while key-value segments with semantic relevance scores lower than the eviction threshold are removed from the cache. When the remaining cache space is insufficient and there are uncached key-value segments waiting to be called, the RAG recall logic is triggered to retrieve the target key-value segment, and at the same time, the key-value segments in the scheduling state are dynamically adjusted after priority sorting.

2. The dynamic KV cache replacement method based on semantic association as described in claim 1, characterized in that, It also includes dynamic identification logic for task stages. Specifically, it updates the current task stage identifier in real time based on user interaction operation sequences, completed review node markers, and node matching rules of the professional review process. User interaction operation sequences include four categories: annotation operations, query operations, jump operations, and export operations. Each review process node corresponds to a set of operation feature combinations. When the matching degree between the collected operation sequence and the feature combination of a certain node reaches a preset matching threshold, the current task stage is determined to be updated to the corresponding node. Different task stages correspond to different retention thresholds and elimination thresholds, and different weight coefficients are used for each dimension when calculating the corresponding semantic relevance score. The evidence chain association weight coefficient is the highest for the evidence sorting stage, the professional rule association weight coefficient is the highest for the rule matching stage, the task matching degree weight coefficient is the highest for the preliminary review stage, and the historical attention weight coefficient is the highest for the conclusion issuance stage.

3. The dynamic KV cache replacement method based on semantic association as described in claim 1, characterized in that, It also includes priority locking logic for key evidence (KV) fragments. When a KV fragment is marked as a core evidence node or a professional rule reference node, its semantic relevance score is directly assigned the highest value and it is continuously retained in the cache regardless of insufficient cache space. It is only triggered to be cleaned up after the corresponding review task is completed. When a user performs an operation on a KV fragment, such as marking it as core evidence, referencing it to the review conclusion, or associating it with a rule clause, the priority of that KV fragment is raised to the highest, and the semantic relevance scores of its associated upstream and downstream evidence KV fragments are simultaneously increased by a preset amount.

4. The dynamic key-value cache replacement method based on semantic association as described in claim 1, characterized in that, The semantic relevance score is a quantitative indicator of the relevance between the retrieval results of unstructured long documents and the current query requirements. It is used to support the priority scheduling of the database's front-end cache. The calculation process of the semantic relevance score first dynamically calibrates the values ​​of the four weight coefficients α, β, γ, and δ based on the current task type, and then completes the quantitative calculation using the following formula: ； in, Let represent the semantic relevance score of the i-th cached key-value segment, and α represent the weight coefficient corresponding to the current task stage. Let represent the matching score between the i-th key-value segment and the current task stage, and β represent the historical attention weight coefficient. γ represents the cumulative historical attention value corresponding to the i-th KV segment, and γ represents the evidence chain association weight coefficient. Let represent the correlation score between the i-th key-value segment and the currently locked evidence chain, and δ represent the correlation weight coefficient of the professional rules. This represents the matching score between the i-th KV segment and the associated professional rule base. The calibration process for each weight coefficient first calls the preset scene weight mapping table to match the initial weight value corresponding to the current task type and task stage, and then dynamically adjusts the weight value based on the hit results of the past 20 query requests.

5. The dynamic KV cache replacement method based on semantic association as described in claim 1, characterized in that, In the dynamic cache adjustment process, when the remaining cache space is insufficient, the key-value segments to be scheduled are sorted from low to high according to their semantic relevance scores. The key-value segment with the lowest score is removed first. If the remaining space is still insufficient to store the key-value segments to be recalled after removal, the key-value segments to be scheduled are traversed again and the removal operation is performed until the cache space meets the requirements. At the same time, when the difference in semantic relevance scores between two key-value segments is less than the preset difference threshold, the key-value segment with the more recent last access time is retained first.

6. The dynamic KV cache replacement method based on semantic association as described in claim 1, characterized in that, When the RAG re-recall logic is triggered by the vector index module of the unstructured long document database, the current query request is first converted into a semantic vector. Then, cosine similarity is calculated between the vector vector and the pre-computed semantic vector of all segments in the long document segment library. The top N segments with the highest similarity are selected as the key-value segments to be recalled. The value of N is dynamically adjusted according to the size of the remaining cache space. If the remaining space is sufficient, N takes a larger value; if the remaining space is insufficient, N takes a smaller value. The newly recalled key-value segments are first stored in the temporary area of ​​the cache, and then adjusted to the corresponding cache area after the semantic relevance score is calculated.

7. The dynamic KV cache replacement method based on semantic association as described in claim 3, characterized in that, The marking process for core evidence nodes is determined jointly by user annotation operations and the automatic identification results of the evidence chain reasoning model. The evidence chain reasoning model adopts a pre-trained graph neural network model. The input is the semantic vector and associated attribute information of all cached KV fragments, and the output is the probability value of each KV fragment belonging to a core evidence node. When the probability value is higher than the preset probability threshold, the KV fragment is automatically marked as a core evidence node. The marking of professional rule reference nodes is determined based on the matching results of the rule number and rule reference statement contained in the KV fragment. It adopts a combination of exact matching and fuzzy matching. First, it matches whether the KV fragment contains a preset rule number prefix. If the match is successful, it performs exact matching of the rule number. If the match is unsuccessful, it performs semantic similarity matching of the rule clause.

8. A dynamic key-value cache replacement system based on semantic association, characterized in that, Includes the following modules: The task information collection module is used to connect to the retrieval request interface of the unstructured long document database, the metadata storage module, the inference output interface of the large language model, and the underlying long document storage database interface to obtain the attribute information of the current review task, the semantic features of the current query request, the attention distribution sequence generated during the historical interaction process, the semantic tags and evidence chain association attributes corresponding to the cached KV fragments. The semantic relevance calculation module has a built-in preset weight mapping table, dynamic calibration logic, and quantization calculation logic. It is used to calculate the semantic relevance score of each cached KV segment based on the collected multi-dimensional structured data and supports multi-task parallel computing. The threshold comparison module has built-in configurable retention and elimination thresholds. The threshold values ​​can be customized by users according to their own business needs, or the system's preset scenario thresholds can be selected. It is used to compare the semantic relevance score of each KV segment with the preset retention and elimination thresholds respectively, and output the status label results corresponding to each KV segment. The cache scheduling module interfaces with the underlying KV cache database and supports atomic cache locking, removal, and marking operations. It is used to lock KV fragments with semantic relevance scores higher than the retention threshold in the cache space, remove KV fragments with semantic relevance scores lower than the eviction threshold from the cache, mark KV fragments between the two thresholds as pending scheduling, and perform priority sorting and removal operations on pending KV fragments when the remaining cache space is insufficient. The re-recall trigger module has a built-in RAG retrieval engine and connects to the underlying long document vector database. It supports high-throughput semantic similarity retrieval with retrieval latency that meets the needs of real-time interaction. When there is an uncached key-value segment to be called, it triggers the RAG re-recall logic to retrieve the target key-value segment and pushes the newly recalled key-value segment to the semantic relevance calculation module to complete the scoring calculation.

9. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores instructions that are loaded and executed by the processor to implement the dynamic key-value cache replacement method based on semantic association as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the dynamic key-value cache replacement method based on semantic association as described in any one of claims 1 to 7.

Images