A data management method and related equipment based on sentence semantic perception

Abstract

The present invention provides a data management method and related equipment based on sentence semantic perception, which relates to the field of cache management technology. The method includes segmenting a long text to obtain multiple sentences and their sorting order; calculating the importance scores of tokens corresponding to the sentences based on the sentences and their sorting order; screening out key tokens in the sentences according to the importance scores of the tokens corresponding to the sentences and the dynamic budget corresponding to the sentences, and storing them in a reserved set of the corresponding sentences, and then calculating the local weighted attention score of each token; performing weighted fusion based on the local weighted attention score of each token in the sentence and the fusion weight of the preset attention head to obtain a sentence-level semantic vector; and constructing a semantic index cache and a key-value pair (KV) content cache based on the sentence-level semantic vector of the sentence and the corresponding reserved set. In this way, a KV cache management method with context-dependent dynamic loading is combined to achieve a more efficient and intelligent reasoning process.

Description

Technical Field

[0001] The present invention relates to the field of cache management technology, and in particular to a data management method based on sentence semantic perception and related equipment. Background Art

[0002] With the widespread application of large language models (LLMs) in natural language processing tasks, they have demonstrated remarkable capabilities in scenarios such as long text generation, multi-turn conversations, and document understanding. However, these models rely on key-value (KV) caching to maintain contextual semantic consistency during inference.

[0003] Existing KV cache management methods are often based on token-level granularity and fail to effectively utilize syntactic and semantic structures, resulting in fragmented contextual information and insufficient semantic representation capabilities. At the same time, they rely on external clustering or auxiliary models, resulting in high computational overhead and poor real-time performance, making it difficult to meet online reasoning and low-latency requirements.

[0004] In this context, although technologies such as fixed windows and dynamic cropping have been used to try to optimize the use of KV cache, fixed windows easily cut off long-distance dependency information, affecting semantic coherence; dynamic cropping, although more flexible, requires continuous calculation of attention scores during the generation process, increasing the system burden.

[0005] Therefore, there is an urgent need for a KV cache management method that has sentence-level semantic perception capabilities, can adaptively retain important context, eliminate redundant content, and combine context-dependent dynamic loading to achieve a more efficient and intelligent reasoning process. Summary of the Invention

[0006] In view of this, an embodiment of the present invention provides a data management method and related equipment based on sentence semantic perception, combined with a KV cache management method of context-dependent dynamic loading, to achieve a more efficient and intelligent reasoning process.

[0007] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0008] A first aspect of an embodiment of the present invention provides a data management method based on sentence semantic perception, the method comprising:

[0009] If a text cache instruction input by the user is received, the long text carried by the text cache instruction is segmented to obtain multiple sentences and their sorting order;

[0010] For each sentence, calculate the importance score of the token corresponding to the sentence based on the sentence and its sort order;

[0011] For each sentence, based on the importance score of the token corresponding to the sentence and the dynamic budget corresponding to the sentence, the key tokens in the sentence are screened out and stored in the reserved set of the corresponding sentence;

[0012] For each sentence's retention set, calculate a local weighted attention score for each token based on all key tokens in the retention set;

[0013] Performing weighted fusion based on the local weighted attention score of each token in the sentence and the fusion weight of the preset attention head to obtain a sentence-level semantic vector;

[0014] A semantic index cache and a key-value pair KV content cache are constructed based on the sentence-level semantic vector of the sentence and the corresponding retention set.

[0015] Optionally, the long text carried by the text cache instruction is segmented to obtain multiple sentences and their sorting order, including:

[0016] Preprocessing the long text carried by the text cache instruction to obtain a preprocessed long text;

[0017] If the pre-processed long text contains a preset symbol, dividing the long text into a plurality of sentences and sorting them according to the preset end symbol;

[0018] If the preprocessed long text does not contain preset symbols, the preset segmentation model is called to segment the long text to obtain multiple sentences and their sorting order, wherein the preset segmentation model is trained based on historical long texts and corresponding sentences.

[0019] Optionally, for each sentence, an importance score of a token corresponding to the sentence is calculated based on the sentence and its sort order, including:

[0020] Determining a current sentence to be processed based on the plurality of sentences and their sorting order;

[0021] For the sentence to be processed, generating a corresponding token based on features in the sentence, where the number of the token is at least one;

[0022] Calculating a target vector for the sentence to be processed based on the tokens within a preset context and the semantic vectors of the tokens;

[0023] For each token of the sentence to be processed, calculate the similarity between the token and the target vector;

[0024] For each token, an importance score of the token is calculated based on the token, the target vector of the sentence to which the token corresponds, and the similarity between the token and the target vector.

[0025] Optionally, for each sentence, based on the importance score of the token corresponding to the sentence and the dynamic budget corresponding to the sentence, the key tokens in the sentence are screened out and stored in the reserved set of the corresponding sentence, including:

[0026] Selecting tokens based on the importance scores of the tokens and sentence weights of the sentences to construct an initial retention set, wherein the sentence weights are the average of the importance scores of each token corresponding to the sentence;

[0027] Optimizing the importance score of each token based on the initial retention set and the importance score of the token to obtain a target importance score for each token;

[0028] For each sentence, processing is performed based on the target importance score of each token in the sentence and the tokens of the sentence to obtain a sentence importance weight of the sentence;

[0029] Determining a dynamic budget allocated to each of the sentences based on the sentence importance weight of each of the sentences and a preset total dynamic budget;

[0030] For each sentence, based on the sentence and its corresponding dynamic budget, key tokens are filtered out from the sentence and written into the corresponding reserved set.

[0031] Optionally, weighted fusion is performed based on the local weighted attention score of each token in the sentence and the fusion weight of the preset attention head to obtain a sentence-level semantic vector, including:

[0032] For each attention head in the model, weighted fusion of each key vector based on the local weighted attention score of each token in the sentence is performed to obtain the sentence vector under each attention head. The key vector refers to the key vector of each token under the attention head.

[0033] The sentence vector under each attention head and the fusion weight of the preset attention head are weightedly fused to obtain the sentence-level semantic vector of the sentence.

[0034] Optionally, also include:

[0035] If it is determined that the current stage is decoding, obtaining the current generated content, and processing the current generated content to generate a corresponding joint score;

[0036] The combined score is used to search from the semantic index cache and the key-value pair KV content cache to load the candidate sentence set and the key-value pair KV corresponding to the currently generated content into the corresponding memory.

[0037] Optionally, using the joint score to search from the semantic index cache and the key-value pair KV content cache to load the candidate sentence set and key-value pair KV corresponding to the currently generated content into the corresponding memory, including:

[0038] Retrieving a set of candidate sentences from the semantic index cache according to the joint score;

[0039] Traversing the key-value pair KV content cache according to the identifier of each sentence in the candidate sentence set to determine the corresponding key-value pair KV;

[0040] The candidate sentence set and key-value pairs KV are stored in the corresponding memory.

[0041] A second aspect of an embodiment of the present invention provides a data management device based on sentence semantic perception, the device comprising:

[0042] a segmentation unit configured to, upon receiving a text cache instruction input by a user, segment the long text carried in the text cache instruction to obtain a plurality of sentences and their sorting order;

[0043] a calculation unit, configured to calculate, for each sentence, an importance score of a token corresponding to the sentence based on the sentence and its sorting order;

[0044] a screening unit configured to screen out key tokens in each sentence according to the importance scores of the tokens corresponding to the sentence and the dynamic budget corresponding to the sentence, and store the key tokens in the retained set of the corresponding sentence;

[0045] a processing unit configured to calculate, for a retained set of each sentence, a local weighted attention score of each token based on all key tokens in the retained set; and perform weighted fusion based on the local weighted attention score of each token in the sentence and a fusion weight of a preset attention head to obtain a sentence-level semantic vector;

[0046] The cache unit is used to build a semantic index cache and a key-value pair KV content cache based on the sentence-level semantic vector of the sentence and the corresponding retention set.

[0047] The third aspect of an embodiment of the present invention shows an electronic device, which includes a processor and a memory, wherein the memory is used to store program code and data for data management based on sentence semantic perception, and the processor is used to call the program instructions in the memory to execute the data management method based on sentence semantic perception as shown in the first aspect of the embodiment of the present invention.

[0048] The fourth aspect of an embodiment of the present invention shows a storage medium, which includes a storage program, wherein when the program is running, the device where the storage medium is located is controlled to execute the data management method based on sentence semantic perception as shown in the first aspect of the embodiment of the present invention.

[0049] Based on the above-mentioned embodiment of the present invention, a data management method based on sentence semantic perception and related equipment are provided, the method including: if a text cache instruction input by a user is received, the long text carried by the text cache instruction is segmented to obtain multiple sentences and their sorting order; for each sentence, the importance score of the token corresponding to the sentence is calculated based on the sentence and its sorting order; for each sentence, according to the importance score of the token corresponding to the sentence and the dynamic budget corresponding to the sentence, the key tokens in the sentence are screened out and stored in the retention set of the corresponding sentence; for the retention set of each sentence, the local weighted attention score of each token is calculated based on all the key tokens in the retention set; weighted fusion is performed based on the local weighted attention score of each token in the sentence and the fusion weight of the preset attention head to obtain a sentence-level semantic vector; based on the sentence-level semantic vector of the sentence and the corresponding retention set, a semantic index cache and a key-value pair KV content cache are constructed. In an embodiment of the present invention, the input long text is first semantically parsed and segmented into sentences, and the importance score of each sentence token is calculated. Then, by combining context-adaptive weights, semantic similarity, and an attention mechanism, the key tokens corresponding to each sentence are dynamically screened and stored in a reserved set of the corresponding sentence. On this basis, a sentence-level semantic vector is constructed, and a multi-scale attention mechanism is used to generate the sentence-level semantic vector. Then, a corresponding semantic index cache and KV content cache are constructed to uniformly store the sentence-level semantic vectors of all sentences in a semantic index structure in the GPU memory, which serves as the basis for rapid semantic retrieval in the subsequent decoding stage. This is combined with a context-dependent dynamic loading KV cache management method to achieve a more efficient and intelligent reasoning process. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0051] Figure 1 A flowchart of a data management method based on sentence semantic perception is shown in an embodiment of the present invention;

[0052] Figure 2 This is a flow chart of another data management method based on sentence semantic perception according to an embodiment of the present invention;

[0053] Figure 3 The figure is a structural diagram of a data management device based on sentence semantic perception according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0055] The terms "first," "second," "third," "fourth," and the like (if any) in the specification and claims of this application and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequential sequence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions, e.g., a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0056] It should be noted that the descriptions of "first", "second", etc. in the present invention are for descriptive purposes only and should not be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0057] In this application, the terms "comprises," "comprising," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0058] This embodiment of the present invention demonstrates a management method based on sentence semantics awareness, aiming to improve the inference efficiency and memory utilization of large language models (LLMs) in long text tasks while ensuring the semantic consistency of generated content. Specifically, in the caching phase, the input long text is first semantically parsed and segmented, using sentences as the basic unit, and the importance score of each sentence token is calculated. Then, by combining context-adaptive weights, semantic similarity, and an attention mechanism, key tokens are dynamically screened, and a diversity penalty strategy is introduced to optimize the retained content. On this basis, a sentence-level semantic vector is constructed, and a semantic representation is generated using a multi-scale attention mechanism. This is followed by the construction of a corresponding semantic index cache and KV content cache. In the decoding phase, the corresponding key-value pairs (KVs) are loaded from the cache on demand based on the semantic vector of the currently generated content, forming a lightweight semantic support set. A dynamic eviction strategy and asynchronous preloading mechanism are then used to achieve efficient GPU cache management. This enables semantically driven, fine-grained KV cache control, significantly improving performance and context retention during long text processing.

[0059] See also Figure 1 , is a flow chart illustrating a sentence semantics-aware cache management method according to an embodiment of the present invention, the method comprising:

[0060] Step S101: When a text cache instruction input by a user is received, the long text carried in the text cache instruction is segmented to obtain a plurality of sentences and their sorting order.

[0061] Optionally, after the user inputs the long text, the corresponding text cache instruction is triggered so that the text cache instruction carries the long text input by the user.

[0062] It should be noted that the specific process of implementing step S101 includes the following steps.

[0063] Step S11: pre-processing the long text carried by the text cache instruction to obtain a pre-processed long text.

[0064] In the specific implementation of step S11 , the long text is traversed according to the preset irrelevant symbols and preset special characters. If it is determined that the corresponding preset irrelevant symbols and preset special characters exist in the long text, they are deleted to obtain a preprocessed long text.

[0065] It should be noted that the preset irrelevant symbols and preset special characters are pre-set. The preset irrelevant symbols include extra spaces, line breaks, etc.; the preset special characters include emoticons, URLs, etc.

[0066] Optionally, preset special characters can be retained or deleted according to actual needs.

[0067] Step S12: Determine whether the pre-processed long text contains a preset symbol. If so, execute step S13; otherwise, execute step S14.

[0068] In the specific implementation of step S12, the long text is traversed to determine whether the long text contains the preset symbol. If it does, step S13 is executed; otherwise, step S14 is executed.

[0069] It should be noted that the preset symbols are set in advance according to actual conditions and can generally be punctuation marks.

[0070] Step S13: Segment the long text into multiple sentences and sort them in order according to preset end symbols.

[0071] It should be noted that the preset end symbol is set in advance based on experience or multiple experiments, and can generally be a period “.”, a question mark “?”, etc.

[0072] For example, for an input long text A "The weather is nice today. I went for a walk in the park.", the long text A is split into two sentences according to a preset end symbol, specifically including sentence 1: The weather is nice today. Sentence 2: I went for a walk in the park, where 1 and 2 represent the sorting order of the sentences.

[0073] Among them, sentence 1 and sentence 2 are sorted according to the text order, that is, the order of sentence 1 is 1, the order of sentence 2 is 2, and sentence 1 is before sentence 2.

[0074] Step S14: calling a preset segmentation model to segment the long text to obtain multiple sentences and their sorting order.

[0075] Among them, the preset segmentation model is trained based on historical long texts and corresponding sentences.

[0076] It should be noted that the process of training a segmentation model based on historical long texts and corresponding sentences includes:

[0077] Acquire historical long texts and corresponding sentences as samples; divide the samples into a training set and a test set; train the initial model based on the training set to obtain a trained initial model; use the trained initial model to understand the context of the long text in the test set in the sample to infer the start and end positions of the sentence, thereby obtaining the corresponding sentence; determine whether the obtained sentence is consistent with the sentence corresponding to the long text in the test set; if so, use the trained initial model as the segmentation model; otherwise, continue to train the initial model using the training set.

[0078] It should be noted that the initial model is the natural language processing BERT model.

[0079] In the specific implementation of step S14, the long text is contextually understood using the preset segmentation model to infer the start and end positions of the sentences, thereby obtaining multiple sentences and the order in which the sentences are sorted.

[0080] In this application, for each sentence, each sentence is treated as an independent unit for subsequent token importance calculation and semantic vector calculation.

[0081] Step S102: For each sentence, calculate the importance score of the token corresponding to the sentence based on the sentence and its sorting order.

[0082] It should be noted that the specific process of implementing step S102 includes:

[0083] Step S21: Based on the plurality of sentences and their sorting order, determine the sentence to be processed currently.

[0084] In the specific implementation of step S21 , the sentences need to be executed in the sorted order, the executed sentences are marked as executed, and the sentences to be processed are determined in order from the smallest to the largest in the sorted order.

[0085] Step S22: for the sentence to be processed currently, generate corresponding tokens based on the features in the sentence, and the number of the tokens is at least one.

[0086] In the specific implementation of step S22, the sentences are divided into words, phrases, symbols or other features, namely tokens, in sequence according to the order of the sentences in the text.

[0087] For example, the sentence "I like cats." might be segmented into a list of tokens like ["I", "like", "cats"]. In the English sentence "I like cats.", tokenization might produce ["I", "like", "cats", "."], where even punctuation marks might be considered separate tokens.

[0088] Step S23: Calculate the target vector of the sentence to be processed based on the tokens within the preset context and the semantic vectors of the tokens.

[0089] It should be noted that the preset context range refers to the range from the first sentence in the sorting order to the current sentence to be processed.

[0090] In the specific implementation of step S23, all tokens from the first sentence in the sorting order to the current sentence to be processed are obtained within the preset context range, which can be expressed as , ,... , where L represents the number of tokens currently generated; then, for each token within the preset context , the semantic vector is calculated through the last layer of the large speech model Transformer hidden Hidden States , the semantic vector is Model( ); Then, based on the semantic vector of each token Substitute into formula (1) to calculate the corresponding average vector and use it as the target vector of the current sentence to be processed .

[0091] Formula (1):

[0092]

[0093] Step S24: For each token of the sentence to be processed , calculate the similarity between the token and the target vector.

[0094] In the specific implementation of step S24, for each token of the sentence to be processed, , the token And the target vector of the sentence currently to be processed Substitute into formula (2) to calculate the similarity between the token and the target vector .

[0095] Formula (2):

[0096]

[0097] Based on this, the target vector corresponding to each sentence in the long text and the similarity between all tokens in each sentence and the corresponding target vector can be calculated through the above steps S21 to S24.

[0098] Step S25: For each token, calculate the importance score of the token based on the token, the target vector of the sentence corresponding to the token, and the similarity between the token and the target vector.

[0099] In the specific implementation of step S25, the final importance score of each token depends not only on its similarity with the target semantics, but also on its relationship with other tokens in the current context. Based on this, for each token, the token , the target vector of the sentence corresponding to the token , and the similarity between the token and the target vector Substitute into formula (3) to calculate and get the token Importance score.

[0100] Formula (3):

[0101]

[0102] in, For the token The importance score of is the similarity between the token and the target vector, Representation token and tokens The attention score between tokens Importance in the current context, combining semantic relevance and attention information; The variable window size is dynamically selected by calculating the contribution of the tokens within a preset context range of the sentence currently to be processed to generating the target vector.

[0103] Optionally, a variable window size is dynamically selected by calculating the contribution of the tokens within the preset context of the sentence to be processed to the generation of the target vector. ,The window size is related to factors such as the complexity of the current generated target and the richness of the context.

[0104] The size of the window will be adjusted according to the changes in the context range to avoid information loss caused by a fixed window.

[0105] Step S103: for each sentence, based on the importance score of the token corresponding to the sentence and the dynamic budget corresponding to the sentence, filter out the key tokens in the sentence and store them in the reserved set of the corresponding sentence;

[0106] It should be noted that the specific implementation of step S103 includes the following steps.

[0107] Step S31: Select tokens based on the importance scores of the tokens and the sentence weights of the sentences to construct an initial retention set.

[0108] The sentence weight is the average of the importance scores of each token corresponding to the sentence.

[0109] In the specific implementation of step S31, first, for each sentence, the mean importance score of each token corresponding to the sentence is calculated, and the mean importance score is used as the sentence importance weight of the sentence; then, it is determined whether the sentence weight of the sentence is greater than a first threshold, and if so, the tokens corresponding to the sentences greater than the first threshold are retained in the initial retention set; then, the importance score of each token of other sentences is determined, and the tokens greater than the second threshold are retained in the initial retention set.

[0110] It should be noted that the first threshold and the second threshold are both set in advance by technical personnel based on actual conditions.

[0111] Optionally, in the preliminary retained set, a diversity penalty mechanism is introduced based on the semantic similarity between tokens to reduce the importance scores of semantically redundant tokens and further optimize the semantic dispersion of the retained token set.

[0112] Step S32: Optimizing the importance score of each token based on the initial retention set and the importance score of the token to obtain a target importance score for each token.

[0113] In order to avoid excessive concentration on local context (such as a conversation or a narrative) and loss of overall context coverage, a diversity penalty term is designed to select tokens from different semantic fragments; in the specific implementation of step S32, for each token i in each sentence, its semantic similarity with the token j in the initial retention set is calculated. ; Semantic similarity and tokens Substitute into formula (4) to calculate and correct the importance score to obtain the target importance score , which is the final importance score of the token.

[0114] Formula (4):

[0115]

[0116] Among them, λ is the preset penalty coefficient, Selected is the selected token set, that is, the initial retention set, For token The importance score of For token The target importance score.

[0117] It should be noted that semantic similarity A similarity matrix , which measures the semantic similarity between tokens i and j.

[0118] Step S33: For each sentence, processing is performed based on the target importance score of each token in the sentence and the tokens of the sentence to obtain the sentence importance weight of the sentence.

[0119] In the specific implementation of step S33, for each sentence, all tokens of the sentence and the target importance score of each token in the sentence are substituted into formula (5) to calculate the sentence importance weight of the sentence. .

[0120] Formula (5):

[0121]

[0122] in, is the set of all tokens contained in sentence s, is the i-th corresponding token The final importance score of .

[0123] Step S34: Determine the dynamic budget allocated to each of the sentences based on the sentence importance weight of each sentence and a preset total dynamic budget τ.

[0124] In the specific implementation of step S34, when reserving tokens, the number of tokens reserved in different parts is allowed to be dynamically adjusted according to the actual complexity of the context. Specifically, for each sentence, the preset total budget τ and the sentence importance weight corresponding to the sentence are set. Substitute into formula (6) to calculate and get the dynamic budget of the sentence , and then get the dynamic budget of each sentence .

[0125] Formula (6):

[0126]

[0127] in, is the budget allocated to sentence s, is the sentence importance weight of sentence s, and τ is the preset total budget τ.

[0128] Optionally, formula (6) can be used to appropriately increase the retention budget for long sentences and important sentences (such as paragraphs with high scores); and to compress the budget for short sentences and redundant content.

[0129] Step S35: For each sentence, based on the sentence and its corresponding dynamic budget, filter out key tokens from the sentence and write them into the corresponding reserved set.

[0130] It should be noted that the number of tokens screened out in a sentence can be one or more.

[0131] In the specific implementation of step S35, for each sentence s, in its dynamic budget Under the restriction, the τ tokens with the highest scores and reasonable distribution in the sentence are screened out and used as key tokens, and then the key tokens are retained in the retention set corresponding to the sentence.

[0132] Optionally, the set is kept as GPU memory and the remaining tokens of the sentence are transferred to the CPU or discarded.

[0133] Step S104: for each sentence's retention set, calculate a local weighted attention score for each token based on all key tokens in the retention set;

[0134] In the specific implementation of step S104, for the retained set of each sentence, first, the attention value between the key tokens of the retained set is calculated; then the attention value between the key tokens of the retained set is substituted into formula (7) for calculation to obtain the local weighted attention score of each token , that is, for all retained tokens in each sentence s, first calculate the local weighted attention score of each token , Used to measure the contribution of token x to the entire sentence under the attention head h.

[0135] Formula (7):

[0136]

[0137] in, is the attention value of token x and token y under attention head h, x is a key token in the retained set , y is another key token in the retained set ; To retain the number of key tokens contained in the set .

[0138] Step S105: performing weighted fusion based on the local weighted attention score of each token in the sentence and the fusion weight of the preset attention head to obtain a sentence-level semantic vector;

[0139] It should be noted that the specific implementation of step S105 includes the following steps.

[0140] Step S41: For each attention head in the model, weighted fusion of each key vector based on the local weighted attention score of each token in the sentence is performed to obtain the sentence vector under each attention head, where each key vector refers to the key vector of each token under the attention head;

[0141] In the specific implementation of step S41, first, for each sentence s, assume that the retained set of the sentence contains tokens, and the key vector of token x is , where h is an attention head; within each attention head h, the local weighted attention score for each token is calculated based on And the key vectors under the sentence Substitute into formula (8) to get the sentence vector under the attention head , and then get the sentence vector under each attention head.

[0142] Formula (8):

[0143]

[0144] Step S42: Perform weighted fusion on the sentence vector under each attention head and the fusion weight of the preset attention head to obtain the sentence-level semantic vector of the sentence.

[0145] In the specific implementation of step S42, the sentence vector under each attention head is And the fusion weight of the preset attention head Formula (9) is input for calculation to obtain the sentence-level semantic vector of the sentence.

[0146] It should be noted that one attention head corresponds to one fusion weight, and the fusion weight corresponding to the hth attention head is , h belongs to H.

[0147] Among them, the number of attention heads is H.

[0148] Formula (9):

[0149]

[0150] in, is the fusion weight corresponding to the h-th attention head.

[0151] In this embodiment of the present invention, for a sentence s, first, in each attention head h, a local normalized weight is calculated based on the attention scores between all tokens in the sentence, i.e., the local weighted attention score , and the key vector for each token Perform weighting to generate a single-head sentence, i.e., a sentence vector Then, based on the heterogeneous features of different attention heads, learnable fusion weights are used Sentence vectors for all attention heads Fusion is performed to finally form a unified sentence-level semantic vector Based on the tokens in the retained set, a sentence-level semantic vector is generated through a multi-scale, hierarchical attention weighting mechanism. .

[0152] Step S106: constructing a semantic index cache and a key-value pair KV content cache based on the sentence-level semantic vector of the sentence and the corresponding retention set.

[0153] The specific implementation of step S106 includes the following steps.

[0154] Step S51: forming a corresponding semantic index based on the sentence-level semantic vector of the sentence and caching it.

[0155] Specifically, extract the sentence-level semantic vector of each sentence ; The sentence-level semantic vector Stored in GPU high-speed memory to form a semantic index cache for each sentence.

[0156] It should be noted that for each sentence, additional structural feature information is further recorded, including but not limited to: sentence length normalization value, token importance mean, and sentence position information.

[0157] The semantic index cache is used for retrieval operations based on multi-factor matching in the decoding stage.

[0158] Step S52: establishing a key-value pair KV content cache based on the tokens in the reserved set.

[0159] In the specific implementation of step S52, the key tokens that are filtered and retained in the retention set of each sentence are saved, and their corresponding Key-Value pairs are saved; the above Key-Value pairs are stored in the CPU memory or low-frequency persistent storage medium, that is, the key-value pair KV content cache; to support subsequent retrieval and loading of Key-Value pairs at the sentence level or clause level granularity.

[0160] The present invention implements the LLM pre-population phase through steps S101 to S106. This phase occurs after the LLM receives the complete input long text but before it begins generating the first output token. This phase primarily processes the input long text to determine sentence-level semantic vectors for each sentence in the long text, the key-value pairs (KVs) for each sentence in the retained token set, and constructs a semantic index cache and a key-value pair (KV) content cache.

[0161] In an embodiment of the present invention, during the caching phase, the input long text is first semantically parsed and segmented using sentences as the basic unit, and the importance score of the tokens of each sentence is calculated; then, the key tokens corresponding to each sentence are dynamically screened in combination with the context-adaptive weight, semantic similarity, and attention mechanism, and stored in the retention set of the corresponding sentence; on this basis, a sentence-level semantic vector is constructed, and a sentence-level semantic vector is generated with the help of a multi-scale attention mechanism, and then a corresponding semantic index cache and KV content cache are constructed to uniformly store the sentence-level semantic vectors of all sentences in a semantic index structure in the GPU memory, as a basis for rapid semantic retrieval in the subsequent decoding phase. At the same time, the corresponding Key / Value is recorded for each key token in the retention set, and these data structures are uniformly organized into cache units divided by sentences, providing a fine-grained context loading basis for the decoding phase.

[0162] Optionally, after the pre-filling phase of steps S101 to S105, the LLM enters the decoding phase. In this phase, the LLM generates output tokens one by one in an autoregressive manner. Each token generated is added to the generated sequence and serves as input for the next step.

[0163] Among them, the decoding phase starts from the last token of the long text processed in the pre-filling phase, and the goal is to generate a subsequent output sequence, which includes outputting multiple tokens.

[0164] It should be noted that if it is the first step of the decoding phase, the last token of the long text processed in the pre-filling phase is used as the input of the current Transformer layer of the LLM; if it is not the first step of the decoding phase, the token generated in the previous step is used as the input of the current Transformer layer.

[0165] Based on the management method based on sentence semantic perception shown in the above embodiment of the present invention, the present invention also shows a flow chart of another management method based on sentence semantic perception, such as Figure 2 As shown, the method includes:

[0166] Step S201: If it is determined that the current stage is the decoding stage, the currently generated content is obtained, and the currently generated content is processed to generate a corresponding joint score.

[0167] In the specific implementation of step S201, if it is determined that the current generated content of a certain attention mechanism of LLM is in the decoding stage, the target vector corresponding to the current generated content and the corresponding sentence-level semantic vector are input into formula (10) for processing to generate the corresponding joint score .

[0168] Formula (10):

[0169]

[0170] in, Including sentence length normalization value, position information, token importance density, etc.; and is the coefficient, which is preset. is the target vector corresponding to the content, is the corresponding sentence-level semantic vector.

[0171] It should be noted that the target vector corresponding to the current generated content ,as well as It can be obtained through the process of the above steps S101 to S105.

[0172] The currently generated content is generally the token output by the current Transformer layer of the LLM.

[0173] Step S202: using the joint score to search from the semantic index cache and the key-value pair KV content cache to load the candidate sentence set and key-value pair KV corresponding to the currently generated content into the corresponding memory.

[0174] It should be noted that the self-attention mechanism within LLM can still access the semantic index cache and key-value pair KV content cache.

[0175] It should be noted that the specific implementation of step S202 includes the following steps.

[0176] Step S61: Retrieving a set of candidate sentences from the semantic index cache according to the joint score;

[0177] In the process of specifically implementing step S61, the Top-K most relevant sentences are retrieved and filtered from the semantic index cache according to the joint score as candidate loading targets. That is, the Top-K, i.e., a preset number of most relevant sentence-level semantic vectors, that are the same or closest to the joint score are retrieved and filtered from the semantic index cache, and the corresponding sentences are used as candidate loading targets, i.e., a set of candidate sentences.

[0178] Step S62: traverse the key-value pair KV content cache according to the identifier of each sentence in the candidate sentence set to determine the corresponding key-value pair KV.

[0179] In the specific implementation of step S62, each sentence is associated with its sentence number or unique identification ID; the identification of each sentence in the candidate sentence set is used to quickly locate it in the KV content cache (CPU memory or disk) to obtain the storage offset and index information of its retained Key-Value pair.

[0180] Step S63: Store the candidate sentence set and key-value pairs KV in the corresponding memory.

[0181] In the specific implementation of step S63 , the KV and candidate sentence set are selectively loaded from the CPU to an independent memory unit in the GPU based on the joint score, so that the next generated content in the LLM decoding stage can be directly called.

[0182] Optionally, in addition to loading the candidate sentence set shown above, the cache loading range can also be divided according to the granularity of clauses, phrases, keyword paragraphs, etc.

[0183] Among them, the clause division method includes but is not limited to punctuation marks (commas, semicolons, etc.), and the grammatical analysis includes token attention clustering.

[0184] Optionally, to improve the management efficiency of KV cache in GPU memory, a multi-dimensional state-aware dynamic cache control mechanism is introduced.

[0185] Specifically, it includes: a soft eviction strategy, which is used to prioritize the release of cache units with low heat and long periods of inaccessibility;

[0186] Dynamic quota adjustment flexibly adjusts the cache resident capacity based on the remaining GPU memory; and an asynchronous pre-loading mechanism loads potentially related sentences into the pre-buffer in advance by predicting future generated semantic vectors, thereby achieving a low-latency, high-semantic consistency generation process.

[0187] Optionally, in order to ensure decoding efficiency and semantic consistency, the present application further includes the following steps after executing step S202.

[0188] Step S71: Determine the contextual semantic information required for currently generated content.

[0189] In the specific implementation of step S71, during each round of content generation, the query vectors of all tokens in the current generation window are collected. If a query vector is determined to be a sentence boundary (such as a period, question mark, or carriage return), the sentence-level query semantic vector is calculated. , as shown in formula (11).

[0190] Formula (11):

[0191]

[0192] in, Indicates the semantic target of the current generated sentence, which is used for subsequent matching operations.

[0193] Next, based on the semantic index cache saved in the GPU (containing the semantic vector of each sentence ), by calculating The cosine similarity of the sentence-level semantic vector of each sentence in the semantic index cache is obtained from the semantic index cache, and the preset number of Top-K sentence cache units with the largest cosine similarity, that is, the most relevant, that is, the contextual semantic information required for the current content generation.

[0194] It should be noted that, in order to improve matching accuracy, LLM can be combined with other structural features (such as sentence length, number of retained tokens, etc.) to perform multi-factor sorting and weighted matching.

[0195] The Top-K sentence cache unit includes the corresponding sentence.

[0196] In the present invention, in order to improve the matching accuracy, other structural features (such as sentence length, number of retained tokens, etc.) can be combined to perform multi-factor ranking and weighted matching.

[0197] Step S72: Based on the contextual semantic information required for the currently generated content, relevant Key-Value pairs are loaded from the cache to form a lightweight semantic support set that supports the current generation.

[0198] During the implementation of step S72, some or all key-value pairs from the Top-K sentence cache are loaded from CPU memory. This process supports key-value loading at the clause or token subset level. The loaded cache units are added to the active state table and participate in the attention calculation during the current generation process. Furthermore, soft eviction, dynamic retention, and asynchronous pre-loading of the GPU cache are performed to ensure that the optimal set of sentences is always available in GPU memory, thereby improving the response speed and semantic accuracy of the generation.

[0199] Optionally, also include:

[0200] During the generation phase, LLM performs efficient attention calculations based on the token query vector of the currently generated content and the key-value pairs loaded in the GPU to generate the next token and automatically add it to the cache of the currently generated content.

[0201] Whenever a complete sentence is generated, the semantic vector of the sentence is recalculated, and the matching basis of the subsequent query stage is updated based on the sentence-level query semantic vector, thereby improving the contextual consistency of the generated content.

[0202] Optionally, LLM continuously records the matching relationship between sentences and cache units during each generation process, building a dynamic semantic matching feedback matrix. This matrix is used to optimize the retrieval accuracy and matching efficiency of subsequent sentences in real time, ensuring that the generated content is highly relevant to the cached information.

[0203] Optionally, during the long generation process, known as the decoding phase, to improve generation efficiency and memory utilization, historical context is semantically compressed, converting low-frequency sentences into summary representations. LLM also employs a soft freeze strategy to ensure that certain high-value semantic sentences remain in GPU memory, maximizing semantic coherence and optimizing memory efficiency.

[0204] Optionally, also include:

[0205] During each generation process of the decoding phase, LLM records the association strength between the candidate sentence cache unit and the token actually called by the attention mechanism to form a triple:

[0206] < , ,call frequency>

[0207] These triplets provide basic data for the construction of the subsequent feedback matrix. LLM constructs a trainable semantic matching matrix based on this data. (where d is the vector dimension). The matrix parameters are optimized by minimizing the following loss function, as shown in formula (12).

[0208] Formula (12):

[0209] +

[0210] in, is the matching error, is a regularization term to prevent overfitting. The optimized matrix M is used to enhance the semantic alignment capability of subsequent retrieval, improving the accuracy and efficiency of retrieval.

[0211] Optionally, LLM dynamically evaluates the popularity of cached sentences by access frequency and maintains a popularity score The heat update follows time decay, as shown in formula (13).

[0212] Formula (13):

[0213]

[0214] in, is the decay coefficient, and I (call count) is the indicator function, representing the number of times the cached sentence is called. Through this formula, the heat score can flexibly reflect the frequency of sentence cache usage and continuously adjust according to time decay.

[0215] LLM divides cached sentences into three categories based on their popularity scores: hot data, which are frequently accessed sentences, are retained in GPU memory to ensure efficient access. Warm data, which are moderately frequently accessed sentences, are stored in CPU memory. Cold data, which are infrequently accessed sentences, are stored on disk.

[0216] In this embodiment of the present invention, during the decoding phase, the relevant key-value pairs (KVs) are loaded from the cache on demand based on the semantic vector of the currently generated content, forming a lightweight semantic support set. This is coupled with a dynamic eviction strategy and asynchronous preloading mechanism to achieve efficient GPU cache management. This enables semantically driven, fine-grained KV cache control, significantly improving performance and context retention during long text processing.

[0217] Corresponding to the data management method based on sentence semantic perception shown in the above embodiment of the present invention, a data management device based on sentence semantic perception is also shown. Figure 3 As shown, the device includes:

[0218] The segmentation unit 301 is configured to segment the long text carried in the text cache instruction upon receiving the text cache instruction input by the user, to obtain a plurality of sentences and their sorting order;

[0219] A calculation unit 302 is configured to calculate, for each sentence, an importance score of a token corresponding to the sentence based on the sentence and its sort order;

[0220] A screening unit 303 is configured to screen out key tokens in each sentence based on the importance scores of the tokens corresponding to the sentence and the dynamic budget corresponding to the sentence, and store the key tokens in the retained set of the corresponding sentence;

[0221] Processing unit 304 is configured to calculate, for each sentence, a local weighted attention score of each token based on all key tokens in the retained set; perform weighted fusion based on the local weighted attention score of each token in the sentence and the fusion weight of the preset attention head to obtain a sentence-level semantic vector;

[0222] The cache unit 305 is configured to construct a semantic index cache and a key-value pair (KV) content cache based on the sentence-level semantic vector of the sentence and the corresponding retention set.

[0223] The specific principles and execution processes of each unit in the data management device based on sentence semantic perception disclosed in the above embodiment of the present invention are the same as the corresponding contents in the data management method based on sentence semantic perception provided in the above embodiment of the present invention. Please refer to the corresponding parts of the data management method based on sentence semantic perception disclosed in the above embodiment of the present invention, and no further details will be given here.

[0224] In an embodiment of the present invention, during the caching phase, the input long text is first semantically parsed and segmented using sentences as the basic unit, and the importance score of the token for each sentence is calculated. Then, the key tokens corresponding to each sentence are dynamically screened in combination with the context-adaptive weight, semantic similarity, and attention mechanism, and stored in the retention set of the corresponding sentence. On this basis, a sentence-level semantic vector is constructed, and a sentence-level semantic vector is generated with the help of a multi-scale attention mechanism, and then a corresponding semantic index cache and KV content cache are constructed to uniformly store the sentence-level semantic vectors of all sentences in a semantic index structure in the GPU memory, as a basis for rapid semantic retrieval in the subsequent decoding phase. At the same time, the corresponding Key Value is recorded for each key token in the retention set. These data structures are uniformly organized into cache units divided by sentences, providing a fine-grained context loading basis for the decoding phase.

[0225] Optionally, based on the data management device based on sentence semantic perception shown in the above embodiment of the present invention, the segmentation unit 301 includes:

[0226] Preprocessing the long text carried by the text cache instruction to obtain a preprocessed long text;

[0227] If the pre-processed long text contains a preset symbol, dividing the long text into a plurality of sentences and sorting them according to the preset end symbol;

[0228] If the preprocessed long text does not contain preset symbols, the preset segmentation model is called to segment the long text to obtain multiple sentences and their sorting order, wherein the preset segmentation model is trained based on historical long texts and corresponding sentences.

[0229] Optionally, based on the data management device based on sentence semantic perception shown in the above embodiment of the present invention, the computing unit 302 includes:

[0230] Determining a current sentence to be processed based on the plurality of sentences and their sorting order;

[0231] For the sentence to be processed, generating a corresponding token based on features in the sentence, where the number of the token is at least one;

[0232] Calculating a target vector for the sentence to be processed based on the tokens within a preset context and the semantic vectors of the tokens;

[0233] For each token of the sentence to be processed, calculate the similarity between the token and the target vector;

[0234] For each token, an importance score of the token is calculated based on the token, the target vector of the sentence to which the token corresponds, and the similarity between the token and the target vector.

[0235] Optionally, based on the data management device based on sentence semantic perception shown in the above embodiment of the present invention, the screening unit 303 includes:

[0236] Selecting tokens based on the importance scores of the tokens and sentence weights of the sentences to construct an initial retention set, wherein the sentence weights are the average of the importance scores of each token corresponding to the sentence;

[0237] Optimizing the importance score of each token based on the initial retention set and the importance score of the token to obtain a target importance score for each token;

[0238] For each sentence, processing is performed based on the target importance score of each token in the sentence and the tokens of the sentence to obtain a sentence importance weight of the sentence;

[0239] Determining a dynamic budget allocated to each of the sentences based on the sentence importance weight of each of the sentences and a preset total dynamic budget;

[0240] For each sentence, based on the sentence and its corresponding dynamic budget, key tokens are filtered out from the sentence and written into the corresponding reserved set.

[0241] Optionally, based on the data management device based on sentence semantic perception shown in the above embodiment of the present invention, a processing unit 304 for obtaining a sentence-level semantic vector by performing weighted fusion based on the local weighted attention score of each token in the sentence and the fusion weight of the preset attention head includes:

[0242] For each attention head in the model, weighted fusion of each key vector based on the local weighted attention score of each token in the sentence is performed to obtain the sentence vector under each attention head. The key vector refers to the key vector of each token under the attention head.

[0243] The sentence vector under each attention head and the fusion weight of the preset attention head are weightedly fused to obtain the sentence-level semantic vector of the sentence.

[0244] Optionally, the data management device based on sentence semantic perception shown in the above embodiment of the present invention further includes:

[0245] A generating unit, configured to obtain currently generated content and process the currently generated content to generate a corresponding joint score if it is determined that the current phase is a decoding phase;

[0246] The retrieval unit uses the joint score to search from the semantic index cache and the key-value pair KV content cache to load the candidate sentence set and key-value pair KV corresponding to the currently generated content into the corresponding memory.

[0247] The method of using the joint score to search from the semantic index cache and the key-value pair KV content cache to load the candidate sentence set and the key-value pair KV corresponding to the currently generated content into the corresponding memory includes:

[0248] Retrieving a set of candidate sentences from a semantic index cache according to the joint score;

[0249] Traversing the key-value pair KV content cache according to the identifier of each sentence in the candidate sentence set to determine the corresponding key-value pair KV;

[0250] The candidate sentence set and key-value pairs KV are stored in the corresponding memory.

[0251] An embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory is used to store data management program code and data based on sentence semantic perception, and the processor is used to call program instructions in the memory to execute and implement the data management method based on sentence semantic perception in the above embodiment.

[0252] An embodiment of the present invention provides a storage medium, which includes the electronic device provided by the above-mentioned embodiment of the present application, and the electronic device is used to execute the data management method based on sentence semantic perception disclosed in the embodiment of the present application.

[0253] Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the method embodiment. The system and system embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without expending creative work.

[0254] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0255] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data management method based on sentence semantic perception, characterized in that: The method comprises: If a text cache instruction input by the user is received, the long text carried by the text cache instruction is segmented to obtain multiple sentences and their sorting order; For each sentence, calculate the importance score of the token corresponding to the sentence based on the sentence and its sort order; For each sentence, based on the importance score of the token corresponding to the sentence and the dynamic budget corresponding to the sentence, the key tokens in the sentence are screened out and stored in the reserved set of the corresponding sentence; For each sentence's retention set, calculate a local weighted attention score for each token based on all key tokens in the retention set; Performing weighted fusion based on the local weighted attention score of each token in the sentence and the fusion weight of the preset attention head to obtain a sentence-level semantic vector; Building a semantic index cache and a key-value pair KV content cache based on the sentence-level semantic vector of the sentence and the corresponding retention set; For each sentence, the importance score of the token corresponding to the sentence is calculated based on the sentence and its sorting order, including: Determining a current sentence to be processed based on the plurality of sentences and their sorting order; For the sentence to be processed, generating a corresponding token based on features in the sentence, where the number of the token is at least one; Calculating a target vector for the sentence to be processed based on the tokens within a preset context and the semantic vectors of the tokens; For each token of the sentence to be processed, calculate the similarity between the token and the target vector; For each token, calculate an importance score for the token based on the token, a target vector of the sentence corresponding to the token, and a similarity between the token and the target vector; For each sentence, the key tokens in the sentence are screened out based on the importance score of the token corresponding to the sentence and the dynamic budget corresponding to the sentence, and are stored in the reserved set of the corresponding sentence, including: Selecting tokens based on the importance scores of the tokens and sentence weights of the sentences to construct an initial retention set, wherein the sentence weights are the average of the importance scores of each token corresponding to the sentence; Optimizing the importance score of each token based on the initial retention set and the importance score of the token to obtain a target importance score for each token; For each sentence, processing is performed based on the target importance score of each token in the sentence and the tokens of the sentence to obtain a sentence importance weight of the sentence; Determining a dynamic budget allocated to each of the sentences based on the sentence importance weight of each of the sentences and a preset total dynamic budget; For each sentence, based on the sentence and its corresponding dynamic budget, filter out key tokens from the sentence and write them into the corresponding reserved set; The sentence-level semantic vector is obtained by weighted fusion based on the local weighted attention score of each token in the sentence and the fusion weight of the preset attention head, including: For each attention head in the model, weighted fusion of each key vector based on the local weighted attention score of each token in the sentence is performed to obtain the sentence vector under each attention head. The key vector refers to the key vector of each token under the attention head. The sentence vector under each attention head and the fusion weight of the preset attention head are weightedly fused to obtain the sentence-level semantic vector of the sentence.

2. The method according to claim 1, characterized in that The long text carried by the text cache instruction is segmented to obtain multiple sentences and their sorting order, including: Preprocessing the long text carried by the text cache instruction to obtain a preprocessed long text; If the pre-processed long text contains a preset symbol, dividing the long text into a plurality of sentences and sorting them according to the preset end symbol; If the preprocessed long text does not contain preset symbols, the preset segmentation model is called to segment the long text to obtain multiple sentences and their sorting order, wherein the preset segmentation model is trained based on historical long texts and corresponding sentences.

3. The method according to claim 1, characterized in that Also includes: If it is determined that the current stage is decoding, obtaining the current generated content, and processing the current generated content to generate a corresponding joint score; The combined score is used to search from the semantic index cache and the key-value pair KV content cache to load the candidate sentence set and the key-value pair KV corresponding to the currently generated content into the corresponding memory.

4. The method according to claim 3, characterized in that Using the joint score to search from the semantic index cache and the key-value pair KV content cache to load the candidate sentence set and key-value pair KV corresponding to the currently generated content into the corresponding memory, including: Retrieving a set of candidate sentences from the semantic index cache according to the joint score; Traversing the key-value pair KV content cache according to the identifier of each sentence in the candidate sentence set to determine the corresponding key-value pair KV; The candidate sentence set and key-value pairs KV are stored in the corresponding memory.

5. A data management device based on sentence semantic perception, characterized in that: The device comprises: a segmentation unit configured to, upon receiving a text cache instruction input by a user, segment the long text carried in the text cache instruction to obtain a plurality of sentences and their sorting order; a calculation unit, configured to calculate, for each sentence, an importance score of a token corresponding to the sentence based on the sentence and its sorting order; a screening unit configured to screen out key tokens in each sentence according to the importance scores of the tokens corresponding to the sentence and the dynamic budget corresponding to the sentence, and store the key tokens in the retained set of the corresponding sentence; a processing unit configured to calculate, for a retained set of each sentence, a local weighted attention score of each token based on all key tokens in the retained set; and perform weighted fusion based on the local weighted attention score of each token in the sentence and a fusion weight of a preset attention head to obtain a sentence-level semantic vector; A cache unit, configured to construct a semantic index cache and a key-value pair (KV) content cache based on the sentence-level semantic vector of the sentence and the corresponding retention set; The computing unit is specifically configured to: Determining a current sentence to be processed based on the plurality of sentences and their sorting order; For the sentence to be processed, generating a corresponding token based on features in the sentence, where the number of the token is at least one; Calculating a target vector for the sentence to be processed based on the tokens within a preset context and the semantic vectors of the tokens; For each token of the sentence to be processed, calculate the similarity between the token and the target vector; For each token, calculate an importance score for the token based on the token, a target vector of the sentence corresponding to the token, and a similarity between the token and the target vector; Wherein, the screening unit is specifically used for: Selecting tokens based on the importance scores of the tokens and sentence weights of the sentences to construct an initial retention set, wherein the sentence weights are the average of the importance scores of each token corresponding to the sentence; Optimizing the importance score of each token based on the initial retention set and the importance score of the token to obtain a target importance score for each token; For each sentence, processing is performed based on the target importance score of each token in the sentence and the tokens of the sentence to obtain a sentence importance weight of the sentence; Determining a dynamic budget allocated to each of the sentences based on the sentence importance weight of each of the sentences and a preset total dynamic budget; For each sentence, based on the sentence and its corresponding dynamic budget, filter out key tokens from the sentence and write them into the corresponding reserved set; The processing unit is specifically configured to: For each attention head in the model, weighted fusion of each key vector based on the local weighted attention score of each token in the sentence is performed to obtain the sentence vector under each attention head. The key vector refers to the key vector of each token under the attention head. The sentence vector under each attention head and the fusion weight of the preset attention head are weightedly fused to obtain the sentence-level semantic vector of the sentence.

6. An electronic device, characterized in that: The electronic device includes a processor and a memory, the memory is used to store program code and data for data management based on sentence semantic perception, and the processor is used to call the program instructions in the memory to execute the data management method based on sentence semantic perception as described in any one of claims 1-4.

7. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is running, the device where the storage medium is located is controlled to execute the data management method based on sentence semantic perception according to any one of claims 1 to 4.

Images